Reinforcement-learning-assisted control of four-roll mills: geometric symmetry and inertial effect

Abstract

Embedding the intrinsic symmetry of a flow system in training its machine learning algorithms has become a significant trend in the recent surge of their application in fluid mechanics. This paper leverages the geometric symmetry of a four-roll mill (FRM) to enhance its training efficiency. Stabilizing and precisely controlling droplet trajectories in a FRM is challenging due to the unstable nature of the extensional flow with a saddle point. Extending the work of Vona & Lauga, this study applies Deep Reinforcement Learning (DRL) to effectively guide a displaced droplet to the center of the FRM. Through direct numerical simulations, we explore the applicability of DRL in controlling FRM flow with moderate inertial effects, i.e., Reynolds number $\sim\mathcal{O}(1)$, a nonlinear regime previously unexplored. The FRM's geometric symmetry allows control policies trained in one of the eight sub-quadrants to be extended to the entire domain, reducing training costs. Our results indicate that the DRL-based control method can successfully guide a displaced droplet to the target center with robust performance across various starting positions, even from substantially far distances. The work also highlights potential directions for future research, particularly focusing on efficiently addressing the delay effects in flow response caused by inertia. This study presents new advances in controlling droplet trajectories in more nonlinear and complex situations, with potential applications to other nonlinear flows. The geometric symmetry used in this cutting-edge reinforcement learning approach can also be applied to other control methods.