Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2212.01014

Reinforcement-learning-based control of convectively-unstable flows

Da Xu·Mengqi Zhang
View original source ↗Read source document ↗Export metadata ↓

Abstract

This work reports the application of a model-free deep-reinforcement-learning-based (DRL) flow control strategy to suppress perturbations evolving in the 1-D linearised Kuramoto-Sivashinsky (KS) equation and 2-D boundary layer flows. The former is commonly used to model the disturbance developing in flat-plate boundary layer flows. These flow systems are convectively unstable, being able to amplify the upstream disturbance, and are thus difficult to control. The control action is implemented through a volumetric force at a fixed position and the control performance is evaluated by the reduction of perturbation amplitude downstream. We first demonstrate the effectiveness of the DRL-based control in the KS system subjected to a random upstream noise. The amplitude of perturbation monitored downstream is significantly reduced and the learnt policy is shown to be robust to both measurement and external noise. One of our focuses is to optimally place sensors in the DRL control using the gradient-free particle swarm optimisation algorithm. After the optimisation process for different numbers of sensors, a specific eight-sensor placement is found to yield the best control performance. The optimised sensor placement in the KS equation is applied directly to control 2-D Blasius boundary layer flows and can efficiently reduce the downstream perturbation energy. Via flow analyses, the control mechanism found by DRL is the opposition control. Besides, it is found that when the flow instability information is embedded in the reward function of DRL to penalise the instability, the control performance can be further improved in this convectively-unstable flow.

Explore related subjects

physics.flu-dyn

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Da Xu, Mengqi Zhang. 2022-12-02. Reinforcement-learning-based control of convectively-unstable flows. https://doi.org/10.1017/jfm.2022.1020

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Magnetoconvection in a spherical shell: Bridging the gap between weak and strong-field dynamo regimes

At moderate levels of convective driving, simulations of dynamos in spherical shells often separate into two broad dipolar regimes characterised either by their relative magnetic field strength (weak/strong) or by their dominant force balance. These regimes can tend smoothly from one to the other but can also be bistable, a phenomenon which occurs particularly at low magnetic diffusivity. Nonlinear simulations of the geodynamo cannot be performed at realistic parameters and hence it is important to ensure that the (strong-field) branch, appropriate for Earth's core, is tracked as a distinguished limit is established towards a suitable parameterisation from the simulations that we can perform. In order to understand the transition to strong-field dynamos, and better understand the mechanisms that occur in both branches, we report on a series of magnetoconvection simulations: models in which an external magnetic field modifies the dynamically-generated dynamo. Our primary input parameters are the magnetic diffusivity, imposed magnetic field strength, and the convective forcing; as these vary, we show how the morphology of the flow can change gradually, as modifications of an existing mode or continuous variations in force composition, and also abruptly, when onset modes become unstable to secondary instabilities and are replaced either by newly-dominant modes, a vacillating regime, or highly nonlinear configurations of the flow. The loss of equatorial symmetry is also investigated, as this typically coincides with the weak-to-strong transition, and we discuss why an anti-symmetric instability emerges inside the tangent cylinder along with its implications for the geodynamo.

physics.flu-dyn↗
arXiv2026

Space-time correlations of passive scalars in colored-noise flows

The space-time correlation of a passive scalar advected by a Gaussian colored-noise velocity with wavenumber-dependent correlation times and power-law spatial spectra is investigated in the present paper. Within the inertial-convective subrange, we derive an analytical solution for the space-time correlation. This solution validates the elliptic approximation (EA) model [He and Zhang, Phys. Rev. E 73, 055303(R) (2006)], demonstrating that the iso-correlation contours are self-similar in the co-moving space-time frame $(r-Uτ, Vτ)$, with a universal spatial-to-temporal intercept ratio of 1.55. Unlike the classic Kraichnan white-noise model, our formulation simultaneously recovers the Obukhov--Corrsin scaling for spatial correlations (when the velocity obeys Kolmogorov scaling) and reproduces the random-sweeping mechanism, yielding Gaussian (rather than exponential) temporal decorrelation of scalar Fourier modes. Our results clarify the underlying decorrelation mechanism of passive scalars: mean-flow advection and large-scale sweeping dominate temporal decorrelation, and small-scale distortion dominates spatial decorrelation.

physics.flu-dyn↗
arXiv2026

Timescale Separation Enables Deep Reinforcement Learning Control of Rotating Detonation Engine Mode Transitions

Rotating detonation engines (RDEs) are a promising propulsion concept that may offer higher thermodynamic efficiency and specific impulse than conventional systems, but nonlinear phenomena, including transitions to oscillatory or chaotic propagation modes, can hinder practical operation. Deep Reinforcement Learning (DRL) has emerged as a promising method for controlling complex nonlinear dynamics such as those observed in RDEs. However, the multi-timescale nature of the RDE system makes direct application of DRL challenging. We address this challenge by reformulating the DRL problem in a moving reference frame that follows the detonation-wave pattern, making the wave structure appear quasi-steady to the agent. This reformulation enables scale separation between fast detonation propagation and slower operating-mode dynamics. We train DRL controllers to modulate spatially segmented injection pressure in a one-dimensional reduced-order RDE model and induce rapid transitions between different mode-locked states. Across a range of actuation periods, initial states, and target modes, controllers trained in the moving frame learn more reliably than those trained in a stationary frame and remain effective over a broader range of actuation periods. These results suggest that symmetry-aware moving reference frame formulations may be useful for related multiscale flow-control problems and that scale separation should be exploited whenever possible to enable DRL control of multi-timescale systems.

physics.flu-dyn↗