Stabilizing Rayleigh-Benard convection with reinforcement learning trained on a reduced-order model

Abstract

Rayleigh-Benard convection (RBC) is a canonical system for buoyancy-driven turbulence and heat transport, central to geophysical and industrial flows. Developing efficient control strategies remains challenging at high Rayleigh numbers, where fully resolved simulations are computationally expensive. We use a control framework that couples data-driven manifold dynamics (DManD) with reinforcement learning (RL) to suppress convective heat transfer. We find a coordinate transformation to a low-dimensional system using POD and autoencoders, and then learn an evolution equation for this low-dimensional state using neural ODEs. The reduced model reproduces key system features while enabling rapid policy training. Policies trained in the DManD environment and deployed in DNS achieve a 16-23 % reduction in the Nusselt number for both single- and dual-boundary actuation. Physically, the learned strategy modulates near-wall heat flux to stabilize and thicken the thermal boundary layer, weaken plume ejection, and damp the wall-driven instabilities that seed convective bursts. Crucially, the controller drives the flow toward a quasi-steady state characterized by suppressed temporal fluctuations and spatially steady heat-flux patterns. This work establishes DManD-RL as a physically interpretable, scalable approach for turbulence control in high-dimensional flows.