Applying reinforcement learning to optical cavity locking tasks: considerations on actor-critic architectures and real-time hardware implementation

Abstract

This proceedings contains our considerations made during and after fruitful discussions held at EuCAIFCon 2025. We explore the use of deep reinforcement learning for autonomous locking of Fabry-Perot optical cavities in non-linear regimes, with relevance to gravitational-wave detectors. A custom Gymnasium environment with a time-domain simulator enabled training of agents such as deep deterministic policy gradient, achieving reliable lock acquisition for both low- and high-finesse cavities, including Virgo-like parameters. We also discuss possible improvements with Twin Delayed DDPG, Soft Actor Critic and meta-reinforcement learning, as well as strategies for low-latency execution and off-line policy updates to address hardware limitations. These studies lay the groundwork for future deployment of reinforcement learning-based control in real optical setups.