Reinforcement Learning Generalization for Nonlinear Systems Through Dual-Scale Homogeneity Transformations

Abstract

Reinforcement learning is an emerging approach to control dynamical systems for which classical approaches are difficult to apply. However, trained agents may not generalize against the variations of system parameters. This paper presents the concept of dual-scale homogeneity, an important property in understating the scaling behavior of nonlinear systems. Furthermore, it also presents an effective yet simple approach to designing a parameter-dependent control law that homogenizes a nonlinear system. The presented approach is applied to two systems, demonstrating its ability to provide a consistent performance irrespective of parameters variations. To demonstrate the practicality of the proposed approach, the control policy is generated by a deep deterministic policy gradient to control the load position of a quadrotor with a slung load. The proposed synergy between the homogeneity transformations and reinforcement learning yields superior performance compared to other recent learning-based control techniques. It achieves a success rate of 96% in bringing the load to its designated target with a 3D RMSE of 0.0253 m. The video that shows the experimental results along with a summary of the paper is available at this link.