Autonomous Six-Degree-of-Freedom Spacecraft Docking Maneuvers via Reinforcement Learning

Abstract

A policy for six-degree-of-freedom docking maneuvers is developed through reinforcement learning and implemented as a feedback control law. Reinforcement learning provides a potential framework for robust, autonomous maneuvers in uncertain environments with low on-board computational cost. Specifically, proximal policy optimization is used to produce a docking policy that is valid over a portion of the six-degree-of-freedom state-space while striving to minimize performance and control costs. Experiments using the simulated Apollo transposition and docking maneuver exhibit the policy's capabilities and provide a comparison with standard optimal control techniques. Furthermore, specific challenges and work-arounds, as well as a discussion on the benefits and disadvantages of reinforcement learning for docking policies, are discussed to facilitate future research. As such, this work will serve as a foundation for further investigation of learning-based control laws for spacecraft proximity operations in uncertain environments.