Model-Based Data-Efficient and Robust Reinforcement Learning

Abstract

A data-efficient learning-based control design method is proposed in this paper. It is based on learning a system dynamics model that is then leveraged in a two-level procedure. On the higher level, a simple but powerful optimization procedure is performed such that, for example, energy consumption in a vehicle can be reduced when hard state and action constraints are also introduced. Load disturbances and model errors are compensated for by a feedback controller on the lower level. In that regard, we briefly examine the robustness of both model-free and model-based learning approaches, and it is shown that the model-free approach greatly suffers from the inclusion of unmodeled dynamics. In evaluating the proposed method, it is assumed that a path is given, while the velocity and acceleration can be modified such that energy is saved, while still keeping speed limits and completion time. Compared with two well-known actor-critic reinforcement learning strategies, the suggested learning-based approach saves more energy and reduces the number of evaluated time steps by a factor of 100 or more.