Convergence of machine learning methods for feedback control laws: averaged feedback learning scheme and data driven methods

Abstract

This work addresses the synthesis of optimal feedback control laws via machine learning. In particular, the Averaged Feedback Learning Scheme (AFLS) and a data driven method are considered. Hypotheses for each method ensuring the convergence of the evaluation of the objective function of the underlying control problem at the obtained feedback-laws towards the optimal value function are provided. These hypotheses are connected to the regularity of the value function and the stability of the dynamics. In the case of AFLS these hypotheses only require Hölder continuity of the value function, whereas for the data driven method the value function must be at least $C^2$. It is demonstrated that these methods are connected via their optimality conditions. Additionally, numerical experiments are provided by applying both methods to a family control problems, parameterized by a positive real number which controls the regularity of the value function. For small parameters the value function is smooth and in contrast for large parameters it is non-differentiable, but semi-concave. The results of the experiments indicate that both methods have a similar performance for the case that the value function is smooth. On the other hand, if the value function is not differentiable, AFLS has a better performance which is consistent with the obtained convergence results.