Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.12110

Adaptive dynamic programming using Lyapunov function constraints

Thomas Göhrt·Pavel Osinenko·Stefan Streif
View original source ↗Read source document ↗Export metadata ↓

Abstract

This work is concerned with a stabilizing adaptive dynamic programming (ADP) approach to approximate solution of a given infinite-horizon optimal control problem. Since the latter problem cannot, in general, be solved exactly, a parametrized function approximator for the infinite-horizon cost function is introduced in ADP (so called ``critic''). This critic is used to adapt the parameters of the function approximator. The so called ``actor'' in turn derives the optimal input of the system. It is a notoriously hard problem to guarantee closed-loop stability of ADP due to the use of approximation structures in the control scheme. Since at least stabilizability is always assumed in the analyses of ADP, it is justified to invoke a respective Lyapunov function. The proposed ADP scheme explicitly uses the said Lyapunov function to simultaneously optimize the critic and guarantee closed-loop stability. A Hessian-free optimization routine is utilized for the actor and critic optimization problems. Convergence to prescribed vicinities of the optima is shown. A computational study showed significant performance improvement for the critic-based approach compared a nominal stabilizing controller for a range of initial conditions.

Explore related subjects

math.OCcs.SYeess.SYmath.DS

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Thomas Göhrt, Pavel Osinenko, Stefan Streif. 2026-10-08. Adaptive dynamic programming using Lyapunov function constraints. https://doi.org/10.1109/lcsys.2019.2919439

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Perturbed Iterate SGD for Lipschitz Continuous Loss Functions with Numerical Error and Adaptive Step Sizes

Motivated by neural network training in finite-precision arithmetic environments, this work studies the convergence of perturbed iterate SGD using adaptive step sizes in an environment with numerical error. Considering a general stochastic Lipschitz continuous loss function, an asymptotic convergence result to a Clarke stationary point is proven as well as the non-asymptotic convergence to an approximate stationary point in expectation. It is assumed that only an approximation of the loss function's stochastic gradient can be computed, in addition to error in computing the SGD step itself.

math.OC↗
arXiv2026

Sharp bounds in perturbed smooth optimization

This paper studies the problem of perturbed convex and smooth optimization. The main results describe how the solution and the value of the problem change if the objective function is perturbed. Examples include linear, quadratic, and smooth additive perturbations. Such problems naturally arise in statistics and machine learning, stochastic optimization, stability and robustness analysis, inverse problems, optimal control, etc. The results provide accurate expansions for the difference between the solution of the original problem and its perturbed counterpart with an explicit error term.

math.OC↗
arXiv2026

Controllability Allocation Scores for Targeted Network Intervention

We introduce the controllability allocation score (CAS), a framework for determining how intervention intensity should be distributed among prescribed candidate input directions with respect to designated target variables, together with the target controllability score (TCS) as its nodewise specialization. We establish existence of the CASs and develop a general uniqueness theory based on restricted injectivity of the allocation-to-Gramian map, including generic uniqueness with respect to the time horizon. We show that restricting attention to target variables can fundamentally alter the optimal intervention allocation compared with the standard full-state setting. To enable scalability, we develop a general surrogate-Gramian framework and derive objective-performance guarantees from relative Gramian errors without requiring uniqueness or closeness of the CASs. For the TCS specialization, we further construct a target-only reduced virtual system and derive explicit bounds showing how the approximation error depends on the coupling between target and non-target nodes and on the dynamical growth rate. Experiments on human brain networks show that the reduced formulation accurately approximates the TCS at short horizons, whereas the two controllability criteria exhibit markedly different approximation accuracy at long horizons.

math.OC↗