A Backstepping Framework for Unconstrained Accelerated Optimization Algorithms

Abstract

This paper introduces a control-theoretic perspective on unconstrained optimization algorithms using the backstepping methods. We model the optimization process as an augmented strict-feedback system given by $\dot{x}_1 = x_2$, $\dot{x}_2 = u$, and $\dot{z} = q(x_1,z)$, with a regulated output $y = \nabla f(x_1)$. This formulation recasts the development of unconstrained optimization algorithms as a feedback control problem, where the goal is to design the input $u$ to ensure $y(t) \to 0$. By employing backstepping, we recursively synthesize the actual feedback law $u$ after initially selecting a virtual control for $x_1$. For convex objective functions, we develop a general synthesis framework for augmented strict-feedback systems and specialize it to the standard strict-feedback case. This unified framework successfully recovers the constant-parameter Nesterov flow and the proportional-integral-derivative (PID) accelerated optimizer as direct corollaries. We further establish that, given a fixed virtual control, the universal second-step law is inverse optimal with respect to an induced outer-tracking problem. This reveals that the optimality of the control law is conditionally dependent on the target manifold prescribed by the virtual control, rather than holding globally across all possible backstepping designs. Finally, we formulate a formal optimal-backstepping theorem that elevates this optimality principle to the virtual-control stage by solving a reduced Hamilton--Jacobi--Bellman problem. These contributions collectively yield a robust and general backstepping-driven paradigm for the analysis and design of continuous-time unconstrained optimization algorithms.