Anderson Acceleration in Nonsmooth Problems: Local Convergence via Active Manifold Identification

Kexin Li; Luwei Bai; Xiao Wang; Hao Wang

Home/Archive/arXiv

arXiv · 2410.09420

Anderson Acceleration in Nonsmooth Problems: Local Convergence via Active Manifold Identification

Kexin Li·Luwei Bai·Xiao Wang·Hao Wang

View original source ↗Read source document ↗Export metadata ↓

Abstract

Anderson acceleration is an effective technique for enhancing the efficiency of fixed-point iterations; however, analyzing its convergence in nonsmooth settings presents significant challenges. In this paper, we investigate a class of nonsmooth optimization algorithms characterized by the active manifold identification property. This class includes a diverse array of methods such as the proximal point method, proximal gradient method, proximal linear method, proximal coordinate descent method, Douglas-Rachford splitting (or the alternating direction method of multipliers), and the iteratively reweighted $\ell_1$ method, among others. Under the assumption that the optimization problem possesses an active manifold at a stationary point, we establish a local R-linear convergence rate for the Anderson-accelerated algorithm. Our extensive numerical experiments further highlight the robust performance of the proposed Anderson-accelerated methods.

Explore related subjects

math.OC cs.LG cs.NA math.NA

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Kexin Li, Luwei Bai, Xiao Wang, Hao Wang. 2024-10-15. Anderson Acceleration in Nonsmooth Problems: Local Convergence via Active Manifold Identification. https://arxiv.org/abs/2410.09420

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Last-Iterate Performance of Gradient Descent and Relaxed Proximal Point via $s$-Composability

The notion of s-composability was introduced to compose optimized stepsize schedules while preserving sharp guarantees. We show that the same joint potential has a second use: it can serve directly as a certificate of sharp last-iterate performance. We develop this viewpoint for constant and silver schedules. For the constant schedule, we prove the s-composability statement posed as an open question by Grimmer et al. (2025), at every finite horizon, through an explicit nonnegative smooth-convex interpolation certificate. The result identifies the unique constant stepsize minimizing the worst-case final gradient norm of smooth convex gradient descent under an initial-distance bound, thereby proving the optimal stepsize, value, and uniqueness predicted by Conjecture 3 of Taylor et al. (2017) without resolving its full worst-case curve. Through the Moreau envelope, the same constant is also the unique constant relaxation minimizing the worst-case final residual of relaxed proximal point. More generally, for every positive s-composable schedule, we determine the exact worst-case final proximal objective-gap constant and give a matching one-dimensional example. Applying this result to the already s-composable original silver schedule yields its exact last-iterate objective-gap constant, closing a question left open by Wang et al. (2025).

math.OC↗

arXiv2026

Complexity of Output Feedback Stabilization

We show that unless P = NP, there cannot be a polynomial-time (or even pseudo-polynomial-time) algorithm for output feedback stabilization of a linear dynamical system with a linear controller. This settles one of the best-known open problems in control theory. The result holds in both continuous and discrete time. We also present a family of stabilizable linear dynamical systems for which no polynomial-time algorithm can write down a stabilizing controller in its standard representation.

math.OC↗

arXiv2026

On Control of Drawdown: Robust Invariance and Optimality

Mitigating \emph{drawdown}, the decline in wealth from its running peak, presents a canonical problem in path-dependent risk control. In this paper, we develop a finite-horizon control framework that enforces a prescribed maximum percentage drawdown limit in multi-asset stochastic systems. Our first result is an exact robust-invariance theorem characterizing every control action that preserves a prescribed drawdown limit against all supported returns. We show that every robustly safe control admits a \emph{drawdown-modulated} form: the product of the current drawdown \emph{cushion} and a feasible \emph{normalized direction}. This yields a complete parameterization of robustly drawdown-safe policies. Additionally, under stagewise-independent returns, we show that optimizing over all robustly safe causal policies reduces to a one-dimensional Bellman recursion and yields an optimal robustly safe state-feedback policy. Finally, we characterize the linear time-invariant (LTI) gains satisfying a prescribed drawdown limit and prove that optimal drawdown modulation achieves no lower expected return under the same limit. Strict expected-return improvement holds for horizons of at least two stages whenever the LTI policy has positive expected one-stage net return.

math.OC↗