Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2206.12061

On the Infimal Sub-differential Size of Primal-Dual Hybrid Gradient Method and Beyond

Haihao Lu·Jinwen Yang
View original source ↗Read source document ↗Export metadata ↓

Abstract

Primal-dual hybrid gradient method (PDHG, a.k.a. Chambolle and Pock method) is a well-studied algorithm for minimax optimization problems with a bilinear interaction term. Recently, PDHG is used as the base algorithm for a new LP solver PDLP that aims to solve large LP instances by taking advantage of modern computing resources, such as GPU and distributed system. Most of the previous convergence results of PDHG are either on duality gap or on distance to the optimal solution set, which are usually hard to compute during the solving process. In this paper, we propose a new progress metric for analyzing PDHG, which we dub infimal sub-differential size (IDS), by utilizing the geometry of PDHG iterates. IDS is a natural extension of the gradient norm of smooth problems to non-smooth problems, and it is tied with KKT error in the case of LP. Compared to traditional progress metrics for PDHG, IDS always has a finite value and can be computed only using information of the current solution. We show that IDS monotonically decays, and it has an $\mathcal O(\frac{1}{k})$ sublinear rate for solving convex-concave primal-dual problems, and it has a linear convergence rate if the problem further satisfies a regularity condition that is satisfied by applications such as linear programming, quadratic programming, TV-denoising model, etc. The simplicity of our analysis and the monotonic decay of IDS suggest that IDS is a natural progress metric to analyze PDHG. As a by-product of our analysis, we show that the primal-dual gap has $\mathcal O(\frac{1}{\sqrt{k}})$ convergence rate for the last iteration of PDHG for convex-concave problems. The analysis and results on PDHG can be directly generalized to other primal-dual algorithms, for example, proximal point method (PPM), alternating direction method of multipliers (ADMM) and linearized alternating direction method of multipliers (l-ADMM).

Explore related subjects

math.OC

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Haihao Lu, Jinwen Yang. 2023-03-11. On the Infimal Sub-differential Size of Primal-Dual Hybrid Gradient Method and Beyond. https://arxiv.org/abs/2206.12061

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

The Operational Impact of Registry size, Cycle length, and Blood Type Distribution in Multi-Registry Kidney Exchange Programs

Kidney exchange programs address donor recipient incompatibility by exchanging donors between incompatible pairs, but single center KEPs often suffer from limited donor pools, which reduce matching efficiency. Multi registry kidney exchange programs offer a promising solution but face challenges, including heterogeneous constraints across registries, cycle-length bounds, and data-sharing limitations. This study uses simulation to compare mKEP allocation against individual registry operation, contrasting unconstrained pooling with two safeguarded mechanisms: a cumulative individual-rationality guarantee and a Shapley value based fair-share mechanism. We examine how registry size, blood-type distribution, cycle-length bounds, and dropout probability affect the size and distribution of achievable gain. Our central finding is that the benefit of joining an mKEP is systematically uneven: under blood group composition asymmetry, the easier-to-match registry gains less than its partner even under both safeguarded mechanisms, and a larger registry gains less than a smaller one when pooled. A registry combining a larger arrival rate with an easier-to-match pool may see its gain under either safeguarded mechanism fall too small to be practically significant, while registries with a higher dropout rate see a larger benefit from pooling. This unevenness concentrates in O-type recipients: an easier-to-match registry sees fewer O-type transplants within itself under pooling, though O-type transplant rates rise system-wide. Tighter cycle-length bounds increase, rather than diminish, the relative transplant-volume benefit of pooling, while match quality is only modestly affected by any factor examined. These results highlight the importance of pairing multi-registry collaboration with a carefully designed, equitable benefit-sharing mechanism to keep participation attractive for all registries involved.

math.OC↗
arXiv2026

A polynomial approximation scheme for nonlinear model reduction by moment matching

We propose a procedure for the numerical approximation of invariance equations arising in the moment matching technique associated with reduced-order modeling of high-dimensional dynamical systems. The Galerkin residual method is employed to find an approximate solution to the invariance equation using a Newton iteration on the coefficients of a monomial basis expansion of the solution. These solutions to the invariance equations can then be used to construct reduced-order models. We assess the ability of the method to solve the invariance PDE system as well as to achieve moment matching and recover the steady-state behaviour of nonlinear systems with state dimension of order 1000 driven by linear and nonlinear signal generators.

math.OC↗
arXiv2026

Computationally Efficient Density-Driven Optimal Control via Analytical KKT Reduction and Contractive MPC

Efficient coordination for collective spatial distribution is a fundamental challenge in multi-agent systems. Prior research on Density-Driven Optimal Control (D2OC) established a framework to match agent trajectories to a desired spatial distribution. However, implementing this as a predictive controller requires solving a large-scale Karush-Kuhn-Tucker (KKT) system, whose computational complexity grows cubically with the prediction horizon. To resolve this, we propose an analytical structural reduction that transforms the T-horizon KKT system into a condensed quadratic program (QP). This formulation achieves O(T) linear scalability, significantly reducing the online computational burden compared to conventional O(T^3) approaches. Furthermore, to ensure rigorous convergence in dynamic environments, we incorporate a contractive Lyapunov constraint and prove the Input-to-State Stability (ISS) of the closed-loop system against reference propagation drift. Numerical simulations verify that the proposed method facilitates rapid density coverage with substantial computational speed-up, enabling long-horizon predictive control for large-scale multi-agent swarms.

math.OC↗