Acceleration for Affine-coupling problems

Abstract

Saddle-point problems play an important role in modern machine learning, including robust optimization and algorithmic fairness. We investigate structured convex--concave minimax problems where the primal variable lies in a high-dimensional, unconstrained space $\mathbb{R}^d$, while the dual variable is confined to a constrained, low-dimensional space $\mathbb{R}^J$, with $J\ll d$. Although existing theoretical frameworks establish that an accelerated rate of $\mathcal{O}(L/T^2)$ is possible, solving the associated subproblems efficiently can become a computational bottleneck in high dimensions. To address this, we introduce a decoupling technique. By restricting second-order updates to the low-dimensional dual space and employing first-order methods in the high-dimensional primal space, our framework combines Newton methods with Nesterov acceleration. For simplex-constrained dual variables and regularizers compatible with a self-concordant barrier, the resulting algorithm achieves a convergence rate of $\mathcal{O}(L/T^2)$ in primal objective suboptimality after $T$ outer iterations, without strong dual concavity. The additional linear-algebra cost per outer iteration is $\widetilde{\mathcal{O}}(dJ^2+J^{3.5})$, alongside one evaluation of the component losses and their Jacobian.