Envelope Cubic Regularized Newton Method for Nonconvex-Strongly-Concave Minimax Optimization

Abstract

We study nonconvex-strongly-concave minimax problem $\min_{\mathbf{x}\in\mathbb{R}^{d_x}}\max_{\mathbf{y}\in\mathbb{R}^{d_y}}f(\mathbf{x},\mathbf{y})$ in which the objective has $ρ$-Lipschitz Hessian and condition number $κ$. We propose envelope cubic regularized Newton method (ECRN) which applies a refined inexact cubic regularized Newton method to the envelope of the primal function $Φ(\cdot) : = \max_{\mathbf{y}\in\mathbb{R}^{d_y}}f(\cdot,\mathbf{y})$. We show that ECRN finds approximate second-order stationary points of $Φ(\cdot)$ with $\widetilde{\mathcal{O}}(\sqrtρ\,κ^{3/4}\varepsilon^{-3/2})$ second-order oracles, improving the condition number dependence $\mathcal{O}(κ^{2})$ achieved by the best known second-order methods. Our improvement comes from the key observation that the Hessian Lipschitz parameter of the envelope is $\mathcal{O}(ρκ^{3/2})$, which is sharper than the order of $\mathcal{O}(ρκ^{3})$ for the function $Φ(\cdot)$. We also conduct a lower bound on second-order methods to show that our complexity is optimal up to a logarithmic factor.