CaCuTe Meets SESOP: Better Theory and Practice with Lipschitz Hessians

Abstract

We combine CaCuTe with sequential subspace optimization (SESOP) for smooth convex minimization under Lipschitz continuity of the Hessian. The resulting method, CaCuSESOP, achieves the global rate $O(k^{-2})$ with a fixed number of Hessian--vector products per iteration, while removing key limitations of classical SESOP. In particular, exact minimization of the original objective is replaced by minimization of a low-dimensional cubic upper model, which needs only be solved to a computable accuracy. The cubic regularization parameter is selected by backtracking. If the Hessian at the minimizer is positive definite, CaCuSESOP automatically attains an accelerated local linear rate with complexity $O(\sqrt{κ_\star}\log(1/\varepsilon))$. Neither the local strong-convexity parameter, the condition number $κ_\star$, nor the optimal value is supplied to the algorithm. We give finite bounds for entering this regime and adapting the restart schedule. Finally, we prove that the global $O(k^{-2})$ rate is optimal for the considered fixed-budget gradient/Hessian--vector-product linear-span oracle class, even with momentum and unrestricted memory.