Variance-Reduced Stochastic Quasi-Newton SQCQP Method for Nonlinear Programming Problems

Abstract

In this paper, we propose a variance-reduced stochastic quasi-Newton sequential quadratic constrained quadratic programming method (VR-QN-SQCQP) for finite-sum nonlinear optimization with equality and inequality constraints. Exact nonlinear-constraint Hessians are replaced entirely by safeguarded symmetric rank-one (SR1) curvature models built from first-order constraint gradients, while PAGE estimates the objective gradient and an exact \(L_1\) merit function provides globalization. Under explicit regularity, boundedness, relative QCQP-accuracy, and step-length assumptions, the analysis establishes almost-sure subsequential convergence to Karush--Kuhn--Tucker (KKT) points and, under stronger uniform step-length and error-control conditions, an expected \(O(\varepsilon^{-2})\) outer-iteration bound for the squared KKT residual. A conditional local stability result also compares learned-curvature directions with their exact-Hessian counterparts. Numerical experiments compare VR-QN-SQCQP against Exact-SQCQP-PAGE on Sparse PCA using 253,680 CDC BRFSS 2015 observations and on a dense nonlinear benchmark. Over ten seeds, both methods converge in all Sparse PCA runs, while on the dense benchmark the proposed method converges in 10/10 runs versus 9/10 for the exact-Hessian variant. The proposed method performs zero exact constraint-Hessian evaluations and reduces median curvature-construction time on the dense benchmark from 0.1230 s to 0.0111 s. Overall runtime benefits remain problem-dependent.