Matching Upper and Lower Bounds for Higher-Order Nonconvex Finite-Sum Optimization

Abstract

We establish tight randomized higher-order oracle complexity for finding first-order stationary points of nonconvex finite sums. Let $n$ be the number of components, $Δ>0$ the initial objective-gap bound, $L_p>0$ an individual $p$-th derivative Lipschitz bound, and $ε>0$ the target gradient norm. For every fixed integer $p\ge 2$, the minimax number of exact component queries returning the value and all derivatives through order $p$, with success probability at least $2/3$, is \[ Θ_p\!\left( n+ΔL_p^{1/p}n^{1-1/(2p)} ε^{-(p+1)/p} \right), \] where the constants depend only on $p$ and the worst case ranges over all finite dimensions. The lower bound holds for unrestricted randomized adaptive algorithms and closes the $\sqrt{n}$ gap between the previously known general-order upper and lower bounds in their dependence on $n$. We extend dense weak hiding to complete higher-order replies while keeping each component's regularity independent of the chain length. The matching upper bound retains the known finite-sum exponent, requires only mean-squared $p$-th derivative increments, and removes the fixed-confidence logarithmic loss by verifying entire recursive-estimation epochs with exact function values. The characterization includes the additive $n$ term for every positive parameter regime; it counts oracle calls with unrestricted internal computation.