Adaptive First-Order Methods with Composite Gradient Error

Abstract

We study first-order optimization with inexact gradients whose errors combine a component proportional to the true gradient norm and a uniformly bounded absolute component. We develop an adaptive gradient descent scheme that selects estimates of the smoothness constant and both error parameters through a computable backtracking condition using exact function values. Its analysis yields convergence for smooth objectives and functions satisfying Polyak-Lojasiewicz condition. For strongly convex objectives, we propose a composite-error intermediate gradient method, that adapts to unknown smoothness while using known error bounds and a strong convexity parameter $μ$. With fixed smoothness $L$, acceleration is retained when $α=\mathcal{O}\left(\left(μ/ L \right)^{1/4}\right)$, larger admissible errors yield intermediate rates. Throughout the range $0 \leqslant α\leqslant 1/3$, the absolute error bound $δ$ contributes at most $\mathcal{O} \left(δ^2/μ\right)$ to the objective residual, without additional amplification by the condition number.