A Twin gradient method for unconstrained optimization

Abstract

We propose a new strategy for gradient-based unconstrained optimization, involving two parallel sequences of iterates that cooperate to determine their stepsizes via a \textit{Twin-Step} principle. Rather than minimizing the objective function individually, the algorithm selects steplengths that minimize the Euclidean distance between the two gradient based processes occurring simultaneously at each iteration. The theoretical analysis shows that the convergence of the mutual distance is governed by the angle between the search directions. In particular the effectiveness of the overall process degrades as the directions approach parallelism. To ensure robustness against collinearity, we introduce a hybrid framework, Twin-ABB$_{\min}$, which switches to the Adaptive Barzilai--Borwein method when the geometric cooperation becomes ineffective. Extensive and very promising numerical results evidence that the Twin phase creates favorable initial conditions for subsequent BB-type iterations.