Adaptive Momentum via Minimal Dual Function for Accelerating Randomized Sparse Kaczmarz

Abstract

Recently, the randomized sparse Kaczmarz method has been accelerated by designing heavy ball momentum adaptively via a minimal-error principle. In this paper, we develop a new adaptive momentum method based on the minimal dual function principle to go beyond the exact measurement restriction of the minimal-error principle. Moreover, by integrating the new adaptive momentum method with the quantile-based sampling, we introduce a general algorithmic framework, called quantile-based randomized sparse Kaczmarz with minimal dual function momentum, which provides a unified approach to exact, noisy, or corrupted linear systems. In addition, we utilize the discrepancy principle and monotone error as stopping rules for the proposed algorithm. Theoretically, we establish linear convergence in expectation of Bregman distance up to a finite horizon related to the contaminated level. At last, we provide numerical illustrations on simulated and real-world data to demonstrate the effectiveness of our proposed method.