RPLSS: A randomized projected linear systems solver

Abstract

The projected linear system solver (PLSS), by incrementally appending columns to a random or deterministic sketching matrix, provides an attractive finite termination property for consistent linear systems. Nevertheless, a critical computational bottleneck of PLSS is accessing the whole coefficient matrix per iteration, making it prohibitive for extremely large-scale problems or applications with missing data. To alleviate this limitation, we propose a unified randomized PLSS (RPLSS) framework, built upon the tailored randomized row or column selection strategies that require only partial matrix information per iteration, for solving a general linear system, whether it is under- or overdetermined, and whether it is consistent or not. Within this framework, we develop a randomized Gaussian Kaczmarz method and its extended variant as row-action solvers, and randomized coordinate descent variants as column-action solvers. Theoretically, we prove that our methods inherit the finite termination property of PLSS, while achieving an exponential convergence rate, overcoming the sluggish convergence inherent in conventional randomized Kaczmarz and coordinate descent methods. Numerical experiments demonstrate the superiority of our method against state-of-the-art randomized methods, particularly in scenarios with large missing data.