arXiv subjects

Dong-Yue Xie

Publications and source records attributed to Dong-Yue Xie.

4 recordsLinked to original sources

arXiv2026

On randomized batch-sampling coordinate descent methods for solving linear least-squares problems

This paper refines randomized block coordinate descent by introducing a coordinate-level batch-sampling distribution characterization, yielding a fine-grained randomized batch-sampling coordinate descent framework, which applies to any block coordinate descent method under an arbitrary static stochastic sampling rule for solving large-scale linear least-squares problems. Based on this framework, we derive new linear convergence rate bounds that are invariant to the column scaling of data matrices. Theoretical analysis and numerical experiments validate that the new bounds are significantly sharper than existing results and better aligned with the practical convergence rate. Furthermore, we proposed a learning-based sampling strategy by parameterizing the batch-sampling distribution and optimizing its parameters through gradient-based training. Preliminary numerical results on computed tomography (CT) reconstruction problems demonstrate that the resulting method converges significantly faster than variants using other fixed sampling rules, and outperforms tested adaptive coordinate descent methods and full gradient descent in computational efficiency, confirming that learned sampling distributions can effectively capture problem-specific structure to accelerate convergence within the framework.

math.OC↗

arXiv2026

Sequential Preconditioned Conjugate Gradient Method for Linear Statistical Models

We propose a randomized iterative method for the ordinary least-squares estimation problem in large-scale linear statistical models, namely the Sequential Preconditioned Conjugate Gradient Method (SPCG). SPCG constructs a sequence of sketched least-squares subproblems with increasing sketch sizes, applies PCG as the inner solver, and warm-starts each subproblem from the previous solution. A final refinement stage is then performed on the full-scale problem. Since most iterations are carried out on smaller subproblems, the overall computational cost is significantly reduced. We establish the convergence theory, prove that SPCG attains OLS prediction accuracy, and derive per-subproblem iteration bounds and complexity estimates. Numerical experiments show that SPCG reaches the target prediction accuracy with fewer iterations and less CPU time than full-data PCG and Iterative Double Sketching (IDS).

math.NA↗

arXiv2026

Sequential Least-Squares Estimators with Fast Randomized Sketching for Linear Statistical Models

We propose a novel randomized framework for the estimation problem of large-scale linear statistical models, namely Sequential Least-Squares Estimators with Fast Randomized Sketching (SLSE-FRS), which integrates Sketch-and-Solve and Iterative-Sketching methods for the first time. By iteratively constructing and solving sketched least-squares (LS) subproblems with increasing sketch sizes to achieve better precisions, SLSE-FRS gradually refines the estimators of the true parameter vector, ultimately producing high-precision estimators. We analyze the convergence properties of SLSE-FRS, and provide its efficient implementation. Numerical experiments show that SLSE-FRS outperforms the state-of-the-art methods, namely the Preconditioned Conjugate Gradient (PCG) method, and the Iterative Double Sketching (IDS) method.

stat.ML↗

arXiv2026

Randomized batch-sampling Kaczmarz methods for solving linear systems

To conduct a more in-depth investigation of randomized solvers for solving linear systems, we adopt a unified randomized batch-sampling Kaczmarz framework with per-iteration costs as low as cyclic block methods, and develop a general analysis technique to establish its convergence guarantee. With concentration inequalities, we derive new expected linear convergence rate bounds. The analysis applies to any randomized non-extended block Kaczmarz methods with arbitrary static stochastic samplings. In addition, the new rate bounds are scale-invariant, which eliminate the dependence on the magnitude of the data matrix. In most experiments, the new bounds are significantly tighter than existing ones and better reflect the empirical convergence behavior of block methods. Within this new framework, the batch-sampling distribution, as a learnable parameter, provides the possibility for block methods to achieve efficient performance in specific application scenarios, which deserves further investigation.

math.NA↗