arXiv subjects

Deren Han

Publications and source records attributed to Deren Han.

At least 19 recordsLinked to original sources

arXiv2026

Randomized iterative methods with Polyak step-size for solving generalized absolute value equations

In this paper, we systematically incorporate the Polyak step-size into the randomized iterative method to improve its efficiency for solving generalized absolute value equations. In particular, we adopt the Polyak step-size within a stochastic iterative setting where the objective function updates dynamically at every step, unlike the classical Polyak step-size designed for deterministic optimization with fixed objective functions. Consequently, this novel implementation differs from the conventional Polyak scheme and demands a dedicated convergence analysis. We rigorously analyze the convergence properties of the proposed method and establish its linear convergence in expectation. Numerical experiments demonstrate that the incorporation of the Polyak step-size substantially improves the computational performance of randomized iterative methods with constant step-sizes.

math.NA↗

arXiv2026

A Two-step Krasnosel'skii-Mann Algorithm with Adaptive Momentum and Its Applications to Image Denoising and Matrix Completion

In this paper, we propose a Two-step Krasnosel'skii-Mann (KM) Algorithm (TKMA) with adaptive momentum for solving convex optimization problems arising in image processing. Such optimization problems can often be reformulated as fixed-point problems for certain operators, which are then solved using iterative methods based on the same operator, including the KM iteration, to ultimately obtain the solution to the original optimization problem. Prior to developing TKMA, we first introduce a KM iteration enhanced with adaptive momentum, derived from geometric properties of an $α$-averaged nonexpansive operator T with $α\in(0,1)$, KM acceleration technique, and information from the composite operator $T^2$. The proposed TKMA is constructed as a convex combination of this adaptive-momentum KM iteration and the Picard iteration of $T^2$. We prove that the sequence generated by TKMA converges weakly to a fixed point of T in a real Hilbert space. Moreover, under $α\in(0,1/2]$ and specific assumptions on the adaptive momentum parameters, we prove that the algorithm achieves an $o\left(1/\sqrt{k}\right)$ convergence rate in terms of the distance between successive iterates. Numerical experiments demonstrate that TKMA outperforms the FPPA, PGA, Fast KM algorithm, and Halpern algorithm on tasks such as image denoising and low-rank matrix completion.

math.OC↗

arXiv2026

On subspace-constrained preconditioning for randomized iterative methods

In this paper, we further investigate and refine the subspace-constrained preconditioning technique to enhance the theoretical and numerical convergence properties of randomized iterative methods for solving linear systems. In particular, we design a QR-like factorization that transforms the original linear system into an equivalent block-orthogonal form, thus avoiding the full-rank assumptions adopted in existing work. Moreover, this reformulation reduces the problem to solving a smaller linear system with a favorable singular value distribution, provided an appropriate initial point is employed. The proposed framework can be implemented implicitly within the iteration and does not require explicitly constructing either a preconditioner matrix or a preconditioned linear system, which eliminates the prohibitive cost of forming a fully preconditioned system. Furthermore, we construct orthogonalized search directions from stochastic gradients and develop accelerated variants of the framework. We prove that the proposed algorithmic framework converges linearly in expectation. Numerical experiments demonstrate the benefits of the proposed preconditioning strategy.

math.NA↗

arXiv2026

Randomized conjugate gradient least squares

We develop a novel randomized conjugate gradient least squares (RCGLS) method for solving least-squares problems, in which iterative sketching is employed at each step to reduce the dimension and hence the computational cost. In particular, we propose a new perspective on the classical CGLS method, where the next descent direction is determined via a constraint correction problem associated with the gradient. Based on this insight, we replace the gradient with a randomized coordinate gradient that naturally satisfies the variance reduction property, leading directly to the proposed RCGLS method. We prove that RCGLS converges linearly in expectation, with a better convergence bound compared to the randomized coordinate descent method. Furthermore, we investigate an implementation of the method that avoids full-dimensional vector operations, which are the major bottleneck of vanilla RCGLS for sparse matrices and render it impractical. We also show how to apply the RCGLS method to solve the ridge regression problem, yielding a lightweight, parallelizable, and accelerated method for such problems. Numerical experiments are provided to confirm our results.

math.NA↗

arXiv2026

Linear convergence of Gearhart-Koshy accelerated Kaczmarz methods for tensor linear systems

The generalized Gearhart-Koshy acceleration is a recent exact affine search technique designed for the method of cyclic projections onto hyperplanes, i.e., the Kaczmarz method. However, its convergence properties, particularly the linear convergence rate, have not been thoroughly established. In this paper, we systematically establish the linear convergence of the generalized Gearhart-Koshy accelerated Kaczmarz method for tensor linear systems, proving that it converges linearly to the unique least-norm solution. Our analysis is general and applies to several popular Kaczmarz variants, including incremental, shuffle-once, and random-reshuffling schemes, and demonstrates that this acceleration approach yields a better convergence upper bound compared to the plain Kaczmarz method. We also propose an efficient Gram-Schmidt-based implementation that computes the next iterate in linear time. Building on this implementation, we establish a connection between this acceleration framework and Arnoldi-type Krylov subspace methods, further highlighting its efficiency and potential. Our theoretical results are supported by numerical experiments.

math.NA↗

arXiv2026

NS-RGS: Newton-Schulz based Riemannian gradient method for orthogonal group synchronization

Group synchronization is a fundamental task involving the recovery of group elements from pairwise measurements. For orthogonal group synchronization, the most common approach reformulates the problem as a constrained nonconvex optimization and solves it using projection-based methods, such as the generalized power method. However, these methods rely on exact SVD or QR decompositions in each iteration, which are computationally expensive and become a bottleneck for large-scale problems. In this paper, we propose a Newton-Schulz-based Riemannian Gradient Scheme (NS-RGS) for orthogonal group synchronization that significantly reduces computational cost by replacing the SVD or QR step with the Newton-Schulz iteration. This approach leverages efficient matrix multiplications and aligns perfectly with modern GPU/TPU architectures. By employing a refined leave-one-out analysis, we overcome the challenge arising from statistical dependencies, and establish that NS-RGS with spectral initialization achieves linear convergence to the target solution up to near-optimal statistical noise levels. Experiments on synthetic data and real-world global alignment tasks demonstrate that NS-RGS attains accuracy comparable to state-of-the-art methods such as the generalized power method, while achieving nearly a 2$\times$ speedup.

stat.ML↗

arXiv2026

Parallelizable Riemannian Alternating Direction Method of Multipliers for Non-convex Pose Graph Optimization

Pose graph optimization (PGO) is fundamental to robot perception and navigation systems, serving as the mathematical backbone for solving simultaneous localization and mapping (SLAM). Existing solvers suffer from polynomial growth in computational complexity with graph size, hindering real-time deployment in large-scale scenarios. In this paper, by duplicating variables and introducing equality constraints, we reformulate the problem and propose a Parallelizable Riemannian Alternating Direction Method of Multipliers (PRADMM) to solve it efficiently. Compared with the state-of-the-art methods that usually exhibit polynomial time complexity growth with graph size, PRADMM enables efficient parallel computation across vertices regardless of graph size. Crucially, all subproblems admit closed-form solutions, ensuring PRADMM maintains exceptionally stable performance. Furthermore, by carefully exploiting the structures of the coefficient matrices in the constraints, we establish the global convergence of PRADMM under mild conditions, enabling larger relaxation step sizes within the interval $(0,2)$. Extensive empirical validation on two synthetic datasets and multiple real-world 3D SLAM benchmarks confirms the superior computational performance of PRADMM.

math.OC↗

arXiv2026

Stochastic dual coordinate descent with adaptive heavy ball momentum for linearly constrained convex optimization

The problem of finding a solution to the linear system $Ax = b$ with certain minimization properties arises in numerous scientific and engineering areas. In the era of big data, the stochastic optimization algorithms become increasingly significant due to their scalability for problems of unprecedented size. This paper focuses on the problem of minimizing a strongly convex function subject to linear constraints. We consider the dual formulation of this problem and adopt the stochastic coordinate descent to solve it. The proposed algorithmic framework, called adaptive stochastic dual coordinate descent, utilizes sampling matrices sampled from user-defined distributions to extract gradient information. Moreover, it employs Polyak's heavy ball momentum acceleration with adaptive parameters learned through iterations, overcoming the limitation of the heavy ball momentum method that it requires prior knowledge of certain parameters, such as the singular values of a matrix. With these extensions, the framework is able to recover many well-known methods in the context, including the randomized sparse Kaczmarz method, the randomized regularized Kaczmarz method, the linearized Bregman iteration, and a variant of the conjugate gradient (CG) method. Additionally, we introduce an equivalent formulation that, in certain cases, substantially reduces the need for full-dimensional vector operations introduced by the momentum term. We prove that, with strongly admissible objective function, the proposed method converges linearly in expectation. Numerical experiments are provided to confirm our results.

math.NA↗

arXiv2025

Research on the descent direction of prediction correction algorithms for pseudo-convex/convex optimization problems

Prediction-correction algorithms are a highly effective class of methods for solving pseudo-convex optimization problems. The descent direction of these algorithms can be viewed as an adjustment to the gradient direction based on the prediction step. This paper investigates the adjustment coefficients of these descent directions and offers explanations from the perspective of differential equations. Unlike existing algorithms where the adjustment coefficient is always set to 1, we establish that the range of the adjustment coefficient lies within (1/2,1] for pseudo-convex optimization problems, and [0,1] for convex optimization problems. We also provide rigorous convergence proofs for these proposed algorithms. Numerical experiment results show that the algorithms perform best when the value of the adjustment coefficient makes the algorithm approach or equal to those in differential equations with higher-order global discrete error.

math.OC↗

arXiv2025

An inverse-free fixed-time stable dynamical system and its forward-Euler discretization for solving generalized absolute value equations

An inverse-free dynamical system is proposed to solve the generalized absolute value equation (GAVE) with a fixed time convergence, where the time of convergence is finite and is uniformly bounded for all initial points. Moreover, an iterative method obtained by using the forward-Euler discretization of the proposed dynamic model is developed and sufficient conditions which guarantee that the discrete iteration globally converge to an arbitrarily small neighborhood of the unique solution of GAVE within a finite number of iterative steps are given. Numerical results illustrate the effectiveness of the proposed methods.

math.NA↗

arXiv2025

Approximating Quantum States with Positive Partial Transposes in Multipartite System via Linearized Proximal Alternative Direction Method of Multipliers

Numerical approximation of quantum states via convex combinations of states with positive partial transposes (bi-PPT state) in multipartite systems constitutes a fundamental challenge in quantum information science. We reformulate this problem as a linearly constrained optimization problem. An approximate model is constructed through an auxiliary variable and a suitable penalty parameter, balancing constraint violation and approximation error. To slove the approximate model, we design a Linearized Proximal Alternating Direction Method of Multipliers (LPADMM), proving its convergence under a prescribed inequality condition on regularization parameters. The algorithm achieves an iteration complexity of $O(1/ε^2)$ for attaining $ε$-stationary solutions. Numerical validation on diverse quantum systems, including three-qubit W/GHZ states and five-partite GHZ and multiGHZ states with noises, confirms high-quality bi-PPT approximations and decomposability certification, demonstrating the utility of our method for quantum information applications.

math-ph↗

arXiv2025

On adaptive stochastic extended iterative methods for solving least squares

In this paper, we propose a novel adaptive stochastic extended iterative method, which can be viewed as an improved extension of the randomized extended Kaczmarz (REK) method, for finding the unique minimum Euclidean norm least-squares solution of a given linear system. In particular, we introduce three equivalent stochastic reformulations of the linear least-squares problem: stochastic unconstrained and constrained optimization problems, and the stochastic multiobjective optimization problem. We then alternately employ the adaptive variants of the stochastic heavy ball momentum (SHBM) method, which utilize iterative information to update the parameters, to solve the stochastic reformulations. We prove that our method converges $R$-linearly in expectation, addressing an open problem in the literature related to designing theoretically supported adaptive SHBM methods. Numerical experiments show that our adaptive stochastic extended iterative method has strong advantages over the non-adaptive one.

math.NA↗

arXiv2025

A simple linear convergence analysis of the randomized reshuffling Kaczmarz method

The random reshuffling Kaczmarz (RRK) method enjoys the simplicity and efficiency in solving linear systems as a Kaczmarz-type method, whereas it also inherits the practical improvements of the stochastic gradient descent (SGD) with random reshuffling (RR) over original SGD. However, the current studies on RRK do not characterize its convergence comprehensively. In this paper, we present a novel analysis of the RRK method and prove its linear convergence towards the unique least-norm solution of the linear system. Furthermore, the convergence upper bound is tight and does not depend on the dimension of the coefficient matrix.

math.NA↗

arXiv2025

A fixed-time stable dynamical model for solving EVLCPs

A fixed-time stable dynamical system for solving the extended vertical linear complementarity problem (EVLCP) is developed. The system is based on the reformulation of EVLCP as a special case of a new kind of generalized absolute value equations. Some properties of the new kind of generalized absolute value equations are explored which are useful for developing a fixed-time stable dynamical system for solving it. Without using any smoothing technique, we develop a dynamical system for solving the new kind of generalized absolute value equations and prove its fixed-time stability. The model is applicable for solving EVLCP. As two by-products, a new condition which guarantees the unique solvability of EVLCP and a new error bound of EVLCP are given. Numerical results are given to demonstrate our claims.

math.NA↗

arXiv2025

The Condition Number in Phase Retrieval from Intensity Measurements

This paper investigates the stability of phase retrieval by analyzing the condition number of the nonlinear map $Ψ_{\boldsymbol{A}}(\boldsymbol{x}) = \bigl(\lvert \langle {\boldsymbol{a}}_j, \boldsymbol{x} \rangle \rvert^2 \bigr)_{1 \le j \le m}$, where $\boldsymbol{a}_j \in \mathbb{H}^n$ are known sensing vectors with $\mathbb{H} \in \{\mathbb{R}, \mathbb{C}\}$. For each $p \ge 1$, we define the condition number $β_{Ψ_{\boldsymbol{A}}}^{\ell_p}$ as the ratio of optimal upper and lower Lipschitz constants of $Ψ_{\boldsymbol{A}}$ measured in the $\ell_p$ norm, with respect to the metric $\mathrm {dist}_\mathbb{H}\left(\boldsymbol{x}, \boldsymbol{y}\right) = \|\boldsymbol{x} \boldsymbol{x}^\ast - \boldsymbol{y} \boldsymbol{y}^\ast\|_*$. We establish universal lower bounds on $β_{Ψ_{\boldsymbol{A}}}^{\ell_p}$ for any sensing matrix $\boldsymbol{A} \in \mathbb{H}^{m \times d}$, proving that $β_{Ψ_{\boldsymbol{A}}}^{\ell_1} \ge π/2$ and $β_{Ψ_{\boldsymbol{A}}}^{\ell_2} \ge \sqrt{3}$ in the real case $(\mathbb{H} = \mathbb{R})$, and $β_{Ψ_{\boldsymbol{A}}}^{\ell_p} \ge 2$ for $p=1,2$ in the complex case $(\mathbb{H} = \mathbb{C})$. These bounds are shown to be asymptotically tight: both a deterministic harmonic frame $\boldsymbol{E}_m \in \mathbb{R}^{m \times 2}$ and Gaussian random matrices $\boldsymbol{A} \in \mathbb{H}^{m \times d}$ asymptotically attain them. Notably, the harmonic frame $\boldsymbol{E}_m \in \mathbb{R}^{m \times 2}$ achieves the optimal lower bound $\sqrt{3}$ for all $m \ge 3$ when $p=2$, thus serving as an optimal sensing matrix within $\boldsymbol{A} \in \mathbb{R}^{m \times 2}$. Our results provide the first explicit uniform lower bounds on $β_{Ψ_{\boldsymbol{A}}}^{\ell_p}$ and offer insights into the fundamental stability limits of phase retrieval.

cs.IT↗

arXiv2025

Enhanced randomized Douglas-Rachford method: Improved probabilities and adaptive momentum

Randomized iterative methods have gained recent interest in machine learning and signal processing for solving large-scale linear systems. One such example is the randomized Douglas-Rachford (RDR) method, which updates the iterate by reflecting it through two randomly selected hyperplanes and taking a convex combination with the current point. In this work, we enhance RDR by introducing improved sampling strategies and an adaptive heavy-ball momentum scheme. Specifically, we incorporate without-replacement and volume sampling into RDR, and establish stronger convergence guarantees compared to conventional i.i.d. sampling. Furthermore, we develop an adaptive momentum mechanism that dynamically adjusts step sizes and momentum parameters based on previous iterates, and prove that the resulting method achieves linear convergence in expectation with improved convergence bounds. Numerical experiments demonstrate that the enhanced RDR method consistently outperforms the original version, providing substantial practical benefits across a range of problem settings.

math.NA↗

arXiv2025

Connecting randomized iterative methods with Krylov subspaces

Randomized iterative methods, such as the randomized Kaczmarz method, have gained significant attention for solving large-scale linear systems due to their simplicity and efficiency. Meanwhile, Krylov subspace methods have emerged as a powerful class of algorithms, known for their robust theoretical foundations and rapid convergence properties. Despite the individual successes of these two paradigms, their underlying connection has remained largely unexplored. In this paper, we develop a unified framework that bridges randomized iterative methods and Krylov subspace techniques, supported by both rigorous theoretical analysis and practical implementation. The core idea is to formulate each iteration as an adaptively weighted linear combination of the sketched normal vector and previous iterates, with the weights optimally determined via a projection-based mechanism. This formulation not only reveals how subspace techniques can enhance the efficiency of randomized iterative methods, but also enables the design of a new class of iterative-sketching-based Krylov subspace algorithms. We prove that our method converges linearly in expectation and validate our findings with numerical experiments.

math.NA↗

arXiv2025

Inexact Regularized Quasi-Newton Algorithm for Solving Monotone Variational Inequality Problems

Newton's method has been an important approach for solving variational inequalities, quasi-Newton method is a good alternative choice to save computational cost. In this paper, we propose a new method for solving monotone variational inequalities where we introduce a merit function based on the merit function. With the help of the merit function, we can locally accepts unit step size. And a globalization technique based on the hyperplane is applied to the method. The proposed method applied to monotone variational inequality problems is globally convergent in the sense that subproblems always have unique solutions, and the whole sequence of iterates converges to a solution of the problem without any regularity assumptions. We also provide extensive numerical results to demonstrate the efficiency of the proposed algorithm.

math.OC↗