arXiv subjects

Lexing Ying

Publications and source records attributed to Lexing Ying.

At least 37 records · Page 2Linked to original sources

arXiv2025

Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithms

Discrete diffusion models have emerged as a powerful generative modeling framework for discrete data with successful applications spanning from text generation to image synthesis. However, their deployment faces challenges due to the high dimensionality of the state space, necessitating the development of efficient inference algorithms. Current inference approaches mainly fall into two categories: exact simulation and approximate methods such as $τ$-leaping. While exact methods suffer from unpredictable inference time and redundant function evaluations, $τ$-leaping is limited by its first-order accuracy. In this work, we advance the latter category by tailoring the first extension of high-order numerical inference schemes to discrete diffusion models, enabling larger step sizes while reducing error. We rigorously analyze the proposed schemes and establish the second-order accuracy of the $θ$-Trapezoidal method in KL divergence. Empirical evaluations on GSM8K-level math-reasoning, GPT-2-level text, and ImageNet-level image generation tasks demonstrate that our method achieves superior sample quality compared to existing approaches under equivalent computational constraints, with consistent performance gains across models ranging from 200M to 8B. Our code is available at https://github.com/yuchen-zhu-zyc/DiscreteFastSolver.

cs.LG↗

arXiv2025

A Randomized Method for Simulating Lindblad Equations and Thermal State Preparation

We study a qDRIFT-type randomized method to simulate Lindblad dynamics by decomposing its generator into an ensemble of Lindbladians, $\mathcal{L} = \sum_{a \in \mathcal{A}} \mathcal{L}_a$, where each $\mathcal{L}_a$ comprises a simple Hamiltonian and a single jump operator. Assuming an efficient quantum simulation is available for the Lindblad evolution $e^{t\mathcal{L}_a}$, we implement $e^{t\mathcal{L}_a}$ for a randomly sampled $\mathcal{L}_a$ at each time step according to a probability distribution $μ$ over the ensemble $\{\mathcal{L}_a\}_{a \in \mathcal{A}}$. This randomized strategy reduces the quantum cost of simulating Lindblad dynamics, particularly in quantum many-body systems with a large or even infinite number of jump operators. Our contributions are two-fold. First, we provide a detailed convergence analysis of the proposed randomized method, covering both average and typical algorithmic realizations. This analysis extends the known results for the random product formula from closed systems to open systems, ensuring rigorous performance guarantees. Second, based on the random product approximation, we derive a new quantum Gibbs sampler algorithm that utilizes jump operators sampled from a Clifford-random circuit. This generator (i) can be efficiently implemented using our randomized algorithm, and (ii) exhibits a spectral gap lower bound that depends on the spectrum of the Hamiltonian. Our results present a new instance of a class of Hamiltonians for which the thermal states can be efficiently prepared using a quantum Gibbs sampling algorithm.

quant-ph↗

arXiv2025

An efficient algorithm for entropic optimal transport under martingale-type constraints

This work introduces novel computational methods for entropic optimal transport (OT) problems under martingale-type conditions. The considered problems include the discrete martingale optimal transport (MOT) problem. Moreover, as the (super-)martingale conditions are equivalent to row-wise (in-)equality constraints on the coupling matrix, our work applies to a prevalent class of OT problems with structural constraints. Inspired by the recent empirical success of Sinkhorn-type algorithms, we propose an entropic formulation for the MOT problem and introduce Sinkhorn-type algorithms with sparse Newton iterations that utilize the (approximate) sparsity of the Hessian matrix of the dual objective. As exact martingale conditions are typically infeasible, we adopt entropic regularization to find an approximate constraint-satisfied solution. We show that, in practice, the proposed algorithms enjoy both super-exponential convergence and robustness with controllable thresholds for total constraint violations.

math.OC↗

arXiv2025

Variational inference and density estimation with non-negative tensor train

This work proposes an efficient numerical approach for compressing a high-dimensional discrete distribution function into a non-negative tensor train (NTT) format. The two settings we consider are variational inference and density estimation, whereby one has access to either the unnormalized analytic formula of the distribution or the samples generated from the distribution. In particular, the compression is done through a two-stage approach. In the first stage, we use existing subroutines to encode the distribution function in a tensor train format. In the second stage, we use an NTT ansatz to fit the obtained tensor train. For the NTT fitting procedure, we use a log barrier term to ensure the positivity of each tensor component, and then utilize a second-order alternating minimization scheme to accelerate convergence. In practice, we observe that the proposed NTT fitting procedure exhibits drastically faster convergence than an alternative multiplicative update method that has been previously proposed. Through challenging numerical experiments, we show that our approach can accurately compress target distribution functions.

math.NA↗

arXiv2025

Blind free deconvolution over one-parameter sparse families via eigenmatrix

This note considers the blind free deconvolution problems of sparse spectral measures from one-parameter families. These problems pose significant challenges since they involve nonlinear sparse recovery. The main technical tool is the eigenmatrix method for solving unstructured sparse recovery problems. The key idea is to turn the nonlinear inverse problem into a linear inverse problem by leveraging the R-transform for free addition and the S-transform for free product. The resulting linear problem is solved with the eigenmatrix method tailored to the domain of the parametric family. Numerical results are provided for both the additive and multiplicative free deconvolutions.

math.NA↗

arXiv2025

Solving high-dimensional Hamilton-Jacobi-Bellman equation with functional hierarchical tensor

This work proposes a novel numerical scheme for solving the high-dimensional Hamilton-Jacobi-Bellman equation with a functional hierarchical tensor ansatz. We consider the setting of stochastic control, whereby one applies control to a particle under Brownian motion. In particular, the existence of diffusion presents a new challenge to conventional tensor network methods for deterministic optimal control. To overcome the difficulty, we use a general regression-based formulation where the loss term is the Bellman consistency error combined with a Sobolev-type penalization term. We propose two novel sketching-based subroutines for obtaining the tensor-network approximation to the action-value functions and the value functions, which greatly accelerate the convergence for the subsequent regression phase. We apply the proposed approach successfully to two challenging control problems with Ginzburg-Landau potential in 1D and 2D with 64 variables.

math.NA↗

arXiv2025

COS-DPO: Conditioned One-Shot Multi-Objective Fine-Tuning Framework

In LLM alignment and many other ML applications, one often faces the Multi-Objective Fine-Tuning (MOFT) problem, i.e., fine-tuning an existing model with datasets labeled w.r.t. different objectives simultaneously. To address the challenge, we propose a Conditioned One-Shot fine-tuning framework (COS-DPO) that extends the Direct Preference Optimization technique, originally developed for efficient LLM alignment with preference data, to accommodate the MOFT settings. By direct conditioning on the weight across auxiliary objectives, our Weight-COS-DPO method enjoys an efficient one-shot training process for profiling the Pareto front and is capable of achieving comprehensive trade-off solutions even in the post-training stage. Based on our theoretical findings on the linear transformation properties of the loss function, we further propose the Temperature-COS-DPO method that augments the temperature parameter to the model input, enhancing the flexibility of post-training control over the trade-offs between the main and auxiliary objectives. We demonstrate the effectiveness and efficiency of the COS-DPO framework through its applications to various tasks, including the Learning-to-Rank (LTR) and LLM alignment tasks, highlighting its viability for large-scale ML deployments.

cs.LG↗

arXiv2025

On a spherically lifted spin model at finite temperature

We investigate an $n$-vector model over $k$ sites with generic pairwise interactions and spherical constraints. The model is a lifting of the Ising model whereby the support of the spin is lifted to a hypersphere. We show that the $n$-vector model converges to a limiting distribution at a rate of $n^{-1/2 + o(1)}$. We show that the limiting distribution for $n \to \infty$ is determined by the solution of an equality-constrained maximization task over positive definite matrices. We prove that the obtained maximal value and maximizer, respectively, give rise to the free energy and correlation function of the limiting distribution. In the finite temperature regime, the maximization task is a log-determinant regularization of the semidefinite program (SDP) in the Goemans-Williamson algorithm. Moreover, the inverse temperature determines the regularization strength, with the zero temperature limit converging to the SDP in Goemans-Williamson. Our derivation draws a curious connection between the semidefinite relaxation of integer programming and the spherical lifting of sampling on a hypercube. To the authors' best knowledge, this work is the first to solve the setting of fixed $k$ and infinite $n$ under unstructured pairwise interactions.

math.PR↗

arXiv2025

Inverse nonlinear fast Fourier transform on SU(2) with applications to quantum signal processing

The nonlinear Fourier transform (NLFT) extends the classical Fourier transform by replacing addition with matrix multiplication. While the NLFT on $\mathrm{SU}(1,1)$ has been widely studied, its $\mathrm{SU}(2)$ variant has only recently attracted attention due to emerging applications in quantum signal processing (QSP) and quantum singular value transformation (QSVT). In this paper, we investigate the inverse NLFT on $\mathrm{SU}(2)$ and establish the numerical stability of the layer stripping algorithm for the first time under suitable conditions. Furthermore, we develop a fast and numerically stable algorithm, called inverse nonlinear fast Fourier transform, for performing inverse NLFT with near-linear complexity. This algorithm is applicable to computing phase factors for both QSP and the generalized QSP (GQSP).

quant-ph↗

arXiv2025

Initialization and training of matrix product state probabilistic models

Modeling probability distributions via the wave function of a quantum state is central to quantum-inspired generative modeling and quantum state tomography (QST). We investigate a common failure mode in training randomly initialized matrix product states (MPS) using gradient descent. The results show that the trained MPS models do not accurately predict the strong interactions between boundary sites in periodic spin chain models. In the case of the Born machine algorithm, we further identify a causality trap, where the trained MPS models resemble causal models that ignore the non-local correlations in the true distribution. We propose two complementary strategies to overcome the training failure -- one through optimization and one through initialization. First, we develop a natural gradient descent (NGD) method, which approximately simulates the gradient flow on tensor manifolds and significantly enhances training efficiency. Numerical experiments show that NGD avoids local minima in both Born machines and in general MPS tomography. Remarkably, we show that NGD with line search can converge to the global minimum in only a few iterations. Second, for the BM algorithm, we introduce a warm-start initialization based on the TTNS-Sketch algorithm. We show that gradient descent under a warm initialization does not encounter the causality trap and admits rapid convergence to the ground truth.

math.NA↗

arXiv2025

How Discrete and Continuous Diffusion Meet: Comprehensive Analysis of Discrete Diffusion Models via a Stochastic Integral Framework

Discrete diffusion models have gained increasing attention for their ability to model complex distributions with tractable sampling and inference. However, the error analysis for discrete diffusion models remains less well-understood. In this work, we propose a comprehensive framework for the error analysis of discrete diffusion models based on Lévy-type stochastic integrals. By generalizing the Poisson random measure to that with a time-independent and state-dependent intensity, we rigorously establish a stochastic integral formulation of discrete diffusion models and provide the corresponding change of measure theorems that are intriguingly analogous to Itô integrals and Girsanov's theorem for their continuous counterparts. Our framework unifies and strengthens the current theoretical results on discrete diffusion models and obtains the first error bound for the $τ$-leaping scheme in KL divergence. With error sources clearly identified, our analysis gives new insight into the mathematical properties of discrete diffusion models and offers guidance for the design of efficient and accurate algorithms for real-world discrete diffusion model applications.

cs.LG↗

arXiv2025

Wavelet-based density sketching with functional hierarchical tensor

We introduce the functional hierarchical tensor under a wavelet basis (FHT-W) ansatz for high-dimensional density estimation in lattice models. Recently, the functional tensor network has emerged as a suitable candidate for density estimation due to its ability to calculate the normalization constant exactly, a defining feature not enjoyed by neural network alternatives such as energy-based models or diffusion models. While current functional tensor network models show good performance for lattice models with weak or moderate couplings, we show that they face significant model capacity constraints when applied to lattice models with strong coupling. To address this issue, this work proposes to perform density estimation on the lattice model under a wavelet transformation. Motivated by the literature on scale separation, we perform iterative wavelet coarsening to separate the lattice model into different scales. Based on this multiscale structure, we design a new functional hierarchical tensor ansatz using a hierarchical tree topology, whereby information on the finer scale is further away from the root node of the tree. Our experiments show that the numerical rank of typical lattice models is significantly lower under appropriate wavelet transformation. Furthermore, we show that our proposed model allows one to model challenging Gaussian field models and Ginzburg-Landau models.

math.NA↗

arXiv2025

Approximation of High-Dimensional Gibbs Distributions with Functional Hierarchical Tensors

The numerical representation of high-dimensional Gibbs distributions is challenging due to the curse of dimensionality manifesting through the intractable normalization constant calculations. This work addresses this challenge by performing a particle-based high-dimensional parametric density estimation subroutine, and the input to the subroutine is Gibbs samples generated by leveraging advanced sampling techniques. Specifically, to generate Gibbs samples, we employ ensemble-based annealed importance sampling, a population-based approach for sampling multimodal distributions. These samples are then processed using functional hierarchical tensor sketching, a tensor-network-based density estimation method for high-dimensional distributions, to obtain the numerical representation of the Gibbs distribution. We successfully apply the proposed approach to complex Ginzburg-Landau models with hundreds of variables. In particular, we show that the approach proposed is successful at addressing the metastability issue under difficult numerical cases.

math.NA↗

arXiv2025

Sparse free deconvolution under unknown noise level via eigenmatrix

This note considers the spectral estimation problems of sparse spectral measures under unknown noise levels. The main technical tool is the eigenmatrix method for solving unstructured sparse recovery problems. When the noise level is determined, the free deconvolution reduces the problem to an unstructured sparse recovery problem to which the eigenmatrix method can be applied. To determine the unknown noise level, we propose an optimization problem based on the singular values of an intermediate matrix of the eigenmatrix method. Numerical results are provided for both the additive and multiplicative free deconvolutions.

math.NA↗

arXiv2024

Multi-Objective Optimization via Wasserstein-Fisher-Rao Gradient Flow

Multi-objective optimization (MOO) aims to optimize multiple, possibly conflicting objectives with widespread applications. We introduce a novel interacting particle method for MOO inspired by molecular dynamics simulations. Our approach combines overdamped Langevin and birth-death dynamics, incorporating a "dominance potential" to steer particles toward global Pareto optimality. In contrast to previous methods, our method is able to relocate dominated particles, making it particularly adept at managing Pareto fronts of complicated geometries. Our method is also theoretically grounded as a Wasserstein-Fisher-Rao gradient flow with convergence guarantees. Extensive experiments confirm that our approach outperforms state-of-the-art methods on challenging synthetic and real-world datasets.

cs.LG↗

arXiv2024

Laplace transform based quantum eigenvalue transformation via linear combination of Hamiltonian simulation

Eigenvalue transformations, which include solving time-dependent differential equations as a special case, have a wide range of applications in scientific and engineering computation. While quantum algorithms for singular value transformations are well studied, eigenvalue transformations are distinct, especially for non-normal matrices. We propose an efficient quantum algorithm for performing a class of eigenvalue transformations that can be expressed as a certain type of matrix Laplace transformation. This allows us to significantly extend the recently developed linear combination of Hamiltonian simulation (LCHS) method [An, Liu, Lin, Phys. Rev. Lett. 131, 150603, 2023; An, Childs, Lin, arXiv:2312.03916] to represent a wider class of eigenvalue transformations, such as powers of the matrix inverse, $A^{-k}$, and the exponential of the matrix inverse, $e^{-A^{-1}}$. The latter can be interpreted as the solution of a mass-matrix differential equation of the form $A u'(t)=-u(t)$. We demonstrate that our eigenvalue transformation approach can solve this problem without explicitly inverting $A$, reducing the computational complexity.

quant-ph↗

arXiv2024

Fast Phase Factor Finding for Quantum Signal Processing

This paper presents two efficient and stable algorithms for recovering phase factors in quantum signal processing (QSP), a crucial component of many quantum algorithms. The first algorithm, the ``Half Cholesky" method, which is based on nonlinear Fourier analysis and fast solvers for structured matrices, demonstrates robust performance across all regimes. The second algorithm, ``Fast Fixed Point Iteration," provides even greater efficiency in the non-fully-coherent regime. Both theoretical analysis and numerical experiments demonstrate the significant advantages of these new methods over all existing approaches.

quant-ph↗

arXiv2024

Quantum Multiple Eigenvalue Gaussian filtered Search: an efficient and versatile quantum phase estimation method

Quantum phase estimation is one of the most powerful quantum primitives. This work proposes a new approach for the problem of multiple eigenvalue estimation: Quantum Multiple Eigenvalue Gaussian filtered Search (QMEGS). QMEGS leverages the Hadamard test circuit structure and only requires simple classical postprocessing. QMEGS is the first algorithm to simultaneously satisfy the following two properties: (1) It can achieve the Heisenberg-limited scaling without relying on any spectral gap assumption. (2) With a positive energy gap and additional assumptions on the initial state, QMEGS can estimate all dominant eigenvalues to $ε$ accuracy utilizing a significantly reduced circuit depth compared to the standard quantum phase estimation algorithm. In the most favorable scenario, the maximal runtime can be reduced to as low as $\log(1/ε)$. This implies that QMEGS serves as an efficient and versatile approach, achieving the best-known results for both gapped and gapless systems. Numerical results validate the efficiency of our proposed algorithm in various regimes.

quant-ph↗