Optimal estimation of Gaussian mixtures via denoised method of moments

Yihong Wu; Pengkun Yang

Home/Archive/arXiv

arXiv · 1807.07237

Optimal estimation of Gaussian mixtures via denoised method of moments

Yihong Wu·Pengkun Yang

View original source ↗Read source document ↗Export metadata ↓

Abstract

The Method of Moments [Pea94] is one of the most widely used methods in statistics for parameter estimation, by means of solving the system of equations that match the population and estimated moments. However, in practice and especially for the important case of mixture models, one frequently needs to contend with the difficulties of non-existence or non-uniqueness of statistically meaningful solutions, as well as the high computational cost of solving large polynomial systems. Moreover, theoretical analysis of the method of moments are mainly confined to asymptotic normality style of results established under strong assumptions. This paper considers estimating a $k$-component Gaussian location mixture with a common (possibly unknown) variance parameter. To overcome the aforementioned theoretic and algorithmic hurdles, a crucial step is to denoise the moment estimates by projecting to the truncated moment space (via semidefinite programming) before solving the method of moments equations. Not only does this regularization ensures existence and uniqueness of solutions, it also yields fast solvers by means of Gauss quadrature. Furthermore, by proving new moment comparison theorems in the Wasserstein distance via polynomial interpolation and majorization techniques, we establish the statistical guarantees and adaptive optimality of the proposed procedure, as well as oracle inequality in misspecified models. These results can also be viewed as provable algorithms for Generalized Method of Moments [Han82] which involves non-convex optimization and lacks theoretical guarantees.

Explore related subjects

math.ST stat.TH

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Yihong Wu, Pengkun Yang. 2019-04-13. Optimal estimation of Gaussian mixtures via denoised method of moments. https://arxiv.org/abs/1807.07237

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Detection Is Harder Than Estimation in Certain Regimes: Inference for Moment and Cumulant Tensors

We study estimation and detection of high-order cumulant tensors from $n$ i.i.d. observations of a $p$-dimensional random vector, with performance measured in the tensor spectral norm. Under sub-Gaussianity, we show that the minimax rate for estimating the order-$d$ cumulant tensor is $\sqrt{p/n}\wedge 1$. In contrast to covariance estimation, the natural plug-in estimator obtained by replacing population moments in the moment--cumulant formula with sample moments, which we call the sample cumulant tensor, is generally not rate-optimal for $d\ge 3$, and we construct an estimator that attains the minimax rate. On the computational side, we study testing whether the $d$-th order cumulant tensor vanishes after whitening. We provide evidence, under a model-specific low-degree conjecture, that detection is computationally hard when $n\ll p^{d/2}$. At the same time, we identify a regime in which an efficiently computable estimator achieves error smaller than the separation at which polynomial-time tests can reliably distinguish the null from the alternative under the same conjecture. This reveals an unusual reverse detection--estimation gap: computationally efficient detection can be harder than computationally efficient estimation. The underlying reason is that the relevant loss, tensor spectral norm, is itself NP-hard to compute, creating a new form of computational--statistical gap.

math.ST↗

arXiv2026

Debiased parametric inference for regime-switching jump diffusions with infinite variation jumps

We study high-frequency inference for an ergodic regime-switching jump diffusion with locally stable infinite-variation jumps. The state and the finite-state regime are observed on the same grid. The mesh satisfies $h_n\to0$, while the horizon satisfies $T_n=nh_n\to\infty$. We estimate structural drift and diffusion parameters in the presence of unknown regime-specific Lévy measures. For a common activity index $1<β<8/5$, we remove two leading jump biases from each regime's truncated variation. We then combine the diffusion estimator with estimated signed tail means in an adaptive drift contrast. We prove joint asymptotic normality at rates $\sqrt{T_n}$ and $\sqrt n$, together with feasible studentization. The growing horizon requires uniform control of the debiasing remainders and random denominators over increasing regime exposures. It also produces an explicit Lévy-measure term in the drift covariance. A fixed-grid median stabilizes the two-step correction without changing its first-order limit. We study finite-sample behavior by simulation and apply the method to 127.5 hours of cattle sensor data. In the application, jaw-activity labels are recorded by a device separate from the sensor that measures the state process.

math.ST↗

arXiv2026

On the Accuracy of Approximate Leave-One-Out with Blockwise Nonsmooth Regularization in High-dimensions

Approximate leave-one-out (ALO) refers to methods that provide computationally efficient and statistically accurate approximations to leave-one-out risk estimation. These methods have recently attracted increased attention because of their connections to other important problems, including machine unlearning and data valuation. For empirical risk minimization, ALO methods often rely on a few Newton steps and therefore require smooth loss functions and regularizers. In high-dimensional problems, however, regularizers are frequently non-smooth. This has motivated the development of heuristic ALO methods for non-smooth regularization, whose empirical accuracy is often strong but whose theoretical justification remains limited. In this paper, we study the accuracy of a heuristic ALO method for a broad class of non-differentiable penalties that promote block sparsity in the proportional high-dimensional regime. Assuming uniformly bounded block sizes, we prove that the expected average absolute discrepancy between the exact leave-one-out loss and its ALO approximation is $O(\log^{3/2}(en)/\sqrt{n})$. Our proof technique differs substantially from existing approaches and relies on obtaining sharp bounds on the number of active groups whose status changes between the full-data and leave-one-out estimators. In addition to yielding stronger guarantees under weaker assumptions than prior work, our analysis provides new insight into the stability of sparse estimators under perturbations of the training data.

math.ST↗