Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.21872

On Minimum Divergence for Distribution Reconstruction

Justin Bec·Thomas Le Corre·Aurélien Garivier
View original source ↗Read source document ↗Export metadata ↓

Abstract

We study the possibility of generalizing maximum-entropy reconstruction, which approximately recovers a probability distribution from finitely many (generalized) moments by minimizing the Kullback--Leibler (KL) divergence to a reference measure. The reconstruction lies in an exponential family and is also the best KL approximation of the unknown distribution within that family: moment matching and information projection coincide. Replacing KL by a smooth f-divergence, the reconstruction still exists in an associated family, but the best approximation property does not. Specifically, a family is called entropy-adapted if the projection of any distribution matches its moments. We show that, up to equivalence, KL is the only $C^2$ f-divergence that is entropy-adapted, whatever the choice of the generalized moments. We illustrate the consequences of this failure numerically.

Explore related subjects

math.STcs.ITmath.ITstat.TH

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Justin Bec, Thomas Le Corre, Aurélien Garivier. 2026-10-15. On Minimum Divergence for Distribution Reconstruction. https://arxiv.org/abs/2610.21872

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Local increment inference for time-inhomogeneous drift in Gaussian processes

We study statistical inference for deterministic drifts in Gaussian process models under high-frequency observations over an expanding time horizon. Using a least squares-type contrast based on first-order increments, we establish consistency and asymptotic normality under conditions on drift accumulation and increment dependence.A key feature is that the convergence rate is determined jointly by the deterministic signal and the full covariance structure of the weighted Gaussian increments, rather than by local noise roughness alone.For power and fixed-frequency periodic drifts under Gaussian and Ornstein-Uhlenbeck covariance kernels, we derive explicit convergence rates and limiting variances, revealing distinct regimes depending on the drift structure and, for periodic drifts, the noise spectrum. These results clarify the respective roles of sampling frequency and observation horizon.

math.ST↗
arXiv2026

Benign Overfitting Property of the Minimum $\ell_q$ -norm Interpolant Estimator in Regression Problem via Feature Space Decomposition

This paper establishes high-probability, non-asymptotic upper bounds for the population excess risk of minimum $\ell_q$-norm interpolating estimators in linear regression, valid for all $1<q<\infty$ under three scenarios: convex concentration, mild moment assumptions and log-concavity. Consequently, we derive sufficient conditions for these estimators to exhibit benign overfitting. Our analysis hinges on a Feature Space Decomposition (FSD) framework, which naturally highlights the self-regularizing properties of minimum norm interpolants as well as the role of alignment between the signal and the features for benign overfitting. From a technical perspective, our approach circumvents the traditionally used Convex Gaussian Min-max Theorem (CGMT) by instead leveraging the Dvoretzky-Milman theorem. We emphasize that the FSD method has the potential to refine standard uniform convergence approaches. Furthermore, the isomorphic Dvoretzky-Milman theorem established herein -- proven under the three scenarios mentioned above -- for the $\ell_q$-norm is of independent mathematical interest.

math.ST↗
arXiv2026

Asymptotic Normality of the Hill Estimator in Linear Preferential Attachment

We prove asymptotic normality of the Hill estimator based on the largest $k_n$ degrees in a homogeneous linear preferential attachment graph with $n$ vertices. Each new vertex sends a fixed number $m\ge1$ of edges sequentially to older vertices, with attachment weight $d+δ$ and fixed $δ>-m$. Let $α=m/(2m+δ)$. For every deterministic sequence $k_n\to\infty$ satisfying $\sqrt{k_n}(k_n/n)^α\to0$, the estimator, centered at $α$ and multiplied by $\sqrt{k_n}$, converges to a centered normal law with an explicit variance strictly smaller than $α^2$. The reduction relative to the Hill variance for independent observations separates into contributions from the different degree laws at different birth times and from competition for new edges. The contribution from competition is $1/m$ times that from different birth times. Representations using Gamma and beta functions give a consistent standard error and a studentized normal limit. The proof uses martingales indexed by edge attachments and comparisons with independent pure-birth processes, while retaining the integer degrees and their ties.

math.ST↗