Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.20020

MCMC Can See Can't See

Daniel Matsui Smola
View original source ↗Read source document ↗Export metadata ↓

Abstract

It is folklore that passing "convergence" diagnostics does not guarantee that a Markov chain has converged. We prove this as an impossibility for any finite checklist of diagnostics. Running more chains, longer chains, and smarter diagnostics does not avoid this impossibility in certifying convergence. Under regularity assumptions, we determine how the worst-case undetected non-convergence decays as more diagnostics are added, and how to choose diagnostics optimally. Folklore suggests that the sampler's slowest mixing modes are the "dangerous" ones that must be monitored; we prove that the slow modes yield nearly optimal diagnostics as sampling proceeds. Given a source class, we also give an anytime-valid bound on the remaining discrepancy and show how undetected bias caps the effective sample size that a finite checklist can certify.

Explore related subjects

stat.COmath.STstat.MLstat.TH

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Daniel Matsui Smola. 2026-10-14. MCMC Can See Can't See. https://arxiv.org/abs/2610.20020

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Multivariate Lévy Adaptive B-Spline Regression

We propose Multivariate Lévy Adaptive B-Spline Regression (MLABS), a Bayesian nonparametric regression model designed to capture nonlinear effects, interactions, and locally varying features of multivariate functions. MLABS represents the regression function as a random sum of low-order tensor-product B-spline basis functions under a finite Lévy random measure prior. The resulting representation adapts to the complexity and local structure of the regression function while favoring a sparse basis representation. For posterior inference, we develop a reversible-jump Markov chain Monte Carlo (MCMC) algorithm with Bayesian backfitting, collapsed trans-dimensional updates, and residual-guided proposals. These computational strategies are designed to improve posterior exploration across basis configurations of varying dimensions and structures. We also establish approximation bounds for functions with low-order interactions and heterogeneous smoothness. These bounds relate approximation accuracy to the number of basis terms. Simulation studies show that MLABS achieves competitive predictive performance across a wide range of regression settings, including high-dimensional problems. MLABS also demonstrates robust predictive performance across benchmark real datasets. In an environmental-mixture application, it attains predictive accuracy comparable to Bayesian kernel machine regression at substantially lower computational cost.

stat.CO↗
arXiv2026

Stereographic Multiple-Try Metropolis

Multiple-proposal MCMC algorithms have recently gained attention for their potential to improve performance, especially through parallel implementation on modern hardware. We introduce Stereographic Multiple-Try Metropolis (SMTM), a novel family of gradient-free algorithms designed for sampling high-dimensional distributions. By integrating multiple-try Metropolis (MTM) with the stereographic MCMC framework, SMTM overcomes the traditional limitations of MTM, particularly its pathological convergence behavior often observed in high dimensions. For both light-tailed and heavy-tailed targets, SMTM not only outperforms classical MTM and the existing stereographic random-walk Metropolis but also demonstrates strong robustness to tuning. These advantages are supported by high-dimensional scaling analysis and validated through extensive simulation studies.

stat.CO↗
arXiv2026

Analytically Optimal Sample Allocations for Nested Multivariate Multifidelity Control Variate Estimators

Monte Carlo estimation, an essential tool for quantifying uncertainty in complex scientific and engineering applications, often has unacceptably large errors when computing resources are limited and each sample requires evaluating an expensive high-fidelity model. Multifidelity control variate methods seek to increase the accuracy of Monte Carlo estimation by combining a limited number of expensive high-fidelity samples with a larger number of cheaper lower-fidelity samples. Many different multifidelity estimators have been proposed, including multilevel Monte Carlo (MLMC), multifidelity Monte Carlo (MFMC), approximate control variates (ACVs), and multilevel best linear unbiased estimators (MLBLUEs). An important design choice in multifidelity estimation is the allocation of a limited computational budget among the available models to minimize estimator error. The state of the art in solving this budget allocation problem relies on numerical optimization; to date, closed form solutions have only been derived for estimator forms known to be suboptimal. In this work, we provide closed form expressions for optimal fractional sample allocations for a large class of nested multifidelity control variate estimators, in which higher-fidelity models are evaluated on a subset of the inputs that lower-fidelity models are evaluated on. Our analysis is based on reformulating nested MLBLUEs as an extended version of the MFMC estimator, and is applicable to general multivariate Monte Carlo estimation. We additionally propose a new approach for integer-valued budget allocation based on a greedy strategy used in combinatorial optimization, as well as a method for robust approximation of control variate coefficients from limited pilot samples. Two numerical examples illustrate our theoretical results and demonstrate the increased accuracy of our extended MFMC formulation relative to existing estimators.

stat.CO↗