The Universal Warmup Path: Automatic Preconditioner Selection for HMC

Abstract

Euclidean Hamiltonian Monte Carlo (HMC) warmup must choose a step size and constant preconditioner from limited, nonstationary draws. Standard warmup follows a fixed schedule and generally requires the preconditioner structure to be specified in advance. We present a multi-chain controller that starts diagonal and, at dimension-derived window endpoints, selects between diagonal and low-rank-plus-diagonal inverse mass matrices and chooses the retained rank, subject to dimension and sample-support caps. When evidence is inconclusive, the controller gathers another scheduled window. Persistent within-/between-chain disagreement means draws do not support treating one constant preconditioner as an adequate global description; the controller retains its within-region matrix and advises a population or tempering method for regional exploration. Poor held-out score--position linearity advises reparameterization. The controller selected low rank in every evaluated headline benchmark NUTS run ($12/12$). Geometric-mean pooled ESS-per-gradient ratios relative to the prespecified Fisher low-rank warmup and Welford diagonal warmup baselines were respectively $2.451$ and $22.572$ on the synthetic ill-conditioned Gaussian benchmark, and $1.951$ and $6.264$ on the German-credit Bayesian logistic-regression posterior; all compared runs passed the post-warmup quality check. This method unifies common HMC warmup heuristics in one evidence-driven controller, reducing manual choices and turning warning signals into actionable guidance.