Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 1908.08484

Minimum Description Length Revisited

Peter Grünwald·Teemu Roos
View original source ↗Read source document ↗Export metadata ↓

Abstract

This is an up-to-date introduction to and overview of the Minimum Description Length (MDL) Principle, a theory of inductive inference that can be applied to general problems in statistics, machine learning and pattern recognition. While MDL was originally based on data compression ideas, this introduction can be read without any knowledge thereof. It takes into account all major developments since 2007, the last time an extensive overview was written. These include new methods for model selection and averaging and hypothesis testing, as well as the first completely general definition of {\em MDL estimators}. Incorporating these developments, MDL can be seen as a powerful extension of both penalized likelihood and Bayesian approaches, in which penalization functions and prior distributions are replaced by more general luckiness functions, average-case methodology is replaced by a more robust worst-case approach, and in which methods classically viewed as highly distinct, such as AIC vs BIC and cross-validation vs Bayes can, to a large extent, be viewed from a unified perspective.

Explore related subjects

stat.MEcs.ITcs.LGmath.ITstat.ML

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Peter Grünwald, Teemu Roos. 2019-12-18. Minimum Description Length Revisited. https://doi.org/10.1142/s2661335219300018

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Interpretable Deep Neural Network for Modeling Functional Surrogates

Developing surrogates for computer models has become increasingly important for addressing complex problems in science and engineering. This article introduces an artificial intelligent (AI) surrogate, referred to as the DeepSurrogate, for analyzing functional outputs with vector-valued inputs. The relationship between the functional output and vector-valued input is modeled as an infinite sequence of unknown functions, each representing the relationship at a specific location within the functional domain. These spatially indexed functions are expressed through a combination of basis functions and their corresponding coefficient functions, both of which are modeled using deep neural networks (DNN). The proposed framework accounts for spatial dependencies across locations, while capturing the relationship between the functional output and scalar predictors. It also integrates a Monte Carlo (MC) dropout strategy to quantify prediction uncertainty, enhancing explainability in the deep neural network architecture. The proposed method enables efficient inference on datasets with approximately 50,000 spatial locations and 20 simulations, achieving results in under 10 minutes using standard hardware. The approach is validated on extensive synthetic datasets and a large-scale simulation from the Sea Lake and Overland Surge from Hurricanes (SLOSH) simulator. An open-source Python package implementing the method is made available.

stat.ME↗
arXiv2026

Elastic Multi-Fidelity Bayesian Model Calibration

Bayesian calibration of functional-output computer models typically relies on dimension reduction techniques, such as functional principal component analysis, which assume that differences among simulator realizations arise only from amplitude variation. When simulator output also exhibits phase variation such as shifts in the timing or location of key features, this assumption is violated. Recent work has addressed this issue through elastic calibration, which aligns functional computer model realizations with observed experimental data prior to dimension reduction. Separately, multi-fidelity methods reduce the cost of calibration by supplementing a small number of expensive high-fidelity simulator runs with a larger ensemble of cheap low-fidelity runs. This is typically done through either a mapping strategy, which corrects low-fidelity predictions toward high-fidelity output, or a fusion strategy, which builds a shared basis across both fidelities. This paper combines these two lines of work, introducing elastic multi-fidelity Bayesian model calibration, which aligns high- and low-fidelity functional output to a common reference before applying multi-fidelity mapping or fusion. On a synthetic two-dimensional calibration problem and a dynamic material properties equation-of-state problem, both elastic multi-fidelity strategies match or improve on the leave-one-out predictive accuracy of a mono-fidelity elastic emulator, with the fusion approach achieving the lowest error. Both strategies also produce tighter calibrated posteriors than the mono-fidelity baseline, with the fusion approach providing the best coverage and parameter estimates closest to the true values.

stat.ME↗
arXiv2026

Model Specification Test for Stationary Functional Time Series

We develop a general framework for model specification testing in stationary functional time series. The approach is based on an autoregressive approximation that represents a broad class of stationary functional processes through coefficient kernels whose dimension and autoregressive order may increase with the sample size. Different model assumptions induce different structural restrictions on these kernels, and our tests are constructed by measuring deviations from the corresponding restrictions. We illustrate this principle for three problems: testing a prescribed order of a functional autoregressive model, testing a functional autoregressive moving-average specification, and testing separability of autoregressive coefficient kernels. The resulting statistics are based on weighted $\mathcal{L}^2$-distances, and the critical values are obtained by a multiplier bootstrap. We establish a quantitative bootstrap approximation that is uniform over a class of weight functions and prove asymptotic validity and consistency of the proposed tests. The methodology allows for data-adaptive weighting and is illustrated by simulations and a data example.

stat.ME↗