Comparison of surrogate-based uncertainty quantification methods for computationally expensive simulators

N. E. Owen; P. Challenor; P. P. Menon; S. Bennani

Home/Archive/arXiv

arXiv · 1511.00926

Comparison of surrogate-based uncertainty quantification methods for computationally expensive simulators

N. E. Owen·P. Challenor·P. P. Menon·S. Bennani

View original source ↗Read source document ↗Export metadata ↓

Abstract

Polynomial chaos and Gaussian process emulation are methods for surrogate-based uncertainty quantification, and have been developed independently in their respective communities over the last 25 years. Despite tackling similar problems in the field, to our knowledge there has yet to be a critical comparison of the two approaches in the literature. We begin by providing a detailed description of polynomial chaos and Gaussian process approaches for building a surrogate model of a black-box function. The accuracy of each surrogate method is then tested and compared for two simulators used in industry: a land-surface model (adJULES) and a launch vehicle controller (VEGACONTROL). We analyse surrogates built on experimental designs of various size and type to investigate their performance in a range of modelling scenarios. Specifically, polynomial chaos and Gaussian process surrogates are built on Sobol sequence and tensor grid designs. Their accuracy is measured by their ability to estimate the mean, standard deviation, exceedance probabilities and probability density function of the simulator output, as well as a root mean square error metric, based on an independent validation design. We find that one method does not unanimously outperform the other, but advantages can be gained in some cases, such that the preferred method depends on the modelling goals of the practitioner. Our conclusions are likely to depend somewhat on the modelling choices for the surrogates as well as the design strategy. We hope that this work will spark future comparisons of the two methods in their more advanced formulations and for different sampling strategies.

Explore related subjects

math.ST stat.TH

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

N. E. Owen, P. Challenor, P. P. Menon, S. Bennani. 2017-01-13. Comparison of surrogate-based uncertainty quantification methods for computationally expensive simulators. https://arxiv.org/abs/1511.00926

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Variance Reduction for Independent Metropolis

Assume that we would like to estimate the expected value of a function $F$ with respect to an intractable density $π$, which is specified up to some unknown normalising constant. We prove that if $π$ is close enough under KL divergence to another density $q$, an independent Metropolis sampler estimator that obtains samples from $π$ with proposal density $q$, enriched with a variance reduction computational strategy based on control variates, achieves smaller asymptotic variance than i.i.d. sampling from $π$. The control variates construction requires no extra computational effort but assumes that the expected value of $F$ under $q$ is analytically available. We illustrate this result by calculating the marginal likelihood in a linear regression model with prior-likelihood conflict and a non-conjugate prior. Furthermore, we propose an adaptive independent Metropolis algorithm that adapts the proposal density such that its KL divergence with the target is being reduced. We demonstrate its applicability in a Bayesian logistic and Gaussian process regression problems and we rigorously justify our asymptotic arguments under easily verifiable and essentially minimal conditions.

math.ST↗

arXiv2026

Edgeworth corrections for the spiked eigenvalues of non-Gaussian sample covariance matrices with applications

Yang and Johnstone (2018) established an Edgeworth correction for the largest sample eigenvalue in a spiked covariance model under the assumption of Gaussian observations, leaving the extension to non-Gaussian settings as an open problem. In this paper, we address this issue by establishing first-order Edgeworth expansions for spiked eigenvalues in both single-spike and multi-spike scenarios with non-Gaussian data. Leveraging these expansions, we construct more accurate confidence intervals for the population spiked eigenvalues and propose a novel estimator for the number of spikes. Simulation studies demonstrate that our proposed methodology outperforms existing approaches in both robustness and accuracy across a wide range of settings, particularly in low-dimensional cases.

math.ST↗

arXiv2026

Measures of Dependence based on Wasserstein distances

Measuring dependence between random variables is a fundamental problem in Statistics, with applications across diverse fields. While classical measures such as Pearson's correlation have been widely used for over a century, they have notable limitations, particularly in capturing nonlinear relationships and extending to general metric spaces. In recent years, the theory of Optimal Transport and Wasserstein distances has provided new tools to define measures of dependence that generalize beyond Euclidean settings. This survey explores recent proposals, outlining two main approaches: one based on the distance between the joint distribution and the product of marginals, and another leveraging conditional distributions. We discuss key properties, including characterization of independence, normalization, invariances, robustness, sample, and computational complexity. Additionally, we propose an alternative perspective that measures deviation from maximal dependence rather than independence, leading to new insights and potential extensions. Our work highlights recent advances in the field and suggests directions for further research in the measurement of dependence using Optimal Transport.

math.ST↗