Normal approximation for the posterior in exponential families

Abstract

In this paper, we obtain quantitative, non-asymptotic, and data-dependent \textit{Bernstein-von Mises type} bounds on the normal approximation of the posterior distribution in exponential family models with arbitrary centring and scaling. Our bounds, stated in the total variation and Wasserstein distances, are valid for univariate and multivariate posteriors alike, and do not require a conjugate prior setting. They are obtained through a refined version of Stein's method of comparison of operators that allows for improved dimensional dependence in high-dimensional settings and may also be of interest in other problems. Our approach is rather flexible and, in certain settings, allows for the derivation of bounds with rates of convergence faster than the usual \( O(n^{-1/2}) \) rate (when \( n \) is the sample size). We illustrate our findings on a variety of exponential family distributions, including the Weibull, multinomial, and linear regression with unknown variance. The resulting bounds have an explicit dependence on the prior distribution and on sufficient statistics of the data from the sample, and thus provide insight into how these factors affect the quality of the normal approximation. Insights from our examples include identification of conditions under which faster \( O(n^{-1}) \) convergence rates occur for Bernoulli data, illustrations of how the quality of the normal approximation is influenced by the choice of standardisation, and dimensional dependence in high-dimensional settings.