Posterior Mode-Guided Dimension Reduction for Bayesian Model Averaging in Heavy-Tailed Linear Regression

Abstract

For large model spaces in linear regression with spike-and-slab priors, the potential entrapment of Markov chain Monte Carlo (MCMC)-based methods poses significant challenges in posterior computation. Existing maximum a posteriori (MAP)-based methods provide more computationally viable alternatives, but fail to perform tail heaviness estimation and uncertainty quantification. To address these problems, we propose a method that blends MAP estimation with MCMC-based stochastic search algorithms within an error framework comprising a combination of the hyperbolic and Student-t distributions. The hyperbolic distribution has the light-tailed normal and heavy-tailed Laplace distributions as limiting cases, but is thinner-tailed than the Student-t family. Including the Student-t distribution in the error density enables better adaptation to heavier tails. Amalgamating the two error densities thus ensures a model with more flexible tail behavior when faced with unknown tail thickness in the data, compared to MAP estimators with fixed levels of tail heaviness that assume the errors have a normal or Laplace distribution. Under this proposed error model, the current work develops a two-step expectation conditional maximization (ECM)-guided MCMC algorithm. First, we conduct an ECM-based posterior maximization to guide variable selection. We then execute a Gibbs sampler on the resulting ECM-guided model space for tail heaviness estimation and uncertainty quantification. Through simulation studies and benchmark real datasets, our proposed method is shown to exhibit several advantages in variable selection and uncertainty quantification over state-of-the-art MAP-based methods. To implement our proposed method, we developed the R package FlexBayesReg, available at https://github.com/shamriddha1998/FlexBayesReg.