Mixture model fitting using conditional models and modal Gibbs sampling

Abstract

Mixture models are a convenient way of modeling data using a convex combination of different parametric distributions. A new algorithm based on Gibbs sampling is used to approximate the posterior distribution of the auxiliary variables, that assign each observation to a group in the mixture, without sampling any other parameter in the model. In particular, the modes of an approximation to the full conditional distributions of the parameters of the densities in the mixture are computed using the Integrated Nested Laplace Approximation. These are plugged-in to the full conditional distribution of the auxiliary variables to draw samples. The posterior distributions of the remainder of the parameters in the mixture are obtained by averaging over their conditional posterior marginals on the auxiliary variables using Bayesian model averaging. This approximation, 'modal' Gibbs sampling, reduces the computational burden in the Gibbs sampling algorithm and provides very good estimates of the posterior distribution of the auxiliary variables. A simulation study supports the validity of 'modal' Gibbs sampling and two examples on well-known datasets are discussed using a mixture of Gaussian and Poisson distributions, respectively.