Bayesian inference over model-spaces increases the accuracy of model comparison and allows formal testing of hypotheses about model distributions in experimental populations

Abstract

Determining the best model or models for a particular data set, a process known as Bayesian model comparison, is a critical part of probabilistic inference. Typically, this process assumes a fixed model-space (that is, a fixed set of candidate models). However, it is also possible to perform Bayesian inference over model-spaces themselves, thus determining which spaces provide the best explanation for observed data. Model-space inference (MSI) allows the effective exclusion of poorly performing models (a process analogous to Automatic Relevance Detection), and thus mitigates against the well-known phenomenon of model dilution, resulting in posterior probability estimates that are, on average, more accurate than those produced when using a fixed model-space. We focus on model comparison in the context of multiple independent data sets (as produced, for example, by multi-subject behavioural or neuroimaging studies), and cast our proposal as a development of random-effects Bayesian Model Selection, the current state-of-the-art in the field. We demonstrate the increased accuracy of MSI using simulated behavioural and neuroimaging data, as well as by assessing predictive performance in previously-acquired empirical data. Additionally, we explore other applications of MSI, including formal testing for a diversity of models within a population, and comparison of model-spaces between populations. Our approach thus provides an important new tool for model comparison.