When precise inference is wrong: Diagnosing model misspecification with applications to cosmic dawn and reionisation

Abstract

As observations of the cosmic dawn (CD) and epoch of reionisation (EoR) rapidly improve in precision, the approximate forward models used to interpret them might struggle to keep pace. Bayesian inference with a misspecified model, one that cannot represent the processes generating the observations, can yield tightly constrained, yet misleading and biased astrophysics, with different parts of the observations favouring incompatible parameter values. Model misspecification can evade a $χ^2$ goodness-of-fit test, and Bayesian model selection requires a better model, which is not always available. We therefore apply the suspiciousness, a statistic developed to quantify tensions between datasets, to CD/EoR inference as a model criticism tool that tests whether these parts can be explained by the same parameters. We first apply it to HERA 21-cm power spectrum (PS) upper limits, which are too weak to reveal misspecification, as expected, and to HST and JWST UV luminosity functions (LFs). Under our galaxy model, the UV LFs at $z\leq10$ and $z>10$ are in tension at $5.0σ$ in the Gaussian limit, a failure visible in the fits, although doubling the uncertainties of the $z>10$ observations reduces it to $2.1σ$. Finally, we analyse a mock SKA-Low AA$^\ast$ 1080 h 21-cm PS with the model that generated it and with a simpler, misspecified model. We find that the misspecified model silently fails: it fits the PS well, passing visual inspection and, for most noise realisations, a $χ^2$ test, yet its inferred astrophysics is wrong. The Bayesian evidence disfavours it, but only relative to the true model, which is unavailable for real observations. The suspiciousness finds no tension for the well-specified model, but a $\sim3.4σ$ tension for the misspecified one, strongly ruling it out without requiring a more adequate candidate model.