Comparing the Empirical Content of Economic Theories

Abstract

A fitted theory that predicts well may do so because of the structure it commits to or because it was fitted to the data it is evaluated on. We propose measures that separate the two by operationalising a theory as a functional form, a support restriction on behavior and a distribution over the parameters, with no parameter fitted to the evaluation sample. Probabilistic completeness is the share of the predictable variation that a rule drawn from the theory captures out of sample. Probabilistic restrictiveness is how much conceivable behavior such a rule excludes; it separates into what the functional form can reach and how the parameter distribution concentrates on it. The parameter distribution can be interpreted as uncertainty about a representative agent or as the distribution of parameters across heterogeneous agents, and the measures adapt to both interpretations through the choice of scoring rule, baseline and ceiling. A scope layer prices the domains a theory claims and compares theories across domains. We apply the measures to expected utility, cumulative prospect theory and disappointment aversion on certainty equivalents for binary lotteries. The functional forms exclude little conceivable behavior, while the literature's parameter distributions exclude most of what the forms leave open. A rule drawn from a literature-informed prior keeps two thirds of fitted prospect theory's completeness, most of expected utility's and none of disappointment aversion's. The flexibility-fit trade-off found for functional forms disappears once the theories are held with their literature priors. Read as populations, disappointment aversion and expected utility cannot produce the dispersion the subjects show, and the literature priors are less well placed for losses than for gains.