Basic Experiment Planning via Information Metrics: the RoboMendel Problem

Abstract

In this paper we outline some mathematical questions that emerge from trying to "turn the scientific method into math". Specifically, we consider the problem of experiment planning (choosing the best experiment to do next) in explicit probabilistic and information theoretic terms. We formulate this as an information measurement problem; that is, we seek a rigorous definition of an information metric to measure the likely information yield of an experiment, such that maximizing the information metric will indeed reliably choose the best experiment to perform. We present the surprising result that defining the metric purely in terms of prediction power on observable variables yields a metric that can converge to the classical mutual information measuring how informative the experimental observation is about an underlying hidden variable. We show how the expectation potential information metric can compute the "information rate" of an experiment as well its total possible yield, and the information value of experimental controls. To illustrate the utility of these concepts for guiding fundamental scientific inquiry, we present an extensive case study (RoboMendel) applying these metrics to propose sequences of experiments for discovering the basic principles of genetics.