On causally asymmetric versions of Occam's Razor and their relation to thermodynamics

Abstract

In real-life statistical data, it seems that conditional probabilities for the effect given their causes tend to be less complex and smoother than conditionals for causes, given their effects. We have recently proposed and tested methods for causal inference in machine learning using a formalization of this principle. Here we try to provide some theoretical justification for causal inference methods based upon such a ``causally asymmetric'' interpretation of Occam's Razor. To this end, we discuss toy models of cause-effect relations from classical and quantum physics as well as computer science in the context of various aspects of complexity. We argue that this asymmetry of the statistical dependences between cause and effect has a thermodynamic origin. The essential link is the tendency of the environment to provide independent background noise realized by physical systems that are initially uncorrelated with the system under consideration rather than being finally uncorrelated. This link extends ideas from the literature relating Reichenbach's principle of the common cause to the second law.