Exact Realization of Deep ReLU Computation in a Microscopic Statistical-Mechanical System

Abstract

Understanding the physical origin of the piecewise-linear computation in deep rectified linear unit (ReLU) networks remains a fundamental challenge. Here we establish an exact statistical-mechanical realization of arbitrary ReLU networks. Starting from a microscopic configuration space and its state multiplicities, we construct a partition function without prescribing a neural-network activation function. The resulting system admits equivalent descriptions in terms of cascaded quantum operations with post selection, or fermionic transport model. We rigorously prove that in $β\to+\infty$ limit, the thermodynamic observables of this system exactly reproduce the hidden states, outputs, and loss function of an arbitrary deep ReLU network. Crucially, we demonstrate that the piecewise-linear behavior of the ReLU network emerges from a first-order phase transition in the microscopic system, where the discontinuities precisely coincide with the boundaries of linear regions of ReLU network. This work establishes an exact physical foundation for deep neural computation and provides a novel statistical-mechanical perspective on neural architecture design.