Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · cond-mat/0602639

Free energy landscapes, dynamics and the edge of chaos in mean-field models of spin glasses

T. Aspelmeier·R. A. Blythe·A. J. Bray·M. A. Moore
View original source ↗Read source document ↗Export metadata ↓

Abstract

Metastable states in Ising spin-glass models are studied by finding iterative solutions of mean-field equations for the local magnetizations. Two different equations are studied: the TAP equations which are exact for the SK model, and the simpler `naive-mean-field' (NMF) equations. The free-energy landscapes that emerge are very different. For the TAP equations, the numerical studies confirm the analytical results of Aspelmeier et al., which predict that TAP states consist of close pairs of minima and index-one (one unstable direction) saddle points, while for the NMF equations saddle points with large indices are found. For TAP the barrier height between a minimum and its nearby saddle point scales as (f-f_0)^{-1/3} where f is the free energy per spin of the solution and f_0 is the equilibrium free energy per spin. This means that for `pure states', for which f-f_0 is of order 1/N, the barriers scale as N^{1/3}, but between states for which f-f_0 is of order one the barriers are finite and also small so such metastable states will be of limited physical significance. For the NMF equations there are saddles of index K and we can demonstrate that their complexity Sigma_K scales as a function of K/N. We have also employed an iterative scheme with a free parameter that can be adjusted to bring the system of equations close to the `edge of chaos'. Both for the TAP and NME equations it is possible with this approach to find metastable states whose free energy per spin is close to f_0. As N increases, it becomes harder and harder to find solutions near the edge of chaos, but nevertheless the results which can be obtained are competitive with those achieved by more time-consuming computing methods and suggest that this method may be of general utility.

Explore related subjects

cond-mat.dis-nn

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

T. Aspelmeier, R. A. Blythe, A. J. Bray, M. A. Moore. 2006-02-27. Free energy landscapes, dynamics and the edge of chaos in mean-field models of spin glasses. https://doi.org/10.1103/physrevb.74.184411

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Deep-layered machines have a built-in Occam's razor

Many complex systems map vast numbers of microscopic descriptions onto far fewer macroscopic outcomes. Surprisingly, these input-output maps strongly favor simple outputs. In an exactly solvable deep-layered machine, we prove that simplicity bias strengthens with every layer. The probability of an output function $F$ approaches $\Pr(F)\propto 2^{-K(F)}$, where $K$ is the Kolmogorov complexity. Our results show that hierarchical composition alone can generate an intrinsic Occam's razor, suggesting a general mechanism for simplicity bias in learning and evolution.

cond-mat.dis-nn↗
arXiv2026

Neural Networks with Exciton-Polariton Waveguides

We propose and numerically study a scheme for neural networks using exciton-polariton waveguide meshes. These enable arbitrary programmable interlayer couplings suited to a restricted Boltzmann machine architecture where polariton phases act as stochastic network variables. We show that such a network can be trained using unsupervised Hebbian and anti-Hebbian updates. This allows the implementation of an associative memory, where the network recovers learned patterns from imperfect inputs.

cond-mat.dis-nn↗
arXiv2026

Finite-width shift of the edge of chaos in deep neural networks

Deep neural networks train best when they are initialised at the edge of chaos, the critical line in the plane of weight and bias variances on which the distance between two inputs neither grows nor shrinks with depth. The line is known for infinitely wide networks; at finite width it moves, and how it moves has been disputed. We treat the logarithm of the distance between two inputs through a network of width $N$ as a random walk and compute its drift and variance to order $1/N$. The edge of chaos moves to a larger weight variance, by $Δσ_w^2=-a_1/(N\,\partial_{σ_w^2}\lnχ_1)$, about $8/N$ for $\tanh$ networks at a bias variance of $0.05$, confirmed by simulations of the exact finite-width recursion. The shift describes a typical network; the average over many networks is dominated by rare networks in which the distance grows, and its critical point moves the other way, which explains why calculations and simulations have disagreed. Backpropagation follows the same random walk, so the shift applies to gradients unchanged. Along the edge of chaos, the information about the similarity of two inputs is constant until the compression over the depth exceeds about one nat, which fixes where to sit on the line. The results give an initialisation rule, Crinit, that places a network on its finite-width edge of chaos without a search. On MNIST and Fashion-MNIST at width $64$ it is more accurate than the standard rules at their usual settings, by a small margin in shallow networks and a large one in deep networks, where most standard rules fail; it trains in fewer steps, and it matches or beats the standard rules with a tuned gain.

cond-mat.dis-nn↗