Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 0801.0253

Toward a statistical mechanics of four letter words

Greg J. Stephens·William Bialek
View original source ↗Read source document ↗Export metadata ↓

Abstract

We consider words as a network of interacting letters, and approximate the probability distribution of states taken on by this network. Despite the intuition that the rules of English spelling are highly combinatorial (and arbitrary), we find that maximum entropy models consistent with pairwise correlations among letters provide a surprisingly good approximation to the full statistics of four letter words, capturing ~92% of the multi-information among letters and even "discovering" real words that were not represented in the data from which the pairwise correlations were estimated. The maximum entropy model defines an energy landscape on the space of possible words, and local minima in this landscape account for nearly two-thirds of words used in written English.

Explore related subjects

q-bio.NCcs.CLphysics.data-anphysics.soc-ph

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Greg J. Stephens, William Bialek. 2007-12-31. Toward a statistical mechanics of four letter words. https://doi.org/10.1103/physreve.81.066119

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

The Platonic brain bridge hypothesis: human brain networks as an architectural prior for multimodal large language models

Multimodal large language models predict brain activity, but brain alignment has been a measurement, not a design tool. We propose the Platonic brain bridge hypothesis: omni models, multimodal large language models that process video, audio and text jointly, converge on brain-like representations usable in both directions. From model to brain, brain-likeness of seven omni models is stable across participants, rises with every input channel in three bases, and our encoders lead the Algonauts 2025 out-of-distribution leaderboard. From brain to model, Brain-MoE fixes the expert partition of a frozen base to the seven networks of human cortex, trains experts on network-labelled Brain-AVQA questions, raises held-out accuracy in all 15 model-benchmark pairs by 6.42 percentage points on average and exceeds capacity-matched random experts in 14. Brain-Scope localizes the correspondence to sparse features whose removal weakens brain prediction. Human brain organization is therefore a usable architectural prior for multimodal large language models.

q-bio.NC↗
arXiv2026

When Teachers Smile or Frown: A Profile-Based Analysis of Achievement Emotions

Achievement emotions shape how students engage with and learn from academic tasks, yet most studies examine individual emotions rather than co-occurring affective profiles and their dynamics. We examined latent achievement-emotion profiles and their transitions following exposure to different instructor facial expressions during a video lecture. Self-reported data from 78 Grade VII and VIII students revealed three profiles: enthusiastic, demotivated, and vulnerable. Profile transitions differed across instructor conditions, with happy expressions favouring more adaptive transitions and angry expressions favouring transitions toward demotivation. Exploratory factor analysis and Bayesian structural modelling further identified preparedness and cognitive restraint as regulatory dimensions associated with profile switching.

q-bio.NC↗
arXiv2026

Nonlinear dynamics of random neural networks with second-order synaptic motifs

Classical theories of random neural networks typically assume independent connectivity, overlooking the local motif structures prevalent in biological circuits. Here, we investigate how four second-order synaptic motifs (chain, reciprocal, convergent, and divergent) shape the dynamics of nonlinear firing-rate networks. While previous studies have established that chain correlations generate outlier eigenvalues, we demonstrate that these motifs also jointly reshape the Jacobian eigenvalue bulk. Using the path-integral formalism, we derive a dynamic mean-field theory which reveals that the chain motif acts as a retarded feedback of the ensemble-mean activity through the response kernel, producing a rich repertoire of dynamical regimes, including ferromagnetic states and limit cycles. At sufficiently large magnitude, negative chain correlations produce a glassy, multistable regime that was previously mainly associated with partially symmetric networks. Our theory also distinguishes convergent from divergent motifs: divergent correlations primarily rescale temporal noise, while convergent correlations suppress temporal chaos by converting nonzero mean activity into quenched heterogeneity. Finally, analyses of the Lyapunov spectrum and participation-ratio dimension show that motif structure changes the geometry of chaotic activity, reducing entropy production and attractor dimensionality even when the effective spectral edge is held fixed. Together, these findings establish second-order motifs as a fundamental structural mechanism governing the dynamical regimes of local cortical circuits.

q-bio.NC↗