Fluctuation-learning relationship in neural networks

Tomoki Kurikawa; Kunihiko Kaneko

Home/Archive/arXiv

arXiv · 2409.13597

Fluctuation-learning relationship in neural networks

Tomoki Kurikawa·Kunihiko Kaneko

View original source ↗Read source document ↗Export metadata ↓

Abstract

Learning in neural systems occurs through change in synaptic connectivity that is driven by neural activity. Learning performance is influenced by both neural activity and the task to be learned. Experimental studies suggest a link between learning speed and variability in neural activity before learning. However, the theoretical basis of this relationship has remained unclear. In this work, using principles from the fluctuation-response relation in statistical physics, we derive two formulae that connect neural activity with learning speed. The first formula shows that learning speed is proportional to the variance of spontaneous neural activity and the neural response to input. The second formula, for small input, indicates that speed is proportional to the variances of spontaneous activity in both target and input directions. These formulae apply to various learning tasks governed by Hebbian or generalized learning rules. Numerical simulations confirm that these formulae are valid beyond their theoretical assumptions, even in cases where synaptic connectivity undergoes large changes. Our theory predicts that learning speed increases with the gain of neuronal activation functions and the number of pre-embedded memories, as both enhance the variance of spontaneous neural fluctuations. Additionally, the formulae reveal which input/output relationships are easier to learn, aligning with experimental data. Thus, our results provide a theoretical foundation for the quantitative relationship between pre-learning neural activity fluctuations and learning speed, offering insights into a range of empirical observations.

Explore related subjects

cond-mat.dis-nn nlin.AO q-bio.NC

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Tomoki Kurikawa, Kunihiko Kaneko. 2024-09-20. Fluctuation-learning relationship in neural networks. https://arxiv.org/abs/2409.13597

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Unifying Physical Backpropagation

Physical computing systems exploit device dynamics for computation, but their gradient-based optimization is challenging: backpropagation through a digital twin suffers from a model-reality gap. On-device gradient computation could resolve this issue, and a handful of theoretical and experimental studies have proposed ways to achieve it. Yet a unifying theory identifying when a physical system can compute the gradient of its own performance has been missing. Here we develop such a unification based on the adjoint method: we identify sufficient conditions under which the adjoint field required for formally exact gradients can be generated on the same hardware that performs the computation. Linear and nonlinear systems obey fundamentally different conditions: for linear systems, damping or gain is admissible provided reciprocity is preserved. For nonlinear trajectory systems, the sufficient conditions are reciprocity of the linearized system and the existence of a time-reversal mirror. Algorithmically, the nonlinear case requires infinitesimal nudging, whereas linear systems admit a finite-amplitude experiment. We recover (quantum) Equilibrium Propagation, Hamiltonian echo backpropagation, fully forward mode training and in situ gradient methods in integrated-photonic and free-space-optical systems. Finally, we show that reciprocity is a special case of more general intertwining conditions. For linear systems, these permit exact on-device gradients in a class of non-Hermitian, non-reciprocal systems. For nonlinear trajectories, they combine with generalized time-reversal mirrors to cover, e.g., PT-symmetric equations. The framework also includes time-dependent parameters and Onsager-reciprocal dynamics, providing a unified basis for formally exact physical learning.

cond-mat.dis-nn↗

arXiv2026

Rank-One Signal Recovery in Sparse Wishart Noise

We study the high-dimensional recovery of a signal vector $\mathbf{x}$ in the presence of sparse Wishart-like noise. We define an $N \times N$ matrix $A = J+(θ/N)\mathbf{xx}^{\top}$, where $\mathbf{xx}^{\top}$ is the rank-one deformation of the random noise matrix $J$. We consider a Wishart-like matrix $J={X}^{\top} X$, where $X$ is a sparse $M \times N$ random matrix with entries $X_{ij} = c_{ij}W_{ij}$, with $c_{ij}$ regulating the density of non-zero elements, and $W_{ij}$ the bond weights. Using the replica method, we compute analytically the top eigenpair statistics of $A$, and their dependence on the signal strength $θ$, the rectangularity ratio $α=\sqrt{M/N}$, and the average connectivity of the noise. The spectral observables are expressed in terms of a system of Recursive Distributional Equations, which are efficiently solved via a Population Dynamics algorithm. They allow us to compute the average largest eigenvalue $\langleλ_1\rangle_{A}$, the average top eigenvector component density, and the average overlap between the top eigenvector of $A$ and $\mathbf{x}$. We identify a critical threshold $θ_{\mathrm{crit}}$--depending on the average connectivity of the noise--that marks a BBP-like phase transition: below this value, $\langleλ_1\rangle_{A}$ is unaffected by the signal, and the overlap vanishes. Thus, the signal is not recoverable from the top eigenvector of $A$. For $θ>θ_{\mathrm{crit}}$, the signal-related outlier eigenvalue becomes $\langleλ_1\rangle_{A}$ and the overlap is nonzero, allowing for recovery of the signal. The results are in excellent agreement with numerical diagonalisation. We show that in the dense limit, the recovery threshold and eigen-statistics converge to the results predicted by the classical BBP transition for additive rank-one deformations of dense Wishart matrices.

cond-mat.dis-nn↗

arXiv2026

Topological Fingerprints of Commensurate Order in Twisted Moiré Lattices

Moiré patterns in twisted bilayer materials exhibit long-range periodic order that is highly sensitive to twist angle. Identifying commensurate angles from real-space atomic structures remains challenging, as spectral and geometric methods emphasize global periodicity and are sensitive to disorder and finite-size effects. Here, we introduce a data-driven topological framework that quantifies moiré periodicity by treating atomic configurations as point-cloud data and extracting multiscale signatures using persistent homology. At the core of our approach is a small-neighborhood separation filter that removes redundant local motifs in persistence diagrams while preserving key structural features, enabling sharp minima in Wasserstein distances between twisted and untwisted reference configurations that accurately recover commensurate angles. We benchmark this framework against geometric and spectral similarity measures and show that the resulting topological descriptors remain stable under positional disorder ranging from weak to strong perturbations in the bond length. We further transfer these descriptors using a constrained Gaussian process surrogate to sparsely sampled configurations of a twisted MoTe$_2$ dichalcogenide bilayer. These results establish topological descriptors with neighborhood separation as a robust framework for identifying commensurate order in moiré systems and tracking commensurate structural similarity under disorder, sparsity, and finite-size effects.

cond-mat.dis-nn↗