Mutual Information and the Edge of Chaos in Reservoir Computers

Thomas L. Carroll

Home/Archive/arXiv

arXiv · 1906.03186

Mutual Information and the Edge of Chaos in Reservoir Computers

Thomas L. Carroll

View original source ↗Read source document ↗Export metadata ↓

Abstract

A reservoir computer is a dynamical system that may be used to perform computations. A reservoir computer usually consists of a set of nonlinear nodes coupled together in a network so that there are feedback paths. Training the reservoir computer consists of inputing a signal of interest and fitting the time series signals of the reservoir computer nodes to a training signal that is related to the input signal. It is believed that dynamical systems function most efficiently as computers at the "edge of chaos", the point at which the largest Lyapunov exponent of the dynamical system transitions from negative to positive. In this work I simulate several different reservoir computers and ask if the best performance really does come at this edge of chaos. I find that while it is possible to get optimum performance at the edge of chaos, there may also be parameter values where the edge of chaos regime produces poor performance. This ambiguous parameter dependance has implications for building reservoir computers from analog physical systems, where the parameter range is restricted.

Explore related subjects

nlin.AO cs.NE

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Thomas L. Carroll. 2019-07-19. Mutual Information and the Edge of Chaos in Reservoir Computers. https://arxiv.org/abs/1906.03186

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Frequency bursts in adaptive delay-coupled oscillators

We report on frequency bursting oscillations in a system of phase oscillators with adaptive and delayed coupling. Adaptation of the coupling strengths is considered slow and depends on the phase shift between the oscillators. We find due to the combined chain of adaptation, collective dynamics, and time delays, the system robustly achieves a state in which the oscillator's frequencies are nearly synchronized but detuned by an integer number of small adaptation frequencies. We demonstrate that this quantization of the detuning is caused by alternating slow and fast transitions. Moreover, the observed motions take the form of bursts of instantaneous frequency, and the number of spikes in each burst corresponds to the quantization level of the detuning. We provide a fast-slow analysis of this phenomenon and explain the mechanisms behind the emergence of bursts. Our findings indicate that these frequency bursting oscillations are robust and exist stably within finite parameter regions.

nlin.AO↗

arXiv2026

Discrete-time Kuramoto model with phase lag: Linear stability analysis and onset of synchronization

We investigate the discrete-time version of the Kuramoto model with phase lag, which comprises globally-coupled phase oscillators of distributed frequencies that are evolving under a nonlinear map. In the continuum limit of an infinite number of oscillators ($N\to \infty$), we derive the exact Frobenius-Perron equation for the time evolution of the single-oscillator probability density, and study linear stability of the incoherent state. Instability signals onset of synchronization. The corresponding synchronization threshold is obtained analytically for the case of a Lorentzian distribution of the oscillator frequencies. The threshold differs from that of the continuous-time Kuramoto model, reflecting the fundamentally different stability conditions for discrete-time maps and continuous-time flows. Beyond synchronization threshold, we observe several interesting nonlinear phenomena: Unlike the classical Kuramoto model, the discrete-time version exhibits periodic and chaotic states. Numerical simulations of the finite-$N$ system confirm the analytical prediction for the synchronization threshold, while highlighting breakdown of the celebrated Ott-Antonsen ansatz invoked to conveniently study the continuous-time Kuramoto model in terms of a low-dimensional description.

nlin.AO↗

arXiv2026

Deviations from global coupling in adaptive oscillator networks: a mean-field theory for the variance of coupling weights

A wide range of physical and biological systems are adaptive networks, in which the dynamics of the nodes and of the edges connecting them co-evolve. Mean-field reductions of such systems typically track only the average coupling strength, and therefore cannot determine when the coupling stays effectively homogeneous and when structured connectivity emerges. Here, we present a second-order moment closure that allows us to derive mean-field equations for the coupling-weight variance in networks of heterogeneous phase oscillators with adaptive coupling, starting from uniform coupling weights. In agreement with network simulations, we find a nonlinear, non-monotonic dependence of the relative weight variance on the oscillator heterogeneity that is mediated by the phase coherence. Moreover, we find that deviations from global coupling strongly depend on an interaction between the oscillator heterogeneity and the adaptation rule. Whereas symmetric adaptation causes a strongly coupled core of coherent oscillators to emerge and creates a bistable regime that is absent without adaptation, antisymmetric adaptation leads to antisymmetric coupling within the same core, thereby destabilizing it. Our equations therefore delineate the regimes in which adaptive networks behave like globally coupled systems from those in which more complex coupling patterns form.

nlin.AO↗