arXiv subjects

Simon Vock

Publications and source records attributed to Simon Vock.

4 recordsLinked to original sources

arXiv2026

Adaptive self-organized criticality in deep neural networks

Deep neural networks are high-dimensional dynamical systems whose function depends on the stable propagation of activity and perturbations across many layers. Maintaining suitable dynamical regimes may therefore be essential for robust learning and for preventing dynamical instabilities during training. Here, we show that the global dynamical state of a deep neural network can be autonomously regulated by purely local homeostatic plasticity. Neuronal activity is inferred from responses across inputs, and individual synapses are strengthened or weakened using only the activity of their postsynaptic neuron. Without measuring any global network property, this rule drives networks from both subcritical and supercritical initial conditions toward a common critical state, characterized by conserved activity propagation and a vanishing largest finite-time Lyapunov exponent. When combined with gradient-based learning, homeostatic adaptation counteracts the training-induced drift toward supercritical dynamics, while revealing a competition between dynamical regulation and task optimization. Our results demonstrate how adaptive self-organization can be implemented in deep neural networks and how local plasticity can control their collective dynamical operating point.

q-bio.NC↗

arXiv2025

Critical dynamics governs deep learning

The rapid advances in artificial intelligence (AI) have largely been driven by scaling deep neural networks (DNNs) - increasing model size, data, and computational resources. Yet performance is ultimately governed by network dynamics. The lack of a principled understanding of DNN dynamics beyond heuristic design has contributed to challenges in robustness, suboptimal performance, high energy consumption, and pathologies in continual and AI-generated content learning. In contrast, the human brain appears largely resilient to these problems, and converging evidence suggests this advantage arises from dynamics poised at a critical phase transition. Inspired by this principle, we propose that criticality provides a unifying framework linking structure, dynamics, and function in DNNs. First, analyzing more than 80 state-of-the-art models, we show that a decade of AI progress has implicitly driven successful networks toward criticality - explaining why some architectures succeeded while others failed. Second, we demonstrate that explicitly incorporating criticality into training improves robustness and accuracy while mitigating key limitations of current models. Third, we show that major AI pathologies - including performance degradation in continual learning and model collapse during training on AI-generated data - reflect a loss of critical dynamics. By maintaining networks near criticality, we provide a principled solution to these failures, demonstrating that criticality-based optimization prevents degradation and collapse. Our results establish criticality as a substrate-independent principle of intelligence, connecting AI progress with fundamental principles of brain function, and offering both theoretical insight and practical strategies to ensure long-term DNN performance and resilience as models scale.

q-bio.NC↗

arXiv2021

Effect of diluted connectivities on cluster synchronization of adaptively coupled oscillator networks

Synchronization in networks of oscillatory units is an emergent phenomenon present in various systems, such as biological, technological, and social systems. Many real-world systems have adaptive properties, meaning that their connectivities change with time, depending on the dynamical state of the system. Networks of adaptively coupled oscillators show various synchronization phenomena, such as hierarchical multifrequency clusters, traveling waves, or chimera states. While these self-organized patterns have been previously studied on all-to-all coupled networks, this work extends the investigations towards more complex networks, analyzing the influence of random network topologies for various degrees of dilution of the connectivities. Using numerical and analytical approaches, we investigate the robustness of multicluster states on networks of adaptively coupled Kuramoto-Sakaguchi oscillators against the random dilution of the underlying network topology. Further, we utilize the master stability approach for adaptive networks in order to highlight the interplay between adaptivity and topology.

nlin.AO↗

arXiv2020

Desynchronization transitions in adaptive networks

Adaptive networks change their connectivity with time, depending on their dynamical state. While synchronization in structurally static networks has been studied extensively, this problem is much more challenging for adaptive networks. In this Letter, we develop the master stability approach for a large class of adaptive networks. This approach allows for reducing the synchronization problem for adaptive networks to a low-dimensional system, by decoupling topological and dynamical properties. We show how the interplay between adaptivity and network structure gives rise to the formation of stability islands. Moreover, we report a desynchronization transition and the emergence of complex partial synchronization patterns induced by an increasing overall coupling strength. We illustrate our findings using adaptive networks of coupled phase oscillators and FitzHugh-Nagumo neurons with synaptic plasticity.

nlin.AO↗