Distilling Neural Population Geometry into Transformers and GRUs for Motor Control

Maryna Kapitonova; Tonio Ball

Home/Archive/arXiv

arXiv · 2611.06931

Distilling Neural Population Geometry into Transformers and GRUs for Motor Control

Maryna Kapitonova·Tonio Ball

View original source ↗Read source document ↗Export metadata ↓

Abstract

We introduce neuro-distillation (Neuro-Dist), a technique designed to directly transfer computationally useful neural population structure from biological brains into artificial controllers. Using three macaque motor-cortical datasets as neuro-distillation teachers, we align controller internal states with cortical neural population geometry during training; controller deployment requires no neural data. We evaluate gated recurrent unit (GRU) networks and Transformers controlling a simulated arm. Neuro-Dist achieves the lowest median error among five methods, including distillation from movement kinematics, in 10 of 18 comparisons across trained, student-held-out, and new target directions. Seed-split aggregate statistical comparisons significantly favor Neuro-Dist over all six controls on trained and new targets, and five on student-held-out targets (Holm-corrected familywise p<0.05). We furthermore find that Neuro-Dist can substantially improve controller robustness to sensory feedback noise. Mechanistically, Neuro-Dist induced low tangling of trajectories in internal controller state space---a functionally important characteristic of motor cortex, demonstrating successful transfer of a known motor-cortical population geometry feature. These findings establish neuro-distillation as a complement to behavioral distillation and point to a novel class of neuro-interfaces that tap biological neural population structure as a computational resource for improving the control of goal-directed behavior in robotics and embodied AI.

Explore related subjects

q-bio.NC

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Maryna Kapitonova, Tonio Ball. 2026-09-26. Distilling Neural Population Geometry into Transformers and GRUs for Motor Control. https://arxiv.org/abs/2611.06931

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

BEAST3D: Animal behavioral analysis and neural encoding from multi-view video via Gaussian splatting

Multi-view video recordings are increasingly used to capture the 3D movements of animals in experimental settings, yet extracting rich 3D representations from these recordings remains challenging. Supervised pose estimation requires extensive manual annotation, while general-purpose 3D reconstruction models trained on generic scene datasets fail on the specialized imagery and sparse-view setting of laboratory experiments. We address these limitations with BEAST3D, a self-supervised pretraining framework that learns 3D visual representations from unlabeled, calibrated multi-view video. BEAST3D uses a vision transformer to predict 3D Gaussian splats that reconstruct held-out views through differentiable rendering, while simultaneously segmenting the animal from the background. BEAST3D reconstructs 3D structure with as few as four views by conditioning directly on known camera parameters--unlike general-purpose models, which must estimate camera geometry from dense overlapping viewpoints that are seldom available in lab settings. Through comprehensive evaluation across four species, we demonstrate that BEAST3D produces rich, viewpoint-invariant features that transfer effectively to three downstream tasks: novel view synthesis, which validates the quality of the learned 3D representations; multi-view pose estimation, which provides the sparse keypoint trajectories widely used in behavioral analysis; and neural encoding, which relates 3D behavioral features to simultaneously recorded neural activity. BEAST3D thus establishes a versatile framework for behavioral analysis that leverages 3D structure in modern multi-view laboratory recordings.

q-bio.NC↗

arXiv2026

Self-Supervised Learning of Local Predictive Directions as a Neural Computation

Natural sensory dynamics are globally nonlinear but often exhibit simple local geometry, with nearby trajectories evolving preferentially along a small number of predictive directions. We ask how neural circuits could discover this geometry directly from temporal experience. We introduce Local Predictive Directions (LPD), a self-supervised framework that learns local directions whose projections maximally predict near-future trajectory evolution. LPD applies past-future canonical correlation analysis to temporal differences, making the learned representation invariant to translations of the underlying state while avoiding estimation of equilibria or a global dynamical model. To represent systems with multiple predictive regimes, we introduce a non-negative similarity-matching objective, with offline and online algorithms, that partitions trajectory segments into predictive domains, each associated with a distinct local predictive direction. The online algorithm admits a natural neural-network interpretation: the resulting activities provide soft domain assignments, while synaptic weights encode the corresponding local predictive filters. On stochastic nonlinear dynamical systems, LPD recovers the local predictive geometry and partitions trajectories into distinct dynamical regimes. On naturalistic sensory streams, the online algorithm produces center-surround spatial receptive fields and ON/OFF sustained and transient temporal filters resembling those of early visual neurons. Our results suggest that predictive learning can provide a self-supervised mechanism by which neural circuits discover the local geometry of environmental dynamics and represent it in receptive fields.

q-bio.NC↗

arXiv2026

Effective pruning of task-trained recurrent neural networks using noisy fluctuations and connection rescaling

Pruning network connections is key to brain function but, despite its importance, there exist few biologically-plausible pruning rules with demonstrated good performance. We evaluate noise-prune, a recently introduced unsupervised local pruning rule for recurrent networks that uses noisy fluctuations to determine the importance of connections. Noise-prune has previously only been empirically tested on random networks without a specific computational function. We show that noise-prune preserves task-performance in task-trained recurrent neural networks, greatly outperforming a strategy that only uses connection magnitude and exceeding a non-local strategy that uses second-order information. Rather than deterministically removing connections that fall below a threshold, noise-prune samples connections to preserve based on their importance and strengthens retained connections to preserve average strength. We show that this sampling and rescaling is essential to good performance, and can augment other pruning strategies. We also find that the optimal empirical degree of rescaling is lower than predicted by the original theoretical argument, and thus develop a new version of noise-prune that outperforms the previous version. Our results demonstrate that a biologically-motivated pruning strategy based on local activity statistics can effectively preserve task performance in functional recurrent networks by combining activity-dependent connection selection with compensatory strengthening of surviving connections.

q-bio.NC↗