Dense auto-hetero associative memories applied to noisy communication channels

Abstract

Networks of interacting Hebbian networks have recently been shown to perform a task beyond associative memory, namely \emph{pattern disentanglement}: when fed with a spurious mixture of stored patterns, the different modules spontaneously specialize on, and retrieve, the different constituents of the mixture. So far, this capability has only been established for pairwise interactions, which limits the number of patterns that can be handled. Here we introduce a dense extension of these modular networks, in which both the auto-associative couplings within each module and the hetero-associative couplings among modules are promoted to higher-order Hebbian interactions. We show that, with a suitable choice of the interaction orders, the network disentangles mixtures while storing a number of patterns that scales linearly with the module size, a regime where its pairwise counterpart fails. Through a statistical-mechanical analysis based on Guerra's interpolation, we derive the self-consistency equations for the order parameters and draw the phase diagrams identifying the region where disentanglement is achieved; these predictions are confirmed by Monte Carlo simulations. Finally, we show that disentanglement provides a natural decoding primitive, and we illustrate it with two applications: the explicit reconstruction of all the hidden patterns from the Hebbian tensors and a stream of unlabeled mixtures, and a proof-of-concept communication protocol in which each message token is transmitted as a masked mixture of hidden patterns and decoded by the network dynamics. Owing to its attractor-based decoding, the protocol degrades gracefully under strong channel corruption, where conventional secure-transmission pipelines fail abruptly.