Search arXivSearch

arXiv subjects

Jean-Christophe Mourrat

Publications and source records attributed to Jean-Christophe Mourrat.

At least 19 recordsLinked to original sources

Efficient computation of the asymptotics of extensive-rank HCIZ integrals

We study the high-dimensional asymptotics of Harish-Chandra-Itzykson-Zuber (HCIZ) integrals in the extensive-rank regime. The limit of these integrals is governed by a one-dimensional boundary-value hydrodynamical problem originally derived by Matytsin (1994) and rigorously proved by Guionnet and Zeitouni (2002). Despite its wide-ranging applications, explicit solutions to this problem are known only in a few specific cases. In this work, we introduce an efficient numerical scheme based on a particle discretization and prove its convergence to the continuous boundary-value problem for generic boundary densities. We validate our approach against known analytical solutions and apply it to generic densities, uncovering interesting dynamical phenomena. The high-dimensional limit of HCIZ integrals appears in various contexts, from the large deviations of random matrix spectra to the limiting free energy of disordered systems, high-dimensional statistics, and machine learning. As such, our contribution opens the way towards the numerical exploration of a wide range of high-dimensional models that were previously intractable.

math.PR

Singular perturbations and hierarchical learning in two-layer neural networks

We study the population gradient flow of an infinitely wide two-layer neural network learning a misspecified single-index model in high dimension. The two layers are optimized jointly, with a perturbative parameter tuning the relative training speed between the first and second layer. This setting was considered by Berthier, Montanari and Zhou in \cite{berthier2024learning}, who conjectured a hierarchical learning scenario with explicit timescales as the second layer is trained faster than the first. In this paper, we prove that the constant and linear components of the hidden link function are indeed recovered within the predicted timescales, at sharp explicit thresholds. We then analyze the onset of learning of the quadratic component and show that the components learned at earlier stages continue to influence the dynamics in an essential way. Our proof is based on quantitative approximation results for singularly perturbed flows evolving near a manifold defined by integral constraints. At a phenomenological level, we also show that the empirical measure of the weights displays singular behaviour when reaching the quadratic component of the hidden link, with a small fraction of neurons growing significantly while the remaining ones rearrange to preserve the components already learned.

cs.LG

Oscillating solutions to the mean-field Langevin descent-ascent flow

We present a counterexample to the statement of convergence of the mean-field Langevin descent-ascent flow on $\mathbb{R}^2$. We consider payoff functions that are shaped as a double well in each coordinate, and for which the deterministic dynamics admits a limit cycle. When the coupling between the two coordinates is sufficiently strong and the entropic regularization sufficiently small, we show that the mean-field dynamics remains close to this cyclic behavior, and in particular, does not converge.

math.OC

One-sided large deviations for the ground-state energy of spin glasses

We describe the large deviations above its typical value of the maximal energy of a spin glass with +/-1 spins. Thanks to the relatively explicit description of the rate function we identify, we then show that the latter is asymptotically quadratic near its minimum if and only if an external magnetic field is present. The proof starts from a Parisi-type formula for the fractional moments of the partition function, which we then leverage to obtain the limit of the Laplace transform of the maximum energy. Using convex-duality arguments, we then rewrite this Laplace transform as a supremum over martingales, and thereby deduce the large-deviation principle with explicit rate function.

math.PR

PL conditions do not guarantee convergence of gradient descent-ascent dynamics

We give an example of a function satisfying a two-sided Polyak-Lojasiewicz condition but for which a gradient descent-ascent flow line fails to converge to the saddle point, circling around it instead. We can even impose the function to be strongly convex in one variable and to satisfy a PL condition in the other variable.

math.OC

On the local nature of the de Almeida-Thouless line for mixed \(p\)-spin glasses

Jagannath and Tobasco~\cite{JagTob} proposed a generalized de Almeida-Thouless (AT) criterion aimed at characterizing the replica symmetric (RS) regime for a broad class of mixed \(p\)-spin glass models with Ising spins. In this paper, we show that this generalized AT condition does not characterize the RS regime in general. Using the Hopf-Lax representation of the Parisi formula, we construct explicit counterexamples within the class of mixed \(p\)-spin models. In particular, we exhibit a model in which the classical AT perturbation is performed around the unique minimizer of the RS free energy, and prove that even in this setting, the AT criterion fails to characterize the RS phase. By contrast, the validity of the classical AT condition for the Sherrington-Kirkpatrick model remains open.

math.PR

Spin glasses and the Parisi formula

Spin glasses are models of statistical mechanics in which a large number of simple elements interact with one another in a disordered fashion. One of the fundamental results of the theory is the Parisi formula, which identifies the limit of the free energy of a large class of such models. Yet many interesting models remain out of reach of the classical theory, and direct generalizations of the Parisi formula yield invalid predictions. I will report here on some partial progress towards the resolution of this problem, which also brings a new perspective on classical results.

math.PR

The convex structure of the Parisi formula for multi-species spin glasses

We study the free energy of mean-field multi-species spin glasses with convex covariance function. For such models with $D$ species, the Parisi formula is known to be valid, and expresses the limit free energy as a supremum over monotone probability measures on $\mathbb{R}_+^D$. We show here that one can transform this representation into a supremum over all probability measures on $\mathbb{R}_+^D$ of a concave functional. We then deduce that the Parisi formula admits a unique maximizer. Using convex-duality arguments, we also obtain a new representation of the free energy as an infimum over martingales in a Wiener space.

math.PR

Operator $\ell^\infty \to \ell^\infty$ norm of products of random matrices

We study the $\ell^\infty \to \ell^\infty$ operator norm of products of independent random matrices with independent and identically distributed entries. For $n$-by-$n$ matrices whose entries are centered, have unit variance, and have a finite moment of order $4\alpha$ for some $\alpha > 1$, we find that the operator norm of the product of $p$ matrices behaves asymptotically like $n^{\frac {p+1}{2}}\sqrt{2/\pi}$. The case of products of possibly non-square matrices with possibly non-centered entries is also covered.

math.PR

Simultaneous replica-symmetry breaking for vector spin glasses

We consider mean-field vector spin glasses with possibly non-convex interactions. Up to a small perturbation of the parameters defining the model, the asymptotic behavior of the Gibbs measure is described in terms of a critical point of an explicit functional. In this paper, we study some properties of these critical points. Under modest assumptions ensuring that different types of spins interact, we show that the replica-symmetry-breaking structures of the different types of spins are in one-to-one correspondence with one another. For instance, if some type of spins displays one level of replica-symmetry breaking, then so do all the other types of spins. This extends the recent results of [Electronic Journal of Probability, 27:1-75, 2022] and [Comm. Math. Phys., 394(3):1101-1152, 2022] that were obtained in the case of multi-species spherical spin glasses with convex interactions.

math.PR

An informal introduction to the Parisi formula

This note is an informal presentation of spin glasses and of the Parisi formula. We also discuss some models for which the Parisi formula is not well-understood, and some partial progress that relies upon a connection with partial differential equations.

math.PR

Color symmetry breaking in the Potts spin glass

The Potts spin glass is an analogue of the Sherrington-Kirkpatrick model in which each spin can take one of $\kappa$ possible values, which we interpret as colors. It was suggested in arXiv:2310.06745 that the order parameter for this model is always invariant with respect to permutations of the colors. We show here that this is false whenever $\kappa \ge 58$.

math.PR

Critical point representation of the mutual information in the sparse stochastic block model

We consider the problem of recovering the community structure in the stochastic block model. We aim to describe the mutual information between the observed network and the actual community structure as the number of nodes diverges while the average degree of a given node remains bounded. Our main contribution is a representation of the limit of this quantity, assuming it exists, as an explicit functional evaluated at a critical point of that functional. While we mostly focus on the two-community setting for clarity, we expect our method to be robust to model generalizations. We also present an example involving four communities where we show the invalidity of a plausible candidate variational formula for this limit.

math.PR

Quantitative equilibrium fluctuations for interacting particle systems

We consider a class of interacting particle systems in continuous space of non-gradient type, which are reversible with respect to Poisson point processes with constant density. For these models, a rate of convergence was recently obtained in 10.1214/22-AOP1573 for certain finite-volume approximations of the bulk diffusion matrix. Here, we show how to leverage this to obtain quantitative versions of a number of results capturing the large-scale fluctuations of these systems, such as the convergence of two-point correlation functions and the Green-Kubo formula.

math.PR

Statistical mechanics of mean-field disordered systems: a Hamilton-Jacobi approach

The goal of this book is to present new mathematical techniques for studying the behaviour of mean-field systems with disordered interactions. We mostly focus on certain problems of statistical inference in high dimension, and on spin glasses. The techniques we present aim to determine the free energy of these systems, in the limit of large system size, by showing that they asymptotically satisfy a Hamilton-Jacobi equation. The first chapter is a general introduction to statistical mechanics, with a focus on the Curie-Weiss model. We give a brief introduction to convex analysis and large deviation principles in Chapter 2, and identify the limit free energy of the Curie-Weiss model using these tools. In Chapter 3, we define the notion of viscosity solution to a Hamilton-Jacobi equation, and use it to recover the limit free energy of the Curie-Weiss model. We discover technical challenges to applying the same method to generalized versions of the Curie-Weiss model, and develop a new selection principle based on convexity to overcome these. We then turn to statistical inference in Chapter 4, focusing on the problem of recovering a large symmetric rank-one matrix from a noisy observation, and we see that the tools developed in the previous chapter apply to this setting as well. Chapter 5 is preparatory work for a discussion of the more challenging case of spin glasses. The first half of this chapter is a self-contained introduction to Poisson point processes, including limit theorems on extreme values of independent and identically distributed random variables, which we believe to be of independent interest. We finally turn to the setting of spin glasses in Chapter 6. For the Sherrington-Kirkpatrick model, we show how to relate the Parisi formula with the Hamilton-Jacobi approach. We conclude with a more informal discussion on the status of current research for more challenging models.

math.PR

On the free energy of vector spin glasses with non-convex interactions

The limit free energy of spin-glass models with convex interactions can be represented as a variational problem involving an explicit functional. Models with non-convex interactions are much less well-understood, and simple variational formulas involving the same functional are known to be invalid in general. We show here that a slightly weaker property of the limit free energy does extend to non-convex models. Indeed, under the assumption that the limit free energy exists, we show that this limit can always be represented as a critical value of the said functional. Up to a small perturbation of the parameters defining the model, we also show that any subsequential limit of the law of the overlap matrix is a critical point of this functional. We believe that these results capture the fundamental conclusions of the non-rigorous replica method.

math.PR

Un-inverting the Parisi formula

The free energy of any system can be written as the supremum of a functional involving an energy term and an entropy term. Surprisingly, the limit free energy of mean-field spin glasses is expressed as an infimum instead, a phenomenon sometimes called an inverted variational principle. Using a stochastic-control representation of the Parisi functional and convex duality arguments, we rewrite this limit free energy as a supremum over martingales in a Wiener space.

math.PR