Andy Lu | Search arXiv

arXiv2026

Braess' Paradox in Uniform Affine Grid Networks

Braess' Paradox is the phenomenon in which adding an edge to a congestion network increases total travel time. We study the paradox in directed rectangular grids where every edge shares the latency function $\ell(x)=ax+b$ and an added chord has latency $\ell_{*}(x) = cx+d$, where $a > 0$ and $b,c,d \ge 0$. Using an analogy with electrical networks, we bound the change in total travel time. We then give a necessary and sufficient condition for a chord to induce the paradox for some choice of nonnegative coefficients and compute the exact proportion of such chords in all grids with dimensions at most $100$. Such chords are scarce, and the fraction is maximized near an aspect ratio of $2:1$. We then improve the established $4/3$ upper bound on the Braess Ratio to one depending only on the grid dimensions, approaching $1.207$ on squares and $4/3$ on thin grids. Finally, we prove any ratio-maximizing chord must have zero latency.

math.CO↗

arXiv2020

Shallow Univariate ReLu Networks as Splines: Initialization, Loss Surface, Hessian, & Gradient Flow Dynamics

Understanding the learning dynamics and inductive bias of neural networks (NNs) is hindered by the opacity of the relationship between NN parameters and the function represented. We propose reparametrizing ReLU NNs as continuous piecewise linear splines. Using this spline lens, we study learning dynamics in shallow univariate ReLU NNs, finding unexpected insights and explanations for several perplexing phenomena. We develop a surprisingly simple and transparent view of the structure of the loss surface, including its critical and fixed points, Hessian, and Hessian spectrum. We also show that standard weight initializations yield very flat functions, and that this flatness, together with overparametrization and the initial weight scale, is responsible for the strength and type of implicit regularization, consistent with recent work arXiv:1906.05827. Our implicit regularization results are complementary to recent work arXiv:1906.07842, done independently, which showed that initialization scale critically controls implicit regularization via a kernel-based argument. Our spline-based approach reproduces their key implicit regularization results but in a far more intuitive and transparent manner. Going forward, our spline-based approach is likely to extend naturally to the multivariate and deep settings, and will play a foundational role in efforts to understand neural networks. Videos of learning dynamics using a spline-based visualization are available at http://shorturl.at/tFWZ2.

cs.LG↗