Convergence of Least Squares Methods for PDEs on R^d with Shallow ReLU^k Networks

Abstract

We establish inverse inequalities in Gaussian Sobolev norms for shallow $\mathrm{ReLU}^k$ trial spaces. The estimates identify the sharp regularity threshold and quantify the dependence on parameter separation and bias magnitude. Based on these estimates, we prove convergence of weighted least squares discretizations of linear PDEs on $\mathbb R^d$ using shallow networks with prescribed inner parameters and finitely many residual samples in an expanding ball. Under suitable stability and regularity assumptions, we obtain an error bound of order $h^{k+(d+1)/2-r}δ_\pm^{-(S-r)}$ in the Gaussian Sobolev norm of order $S$, up to logarithmic factors, where $r$ is the PDE order, $h$ is the parameter fill distance, and $δ_\pm$ is the antipodal separation. The convergence results cover deterministic and random sampling and extend to fixed bounded regions by restriction. We construct parameter sets with $δ_\pm\asymp h$, yielding a convergence rate in terms of the network width that matches a lower bound up to logarithmic factors. Numerical experiments support the theoretical results and provide practical guidance on balancing accuracy and stability.