Neural Network Solvers for Nonlinear ODEs: Benchmarking MLPs and B-Spline/Chebyshev KANs

Abstract

Three neural networks are compared as solvers of nonlinear ordinary differential equations: a multilayer perceptron (MLP), a Kolmogorov-Arnold network (KAN) whose edges are B-splines, and a KAN whose edges are Chebyshev series. Each network is trained only on the residual of the equation and on its initial or boundary conditions. The exact solution, or a numerical solution accurate to $10^{-13}$, is used only afterward to measure the error. Five test problems of differential orders one to four are investigated: the logistic equation, a singular Lane-Emden type initial value problem, a singular boundary value problem, the Falkner-Skan equation with suction, a fourth-order problem. The same training budget and the same choice of five activation functions are allocated to all networks. No numerical failures are encountered in any of the \nRuns{} training runs. For every problem, the best KAN is found to be at least as accurate as the best MLP while \paramsKANmin{} to \paramsKANmax{} parameters are used instead of \paramsMLP{}; the margin ranges from a tie to a factor of three. In the Falkner-Skan setting of an earlier study, the B-spline KAN is shown to be 34 to 220 times more accurate at every published test point. Much smaller MLPs are also found to be competitive, whereas at orders three and four, failures are experienced more frequently by KANs, and a coarse basis is required. The derivatives in the equation are distorted by the usual grid extension of a B-spline KAN; consequently, it is replaced with knot insertion, by which the logistic error is lowered more than a hundredfold, though no improvement is provided at order four.