Estimating the Jacobian matrix of an unknown multivariate function from sample values by means of a neural network

Abstract

We describe, implement and test a novel method for training neural networks to estimate the Jacobian matrix $J$ of an unknown multivariate function $F$. The training set is constructed from finitely many pairs $(x,F(x))$ and it contains no explicit information about $J$. The loss function for backpropagation is based on linear approximations and on a nearest neighbor search in the sample data. We formally establish an upper bound on the uniform norm of the error, in operator norm, between the estimated Jacobian matrix provided by the algorithm and the actual Jacobian matrix, under natural assumptions on the function, on the training set and on the loss of the neural network during training. The Jacobian matrix of a multivariate function contains a wealth of information about the function and it has numerous applications in science and engineering. The method given here represents a step in moving from black-box approximations of functions by neural networks to approximations that provide some structural information about the function in question.