What Inverse Problems Can Be Solved Using Neural Networks?

Abstract

The relationship between classical methods for solving inverse problems and neural network methods is examined. An answer is provided to the question of what inverse problems can be solved using neural networks. The answer is that neural network algorithms can solve only conditionally well-posed problems (well-posed according to Tikhonov). The reason is that the design of standard neural network algorithms violates the well-known Bakushinsky veto from the theory of ill-posed problems. It requires that any method theoretically guaranteeing a stable solution to an ill-posed problem must depend explicitly on the error level of data. The article highlights the distinction between the concept of regularization (RA) in the theory of ill-posed problems and regularization as understood in neural network training, where Bakushinsky's veto is violated. Due to this violation, any changes to the neural network architecture, regularization during training, or training dataset augmentation cannot alter the answer specified above. It is theoretically demonstrated that an ill-posed inverse problem can be solved only by using a parametric set of neural networks that explicitly depend on the known error levels (as parameters) of data. Accordingly, as an alternative to standard neural network algorithms, \neural network based regularization algorithms (NNBRAs) are proposed to solve ill-posed inverse problems. One approach to their construction involves using a regularized dataset. Such a dataset can be generated from perturbed inverse-problem data using a specific base RA explicitly depended on the error level of the data. Test solving the simplest finite-dimensional ill-posed inverse problem demonstrate that the accuracy of the NNBRA approximate solutions is comparable to that of the base RA. At the same time, errors of conventional neural network methods may be unacceptable.