Exploring hierarchical framework of nonlinear sparse Bayesian learning algorithm through numerical investigations

Abstract

Sparse Bayesian learning (SBL) has been extensively utilized in data-driven modeling to combat the issue of overfitting. While SBL excels in linear-in-parameter models, its direct applicability is limited in models where observations possess nonlinear relationships with unknown parameters. Recently, a semi-analytical Bayesian framework known as nonlinear sparse Bayesian learning (NSBL) was introduced by the authors to induce sparsity among model parameters during the Bayesian inversion of nonlinear-in-parameter models. NSBL relies on optimally selecting the hyperparameters of sparsity-inducing Gaussian priors. It is inherently an approximate method since the uncertainty in the hyperparameter posterior is disregarded as we instead seek the maximum a posteriori (MAP) estimate of the hyperparameters (type-II MAP estimate). This paper aims to investigate the hierarchical structure that forms the basis of NSBL and validate its accuracy through a comparison with a one-level hierarchical Bayesian inference as a benchmark in the context of three numerical experiments: (i) a benchmark linear regression example with Gaussian prior and Gaussian likelihood, (ii) the same regression problem with a highly non-Gaussian prior, and (iii) an example of a dynamical system with a non-Gaussian prior and a highly non-Gaussian likelihood function, to explore the performance of the algorithm in these new settings. Through these numerical examples, it can be shown that NSBL is well-suited for physics-based models as it can be readily applied to models with non-Gaussian prior distributions and non-Gaussian likelihood functions. Moreover, we illustrate the accuracy of the NSBL algorithm as an approximation to the one-level hierarchical Bayesian inference and its ability to reduce the computational cost while adequately exploring the parameter posteriors.