Solving Approximation Tasks with Greedy Deep Kernel Methods

Abstract

Kernel methods are versatile tools for function approximation and surrogate modeling. In particular, greedy techniques offer computational efficiency and reliability through inherent sparsity and provable convergence. Inspired by the success of deep neural networks and structured deep kernel networks, we consider deep, multilayer kernels for greedy approximation. This multilayer structure, consisting of linear kernel layers and optimizable kernel activation function layers in an alternating fashion, increases the expressiveness of the kernels and thus of the resulting approximants. Compared to standard kernels, deep kernels are able to adapt kernel intrinsic shape parameters automatically, incorporate transformations of the input space and induce a data-dependent reproducing kernel Hilbert space. For this, deep kernels need to be pretrained using a specifically tailored optimization objective. In this work, we not only introduce deep kernel greedy models, but also present numerical investigations and comparisons with neural networks, which clearly show the advantages in terms of approximation accuracies. As applications we consider the approximation of model problems, the prediction of breakthrough curves for reactive flow through porous media and the approximation of solutions for parameterized ordinary differential equation systems.