A Finite Data Based Regularization in a Normed Linear Space Setting

Abstract

One of the basic problems in mathematical learning theory is to identify a function $f: Ω\to {\mathbb R}$ with certain specific properties which fits a given training data $\{(x_i, ξ_i)\in Ω\times {\mathbb R}: i=1, \ldots, n\}$, in the sense that, $f(x_i) = ξ_i$ for $i=1, \ldots, n$, where $Ω$ is a compact subset of ${\mathbb R}^d$ for some $d\in {\mathbb N}$ and $f$ is required to have certain specified characteristics. We address this problem when $f$ belongs to an arbitrary normed linear space ${X}$, and the evaluation maps $f\mapsto f(x_i)$ are replaced by maps of the form $ f\mapsto φ_i(f)$ on $X$, where $φ_1, \ldots, φ_n$ are continuous linear functionals on ${X}$. Using an inner product structure on ${\mathbb R}^n$, we shall device a method of least-squares for obtaining an approximate solution for the above problem and also identify a subspace of ${X}$ in which the least-square solution is unique, which in turn is shown to be equivalent to solving a matrix equation. In the context when the matrix under consideration is ill-conditioned, a regularized equation is devised, and order optimal error estimates are derived for the exact targeted data $ξ=(ξ_1, \ldots, ξ_n)$ and also when it is noisy, by choosing the regularization parameter appropriately.