Adaptive Schauder Stochastic Mirror Descent in Banach Spaces

Abstract

In this paper, we introduce an adaptive regularization strategy for stochastic mirror descent (SMD) to solve a class of risk functional minimization problems in infinite-dimensional Banach spaces. This regularization strategy centers on using a Schauder basis to construct a nested family of finite-dimensional subspaces, with the dimension chosen adaptively according to the sample size $n$. We then restrict each SMD subproblem to the corresponding subspace and project the stochastic gradient onto its dual space. This yields closed-form solutions to the SMD subproblems and coordinate-wise updates of the basis coefficients, enabling an implementation with low computational and storage complexity. The subspace dimension also serves as a regularization parameter that balances approximation and optimization errors. For risk functional minimization in $\mathcal{L}^p$ spaces with $1<p<\infty$, we construct Bregman distances adapted to the geometry of the underlying Banach spaces using $\max\{2,p\}$-convex functionals induced by their uniform convexity. At the non-uniformly convex $\mathcal{L}^1$ endpoint, we instead construct a locally strongly convex functional based on the entropy function. By developing a new analytical framework, we establish a convergence rate of $\mathcal O\left(n^{-\min\{\frac12,\frac1p\}}\right)$, up to logarithmic factors. In the misspecified setting, where the minimizer satisfies only weaker regularity conditions, we prove that the risk functional still converges to its minimum value. Finally, we apply the method to statistical inverse problems and illustrate its empirical performance through numerical experiments in both settings.