Distributed online learning with growing dimensions for stochastic large regression

Abstract

This paper investigates distributed online estimation and prediction for stochastic large regression models with infinitely many unknown parameters. Existing distributed estimation methods are mostly restricted to finite-dimensional parameter spaces, making them inadequate for emerging infinite-dimensional modeling applications. To tackle this problem, this paper proposes a fully distributed online recursive least squares algorithm with dynamically growing model dimension.The algorithm is constructed with an incrementally augmentable recursive structure, where new parameter dimensions can be embedded in the recursion via zero-padded covariance and estimate augmentation without re-initialization.The almost sure convergence of the algorithm is established under a weak cooperative non-persistent excitation condition, without requiring independence or stationarity assumptions on the regressors.This condition integrates spatio-temporal network information and extends existing finite-dimensional results. The paper further derives an almost sure upper bound on the accumulated prediction regret without any excitation conditions. It is proven that the average regret converges to zero almost surely for a broad class of dimension growth sequences, with the optimal finite-dimensional regret bound recovered as a special case. Finally, numerical simulations are conducted to verify the effectiveness of our theoretical findings.