Empirical Wavelet-based Estimation for Non-linear Additive Regression Models

Abstract

Additive regression models are actively researched in the statistical field because of their usefulness in the analysis of responses determined by non-linear relationships with multivariate predictors. In this kind of statistical models, the response depends linearly on unknown functions of predictor variables and typically, the goal of the analysis is to make inference about these functions. In this paper, we consider the problem of Additive Regression with random designs from a novel viewpoint: we propose an estimator based on an orthogonal projection onto a multiresolution space using empirical wavelet coefficients that are fully data driven. In this setting, we derive a mean-square consistent estimator based on periodic wavelets on the interval $[0,1]$. For construction of the estimator, we assume that the joint distribution of predictors is non-zero and bounded on its support; We also assume that the functions belong to a Sobolev space and integrate to zero over the $[0,1]$ interval, which guarantees model identifiability and convergence of the proposed method. Moreover, we provide the $\mathbb{L}_{2}$ risk analysis of the estimator and derive its convergence rate. Theoretically, we show that this approach achieves good convergence rates when the dimensionality of the problem is relatively low and the set of unknown functions is sufficiently smooth. In this approach, the results are obtained without the assumption of an equispaced design, a condition that is typically assumed in most wavelet-based procedures. Finally, we show practical results obtained from simulated data, demonstrating the potential applicability of our method in the problem of additive regression models with random designs.