M-Tensor Formalism: A Non-iterative High Dimensional Least Squares Regression for Nonlinear Models with Scarce Data

Abstract

We present a multilinear regression framework based on tensor algebra tailored to high-dimensional contexts where data is scarce. We exploit algebraic properties of a partial tensor product, namely the m-tensor product, to leverage structured equations with separated variables. The proposed method combines kernel properties along with tensor algebra to prevent explicit construction of the exponentially large feature space and tackle approximations up to hundreds of parameters while avoiding the fixed-point strategy. This is achieved by only ever employing the regression operator in a factorized form. We present this formalism along with different regularization techniques suited for low amount of data with a high number of parameters while preserving well-known matrix-based properties. We demonstrate complexity scaling on a general benchmark to show robustness for engineering problems and ease of implementation.