A Generating Polynomial Based Two-Stage Optimization Method for Tensor Rank Decomposition

Abstract

The tensor rank decomposition, also known as canonical polyadic(CP) or simply tensor decomposition, has a long history in multilinear algebra. However, computing a rank decomposition becomes particularly challenging when the rank lies between its largest and second-largest dimensions. Moreover, for high-order tensor decompositions, a common approach is to first find a decomposition of its flattening order-3 tensor, where a significant gap often exists between the largest and the second-largest dimension, also making this case crucial in practice. For such a case, traditional optimization methods, such as the nonlinear least squares or alternating least squares methods, often fail to produce correct tensor decompositions. There are also direct methods that solve tensor decompositions algebraically. However, these methods usually require the tensor decomposition to be unique and can be computationally expensive, especially when the tensor rank is high. This paper introduces a new generating polynomial (GP) based two-stage algorithm for finding the order-3 nonsymmetric tensor decomposition, even when the tensor decomposition is not unique, assuming the rank does not exceed the largest dimension. The proposed method reformulates the tensor decomposition problem into two sequential optimization problems. Notably, if the first-stage optimization yields a partial solution, it will be effectively utilized in the second stage. We establish the theoretical equivalence between the CP decomposition and the global minimizers of those two-stage optimization problems. Numerical experiments demonstrate that our approach is very efficient and robust, capable of finding tensor decompositions in scenarios where the current state-of-the-art methods often fail.