Singular value decompositions over $\ast$-algebras with applications to tensor decompositions

Abstract

The singular value decomposition (SVD) provides optimal low-rank approximations of real and complex matrices by the Eckart--Young theorem, and SVD-based tensor decomposition algorithms such as the higher-order singular value decomposition (HOSVD) and the tensor train singular value decomposition (TT-SVD) inherit quasi-optimality guarantees from it. We extend the SVD to matrices over a finite-dimensional $\ast$-algebra $K$. Our main contribution is a characterization of the algebras $K$ for which every matrix admits an SVD, and of the inner products on $K$ for which the Eckart-Young theorem holds. We introduce a notion of ordered SVD, show that ordered SVDs reveal the rank of a matrix over $K$ and yield optimal truncations, and give an explicit algorithm that computes an ordered SVD from the structure of the algebra. We then extend the Tucker, tensor train, and tree tensor network (TTN) decompositions and their SVD-based decomposition algorithms to tensors over $K$, and show that TT-SVD retains a quasi-optimality guarantee over every such $K$, whereas HOSVD, ST-HOSVD, and TTN-SVD do so when $K$ is commutative. Viewed in real or complex coordinates, these constructions give novel decomposition formats of ordinary tensors. We illustrate our new methods by compressing data from a variety of scientific applications, and show that they can achieve lower approximation error than their classical counterparts at equal storage.