Masked Toeplitz covariance estimation

Abstract

The problem of estimating the covariance matrix $Σ$ of a $p$-variate distribution based on its $n$ observations arises in many data analysis contexts. While for $n>p$, the classical sample covariance matrix $\hatΣ_n$ is a good estimator for $Σ$, it fails in the high-dimensional setting when $n\ll p$. In this scenario one requires prior knowledge about the structure of the covariance matrix in order to construct reasonable estimators. Under the common assumption that $Σ$ is sparse, a refined estimator is given by $M\cdot\hatΣ_n$, where $M$ is a suitable symmetric mask matrix indicating the nonzero entries of $Σ$ and $\cdot$ denotes the entrywise product of matrices. In the present work we assume that $Σ$ has Toeplitz structure corresponding to stationary signals. This suggests to average the sample covariance $\hatΣ_n$ over the diagonals in order to obtain an estimator $\tildeΣ_n$ of Toeplitz structure. Assuming in addition that $Σ$ is sparse suggests to study estimators of the form $M\cdot\tildeΣ_n$. For Gaussian random vectors and, more generally, random vectors satisfying the convex concentration property, our main result bounds the estimation error in terms of $n$ and $p$ and shows that accurate estimation is indeed possible when $n \ll p$. The new bound significantly generalizes previous results by Cai, Ren and Zhou and provides an alternative proof. Our analysis exploits the connection between the spectral norm of a Toeplitz matrix and the supremum norm of the corresponding spectral density function.