Covariance Identification from Regression Residuals

Abstract

Regression residuals are projections of the errors, not observations of them. This paper characterizes what residuals identify about the error covariance. Residualization acts on the covariance through a known linear map, and under an unrestricted covariance model the directions it destroys are exactly those generated by the column space of the regressors. A linear covariance functional is identified if and only if it does not load on these lost directions, and restricting the covariance model, for instance to diagonal or clustered form, shrinks them. The central result separates projection loss from moment-selection loss: projecting the residual covariance onto the maintained covariance structure loses no identifying information, so cross-cluster, long-lag, and long-distance residual products add no identifying directions. They can still improve conditioning, since restricting to the aligned moments squares the condition number. This equivalence relies on the orthogonal-projection geometry of OLS, and generally fails for non-orthogonal transformation. Quadratic statistics in the outcome that are free of the regression coefficients additionally identify the covariance between the fitted and residual components, but not the covariance within the span of the regressors. Because the OLS sandwich variance depends only on that block, it is not identified by any such statistic under an unrestricted covariance model; robust variance estimation draws its identification from maintained restrictions on the covariance structure.