Reading Multiple Regression from an Ellipsoid Inscribed in the Unit Hypercube

Abstract

Consider the linear regression of a response $Y$ on explanatory variables $X_1,\dots,X_p$ with a positive definite joint correlation matrix. Let $\mathbf{R}$ be the correlation matrix of the explanatory variables and $\mathbf{r}$ the vector of their correlations with $Y$. The ellipsoid $E=\{\mathbf{x} : \mathbf{x}^{\top}\mathbf{R}^{-1}\mathbf{x}=1\}$ is inscribed in the cube $[-1,1]^p$ and touches the face $x_j=1$ at $T_j$, the $j$th column of $\mathbf{R}$. The point $P=\mathbf{r}$ lies inside $E$, and three kinds of regression quantities are read off this diagram by elementary geometric constructions. The multiple correlation is $|OP|/|OP'|$, where $O$ is the origin and $P'$ is the point where the ray from $O$ through $P$ meets $E$ (for $P\ne O$). The standardized regression coefficients are the coordinates of $P$ in the basis $T_1,\dots,T_p$. The partial correlation between $X_i$ and $Y$ given the other explanatory variables is the position of $P$ on the chord of $E$ through $P$ parallel to the $x_i$-axis, on a linear scale from $-1$ to $1$ increasing with $x_i$; furthermore, the parallel chord through the origin has half-length $1/\sqrt{\mathrm{VIF}_i}$. Examples illustrate sign reversal and standardized coefficients outside the interval $[-1,1]$.