Activity 7.3.2

Suppose that we work with a dataset having 100 five-dimensional data points. The demeaned data matrix $A$ is therefore $5\times100$ and leads to the covariance matrix $C=\frac1{100}~AA^T$, which is a $5\times5$ matrix. Because $C$ is symmetric, the Spectral Theorem tells us it is orthogonally diagonalizable so suppose that $C = QDQ^T$ where $$Q = \begin{bmatrix} \uvec_1 & \uvec_2 & \uvec_3 & \uvec_4 & \uvec_5 \end{bmatrix},\hspace{24pt} D = \begin{bmatrix} 13 & 0 & 0 & 0 & 0 \\ 0 & 10 & 0 & 0 & 0 \\ 0 & 0 & 2 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \end{bmatrix}.$$

1. What is $V_{\uvec_2}$, the variance in the $\uvec_2$ direction?

2. Find the variance of the data projected onto the line defined by $\uvec_4$. What does this say about the data?

3. What is the total variance of the data?

4. Consider the 2-dimensional subspace spanned by $\uvec_1$ and $\uvec_2$. If we project the data onto this subspace, what fraction of the total variance is represented by the variance of the projected data?

5. How does this question change if we project onto the 3-dimensional subspace spanned by $\uvec_1$, $\uvec_2$, and $\uvec_3$?

6. What does this tell us about the data?