Activity 7.1.5

Let's return to the dataset from the previous activity in which we have demeaned data points: $$\dtil_1=\twovec{-1}{-1},\hspace{24pt} \dtil_2=\twovec{0}{-1},\hspace{24pt} \dtil_3=\twovec{1}{2}.$$ Our goal is to compute the variance $V_{\uvec}$ in the direction defined by a unit vector $\uvec$.

To begin, form the demeaned data matrix $$A = \begin{bmatrix} \dtil_1 & \dtil_2 & \dtil_3 \end{bmatrix}$$ and suppose that $\uvec$ is a unit vector.

1. Write the vector $A^T\uvec$ in terms of the dot products $\dtil_j\cdot\uvec$.

2. Explain why $V_{\uvec} = \frac13|A^T\uvec|^2$.

3. Apply this proposition to explain why $$V_{\uvec} = \frac13|A^T\uvec|^2 = \frac13 (A^T\uvec)\cdot(A^T\uvec) = \uvec^T\left(\frac13 AA^T\right)\uvec = \uvec\cdot\left(\frac13 AA^T\right)\uvec$$

4. In general, the matrix $C=\frac1N~AA^T$ is called the covariance matrix of the dataset, and it is useful because the variance $V_{\uvec} = \uvec\cdot(C\uvec)$, as we have just seen. Find the matrix $C$ for our dataset with three points.

5. Use the covariance matrix to find the variance $V_{\uvec_1}$ when $\uvec_1=\twovec{1/\sqrt{5}}{2/\sqrt{5}}$.

6. Use the covariance matrix to find the variance $V_{\uvec_2}$ when $\uvec_2=\twovec{-2/\sqrt{5}}{1/\sqrt{5}}$. Since $\uvec_1$ and $\uvec_2$ are orthogonal, verify that the sum of $V_{\uvec_1}$ and $V_{\uvec_2}$ gives the total variance.

7. Explain why the covariance matrix $C$ is a symmetric matrix.