Principal Component Analysis

Suppose that we work with a dataset having 100 five-dimensional data points. The demeaned data matrix $A$ is therefore $5\times100$ and leads to the covariance matrix $C=\frac1{100}~AA^T$, which is a $5\times5$ matrix. Because $C$ is symmetric, the Spectral Theorem tells us it is orthogonally diagonalizable so suppose that $C = QDQ^T$ where

1. What is $V_{\uvec_2}$, the variance in the $\uvec_2$ direction?

2. Find the variance of the data projected onto the line defined by $\uvec_4$.…

Suppose that and that we have two datasets, one whose covariance matrix is $C_1 = QD_1Q^T$ and one whose covariance matrix is $C_2 = QD_2Q^T$. For each dataset, find

1. the total variance.

2. the fraction of variance represented by the first principal component.

3. a verbal description of how the demeaned data points appear when plotted in the plane.

Suppose that a dataset has mean $\threevec{13}{5}{7}$ and that its associated covariance matrix is $C=\begin{bmatrix} 275 & -206 & 251 \\ -206 & 320 & -206 \\ 251 & -206 & 275 \end{bmatrix}$.

1. What fraction of the variance is represented by the first two principal components?

2. If $\threevec{30}{-3}{26}$ is one of the data points, find the coordinates when the demeaned point is projected into the plane defined by the first two principal components.

3. If a projected data point has coordinates $\twovec{12}{-25}$, find an estimate for the original data p…

Determine whether the following statements are true or false and explain your thinking.

1. If the eigenvalues of the covariance matrix are $\lambda_1$, $\lambda_2$, and $\lambda_3$, then $\lambda_3$ is the variance of the demeaned data points when projected on the third principal component $\uvec_3$.

2. Principal component analysis always allows us to construct a smaller dimensional representation of a dataset without losing any information.

3. If the eige…