Activity 7.4.3

In this activity, we will construct the singular value decomposition of $A=\begin{bmatrix} 1 & 0 & -1 \\ 1 & 1 & 1 \end{bmatrix}$. Notice that this matrix is not square so there are no eigenvalues and eigenvectors associated to it.

1. Construct the Gram matrix $G=A^TA$ and find an orthogonal diagonalization of it.

2. Identify the singular values of $A$ and the right singular vectors $\vvec_1$, $\vvec_2$, and $\vvec_3$. What is the dimension of these vectors? How many nonzero singular values are there?

3. Find the left singular vectors $\uvec_1$ and $\uvec_2$ using the fact that $A\vvec_i = \sigma_i\uvec_i$. What is the dimension of these vectors? What happens if you try to find a third left singular vector $\uvec_3$ in this way?

4. As before, form the orthogonal matrices $U$ and $V$ from the left and right singular vectors. What are the shapes of $U$ and $V$? How do these shapes relate to the number of rows and columns of $A$?

5. Now form $\Sigma$ so that it has the same shape as $A$: $$\Sigma = \begin{bmatrix} \sigma_1 & 0 & 0 \\ 0 & \sigma_2 & 0 \end{bmatrix}$$ and verify that $A = U\Sigma V^T$.

6. How can you use this singular value decomposition of $A=U\Sigma V^T$ to easily find a singular value decomposition of $A^T=\begin{bmatrix} 1 & 1 \\ 0 & 1 \\ -1 & 1 \\ \end{bmatrix}$?