Singular Value Decompositions

In this activity, we will construct the singular value decomposition of $A=\begin{bmatrix} 1 & 0 & -1 \\ 1 & 1 & 1 \end{bmatrix}$. Notice that this matrix is not square so there are no eigenvalues and eigenvectors associated to it.

1. Construct the Gram matrix $G=A^TA$ and find an orthogonal diagonalization of it.

2. Identify the singular values of $A$ and the right singular vectors $\vvec_1$, $\vvec_2$, and $\vvec_3$. What is the dimension of these vectors? How many nonzero singular values a…

Let's suppose that a matrix $A$ has a singular value decomposition $A=U\Sigma V^T$ where

1. What is the shape of $A$; that is, how many rows and columns does $A$ have?

2. Suppose we write a three-dimensional vector $\xvec$ as a linear combination of right singular vectors: We would like to find an expression for $A\xvec$.

   To begin, $V^T\xvec = \threevec{\vvec_1\cdot\xvec} {\vvec_2\cdot\xvec} {\vvec_3\cdot\xvec} = \threevec{c_1}{c_2}{c_3}$.

   Now $\Sigma V^T \xvec = \begin{bmatrix} 20 & 0 & 0 \\ 0 & 5 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}\threevec{c_1}{c_2}{c_3} = \cfourvec{20c_1}{5c_2}00$.

   And finally, $A\xvec = U\Sigma V^T\xvec = \begin{bmatrix} \uvec_1 & \uvec_2 & \uvec_3 & \uvec_4 \end{bmatrix} \cfourvec{20c_1}{5c_2}00 = 20c_1\uvec_1 + 5c_2\uvec_2$.

   To summarize, we have $A\xvec = 20c_1\uvec_1 + 5c_2\uvec_2$.

   What condition on $c_1$, $c_2$, and $c_3$ must be satisfied if $\xvec$ is a s…

Suppose we have a singular value decomposition $A = U\Sigma V^T$ where

1. What is the shape of $A$? What is $\rank(A)$?

2. Identify bases for $\col(A)$ and $\col(A^T)$.

3. Explain why

4. Explain why

5. If $A = U\Sigma V^T$, explain why $A=U_r\Sigma_rV_r^T$ where the columns of $U_r$ are an orthonormal basis for $\col(A)$, $\Sigma_r$ is a square, diagonal, invertible matrix, and the columns of $V_r$ form an orthonormal basis for $\col(A^T)$.

Consider the matrix $A = \begin{bmatrix} 1 & 2 & 1 \\ 0 & -1 & 2 \\ \end{bmatrix}$.

1. Find the Gram matrix $G=A^TA$ and use it to find the singular values and right singular vectors of $A$.

2. Find the left singular vectors.

3. Form the matrices $U$, $\Sigma$, and $V$ and verify that $A=U\Sigma V^T$.

4. What is $\rank(A)$ and what does this say about $\col(A)$?

5. Determine an orthonormal basis for $\nul(A)$.

Find singular value decompositions for the following matrices:

1. $\begin{bmatrix} 0 & 0 \\ 0 & -8 \end{bmatrix}$.

2. $\begin{bmatrix} 2 & 3 \\ 0 & 2 \end{bmatrix}$.

3. $\begin{bmatrix} 4 & 0 & 0 \\ 0 & 0 & 2 \end{bmatrix}$

4. $\begin{bmatrix} 4 & 0 \\ 0 & 0 \\ 0 & 2 \end{bmatrix}$

Consider the matrix $A = \begin{bmatrix} 2 & 1 \\ 1 & 2 \end{bmatrix}$.

1. Find a singular value decomposition of $A$ and verify that it is also an orthogonal diagonalization of $A$.

2. If $A$ is a symmetric, positive semidefinite matrix, explain why a singular value decomposition of $A$ is an orthogonal diagonalization of $A$.

Suppose that the matrix $A$ has the singular value decomposition

1. What are the dimensions of $A$?

2. What is $\rank(A)$?

3. Find orthonormal bases for $\col(A)$, $\nul(A)$, $\col(A^T)$, and $\nul(A^T)$.

4. Find the orthogonal projection of $\bvec=\fourvec102{-1}$ onto $\col(A)$.

Consider the matrix $A = \begin{bmatrix} 1 & 0 & -1 \\ 2 & 2 & 0 \\ -1 & 1 & 2\\ \end{bmatrix}$.

1. Construct the Gram matrix $G$ and use it to find the singular values and right singular vectors $\vvec_1$, $\vvec_2$, and $\vvec_3$ of $A$. What are the matrices $\Sigma$ and $V$ in a singular value decomposition?

2. What is $\rank(A)$?

3. Find as many left singular vectors $\uvec_j$ as you can using the relationship $A\vvec_j=\sigma_j\uvec_j$.

4. Find an orthonormal basis for $\nul(A^T)$ and use it to construct the matrix $U$ so that $A=U\Sigma V^T$.

5. State an orthonormal basis for $\nul(A)$ and an…

Suppose that $A$ is a square $m\times m$ matrix with singular value decomposition $A=U\Sigma V^T$.

1. If $A$ is invertible, find a singular value decomposition of $A^{-1}$.

2. What condition on the singular values must hold for $A$ to be invertible?

3. How are the singular values of $A$ and the singular values of $A^{-1}$ related to one another?

4. How are the right and left singular vectors of $A$ related to the right and left singular vectors of $A^{-1}$?

1. If $Q$ is an orthogonal matrix, remember that $Q^TQ=I$. Explain why $\det Q = \pm 1$.

2. If $A=U\Sigma V^T$ is a singular value decomposition of a square matrix $A$, explain why $|\det A|$ is the product of the singular values of $A$.

3. What does this say about the singular values of $A$ if $A$ is invertible?

If $A$ is a matrix and $G=A^TA$ its Gram matrix, remember that

1. For a general matrix $A$, explain why the eigenvalues of $G$ are nonnegative.

2. Given a symmetric matrix $A$ having an eigenvalue $\lambda$, explain why $\lambda^2$ is an eigenvalue of $G$.

3. If $A$ is symmetric, explain why the singular values of $A$ equal the absolute value of its eigenvalues: $\sigma_j = |\lambda_j|$.

Determine whether the following statements are true or false and explain your reasoning.

1. If $A=U\Sigma V^T$ is a singular value decomposition of $A$, then $G=V(\Sigma^T\Sigma)V^T$ is an orthogonal diagonalization of its Gram matrix.

2. If $A=U\Sigma V^T$ is a singular value decomposition of a rank 2 matrix $A$, then $\vvec_1$ and $\vvec_2$ form an orthonormal basis for the column space $\col(A)$.

3. If $A$ is a symmetric matrix, then its set of singular values is the same as its set of…

Suppose that $A=U\Sigma V^T$ is a singular value decomposition of the $m\times n$ matrix $A$. If $\sigma_1,\ldots,\sigma_r$ are the nonzero singular values, the general form of the matrix $\Sigma$ is

1. If you know that the columns of $A$ are linearly independent, what more can you say about the form of $\Sigma$?

2. If you know that the columns of $A$ span $\real^m$, what more can you say about the form of $\Sigma$?

3. If you know that the columns of $A$ are linearly independent and span $\real^m$, w…