Using Singular Value Decompositions

Consider the equation $A\xvec=\bvec$ where

1. Find a singular value decomposition for $A$ using the Sage cell below. What are singular values of $A$?

2. What is $r$, the rank of $A$? How can we identify an orthonormal basis for $\col(A)$?

3. Form the reduced singular value decomposition $U_r\Sigma_rV_r^T$ by constructing: the matrix $U_r$, consisting of the first $r$ columns of $U$; the matrix $V_r$, consisting of the first $r$ columns of $V$; and $\Sigma_r$, a square $r\times r$ diagona…

Suppose that

1. Find the singular values of $A$. What is $\rank(A)$?

2. Find the sequence of matrices $A_1$, $A_2$, $A_3$, and $A_4$ where $A_k$ is the rank $k$ approximation of $A$.

Suppose we would like to find the best quadratic function fitting the points

1. Set up a linear system $A\xvec = \bvec$ describing the coefficients $\xvec = \threevec{\beta_0}{\beta_1}{\beta_2}$.

2. Find the singular value decomposition of $A$.

3. Use the singular value decomposition to find the least-squares approximate solution $\xhat$.

Remember that the outer product of two vectors $\uvec$ and $\vvec$ is the matrix $\uvec~\vvec^T$.

1. Suppose that $\uvec = \twovec{2}{-3}$ and $\vvec=\threevec201$. Evaluate the outer product $\uvec~\vvec^T$. To get a clearer sense of how this works, perform this operation without using technology.

   How is each of the columns of $\uvec~\vvec^T$ related to $\uvec$?

2. Suppose $\uvec$ and $\vvec$ are general vectors. What is $\rank(\uvec~\vvec^T)$ and what is a basis for its column space $\col(\uvec~\vvec^T)$?

3. Suppose that $\uvec$ is a unit vector. What is the effect…

Suppose that $A=U_r\Sigma_rV_r^T$ is a reduced singular value decomposition of the $m\times n$ matrix $A$. The matrix $A^+ = V_r\Sigma_r^{-1}U_r^T$ is called the Moore-Penrose inverse of $A$.

1. Explain why $A^+$ is an $n\times m$ matrix.

2. If $A$ is an invertible, square matrix, explain why $A^+=A^{-1}$.

3. Explain why $AA^+\bvec=\bhat$, the orthogonal projection of $\bvec$ onto $\col(A)$.

4. Explain why $A^+A\xvec=\xhat$, the orthogonal projection of $\xvec$ onto $\col(A^T)$.