Orthogonal least squares

Suppose we write the linear system as $A\xvec=\bvec$.

1. Find an orthogonal basis for $\col(A)$.

2. Find $\bhat$, the orthogonal projection of $\bvec$ onto $\col(A)$.

3. Find a solution to the linear system $A\xvec = \bhat$.

Determine whether the following statements are true or false and explain your thinking.

1. If $A\xvec=\bvec$ is consistent, then $\xhat$ is a solution to $A\xvec=\bvec$.

2. If $R^2=1$, then the least-squares approximate solution $\xhat$ is also a solution to the original equation $A\xvec=\bvec$.

3. Given the $QR$ factorization $A=QR$, we have $A\xhat=Q^TQ\bvec$.

4. A $QR$ factorization provides a method for finding the least-squares approximate solution to $A\xvec=\bvec$ that is more reliable than solving the…

Explain your response to the following questions.

1. If $\xhat=\zerovec$, what does this say about the vector $\bvec$?

2. If the columns of $A$ are orthonormal, how can you easily find the least-squares approximate solution to $A\xvec=\bvec$?

We observed that if the columns of $A$ are linearly independent, then there is a unique least-squares approximate solution to the equation $A\xvec=\bvec$ because the equation $A\xhat=\bhat$ has a unique solution. We also said that $\xhat$ is the unique solution to the normal equation $A^TA\xhat = A^T\bvec$ without explaining why this equation has a unique solution. This exercise offers an explanation.

Assuming that the columns of $A$ are linearly indepen…

This problem is about the meaning of the coefficient of determination $R^2$ and its connection to variance, a topic that appears in the next section. Throughout this problem, we consider the linear system $A\xvec=\bvec$ and the approximate least-squares solution $\xhat$, where $A\xhat=\bhat$. We suppose that $A$ is an $m\times n$ matrix, and we will denote the $m$-dimensional vector $\onevec = \fourvec11{\vdots}1$.

1. Explain why $\bbar$, the mean of the components of $\bvec$, can be found a…