Quadratic forms

Let's look at some more examples of quadratic forms.

1. Consider the symmetric matrix $D = \begin{bmatrix} 3 & 0 \\ 0 & -1 \\ \end{bmatrix}$. Write the quadratic form $q_D(\xvec)$ defined by $D$ in terms of the components of $\xvec=\twovec{x_1}{x_2}$. What is the value of $q_D\left(\twovec2{-4}\right)$?

2. Given the symmetric matrix $A=\begin{bmatrix} 2 & 5 \\ 5 & -3 \end{bmatrix}$, write the quadratic form $q_A(\xvec)$ defined by $A$ and evaluate $q_A\left(\twovec{2}{-1}\right)$.

3. Suppose that $q\left(\twovec{x_1}{x_2}\right) = 3x_1^2 - 4x_1x_2 + 4x_2^2$. Find a symmetric matrix $A$ such that $q$ is the quadratic form defined by $A$.

4. Suppose that $q$ is a quadratic form and that $q(\xvec) = 3$…

This activity explores the relationship between the eigenvalues of a symmetric matrix and its definiteness.

1. Consider the diagonal matrix $D=\begin{bmatrix} 4 & 0 \\ 0 & 2 \\ \end{bmatrix}$ and write its quadratic form $q_D(\xvec)$ in terms of the components of $\xvec=\twovec{x_1}{x_2}$. How does this help you decide whether $D$ is positive definite or not?

2. Now consider $D=\begin{bmatrix} 4 & 0 \\ 0 & 0 \\ \end{bmatrix}$ and write its quadratic form $q_D(\xvec)$ in terms of $x_1$ and $x_2$. What can you say about the definiteness of $D$?

3. If $D$ is a diagonal…

Suppose that $A = \begin{bmatrix} 4 & 2 \\ 2 & 7 \end{bmatrix}$.

1. Find an orthogonal diagonalization of $A$.

2. Evaluate the quadratic form $q_A\left(\twovec11\right)$.

3. Find the unit vector $\uvec$ for which $q_A(\uvec)$ is as large as possible. What is the value of $q_A(\uvec)$ in this direction?

4. Find the unit vector $\uvec$ for which $q_A(\uvec)$ is as small as possible. What is the value of $q_A(\uvec)$ in this direction?

Consider the quadratic form

1. Find a matrix $A$ such that $q(\xvec) = \xvec^TA\xvec$.

2. Find the maximum and minimum values of $q(\uvec)$ among all unit vectors $\uvec$ and describe the directions in which they occur.

Suppose that $A$ is a demeaned data matrix:

1. Find the covariance matrix $C$.

2. What is the variance of the data projected onto the line defined by $\uvec=\twovec{1/\sqrt{2}}{1/\sqrt{2}}$.

3. What is the total variance?

4. In which direction is the variance greatest and what is the variance in this direction?

Consider the matrix $A = \begin{bmatrix} 4 & -3 & -3 \\ -3 & 4 & -3 \\ -3 & -3 & 4 \\ \end{bmatrix}$.

1. Find $Q$ and $D$ such that $A=QDQ^T$.

2. Find the maximum and minimum values of $q(\uvec) = \xvec^TA\xvec$ among all unit vectors $\uvec$.

3. Describe the direction in which the minimum value occurs. What can you say about the direction in which the maximum occurs?

Consider the matrix $B = \begin{bmatrix} -2 & 1 \\ 4 & -2 \\ 2 & -1 \\ \end{bmatrix}$.

1. Find the matrix $A$ so that $q\left(\twovec{x_1}{x_2}\right) = \len{B\xvec}^2=q_A(\xvec)$.

2. Find the maximum and minimum values of $q(\uvec)$ among all unit vectors $\uvec$ and describe the directions in which they occur.

3. What does the minimum value of $q(\uvec)$ tell you about the matrix $B$?

Consider the quadratic form

1. What can you say about the definiteness of the matrix $A$ that defines the quadratic form?

2. Find a matrix $Q$ so that the change of coordinates $\yvec = Q^T\xvec$ transforms the quadratic form into one that has no cross terms. Write the quadratic form in terms of $\yvec$.

3. What are the maximum and minimum values for $q(\uvec)$ among all unit vectors $\uvec$?

Explain why the following statements are true.

1. Given any matrix $B$, the matrix $B^TB$ is a symmetric, positive semidefinite matrix.

2. If both $A$ and $B$ are symmetric, positive definite matrices, then $A+B$ is a symmetric, positive definite matrix.

3. If $A$ is a symmetric, invertible, positive definite matrix, then $A^{-1}$ is also.

Determine whether the following statements are true or false and explain your reasoning.

1. If $A$ is an indefinite matrix, we can't know whether it is positive definite or not.

2. If the smallest eigenvalue of $A$ is 3, then $A$ is positive definite.

3. If $C$ is the covariance matrix associated with a dataset, then $C$ is positive semidefinite.

4. If $A$ is a symmetric $2\times2$ matrix and the maximum and minimum values of $q_A(\uvec)$ occur…

Consider the function $f(x,y,z) = x^4 + y^4 +z^4 - 4xyz$.

1. Show that $f$ has a critical point at $(-1,1,-1)$ and construct the Hessian $H$ at that point.

2. Find the eigenvalues of $H$. Is this a definite matrix of some kind?

3. What does this imply about whether $(-1,1,-1)$ is a local maximum or minimum?