Let \(W
=\left\{\ba{r}x_1\\x_2\\x_3\\x_4\ea \in
\R^4\ \Bigg|\ x_1+2x_2+3x_3+4x_4=0\right\}\text{.}\)
(a)
Find bases for \(W\) and \(W^\perp.\)
Solution.
\(W\) is the solution space to \(Ax=0\) where \(A\) is the \(1\times 4\) matrix \(A =
\ba{rrrr}1\amp2\amp3\amp4\ea\text{;}\) it is a hyperplane in \(\R^4.\) We easily read a set of independent solutions from the matrix \(A\) which is already in reduced row-echelon form. Taking \(x_2, x_3, x_4\) as free variables, we may take as a basis:
One alternative is that having gone to the trouble of finding an orthogonal basis for \(W,\) we could brute force the answer from Definition 3.2.13.
Other alternatives: if we made our orthogonal basis for \(W\) into an orthonormal one, we could use Corollary 3.3.2. Or perhaps with a bit less fuss, we could simply take advantage of Proposition 3.3.3 as follows: Let
\begin{equation*}
A =
\ba{rrr}-2\amp-3\amp-4\\1\amp0\amp0\\0\amp1\amp0\\0\amp0\amp1\ea.
\end{equation*}
Note that \(v_3^\prime = \ba{r}1\\1\\1\ea\) is another obvious choice for an independent eigenvector, though not as useful for a later part (since \(v_2\) and \(v_3\) are orthogonal).
(d)
Find a matrix \(P\) so that \(P^{-1}AP\) is diagonal.
Solution.
The matrix \(P\) is any matrix with the eigenvectors as columns. For example, if we want the diagonal matrix to be
Determine whether the matrix \(A\) is orthogonally diagonalizable. If not, why; if so, find an orthogonal matrix \(Q\) so that \(Q^TAQ\) is diagonal.
Solution.
Since \(A\) is a real symmetric matrix, we know it is orthogonally diagonalizable. The columns of the matrices \(P\) above have orthogonal columns. We need only normalize the columns, say
View \(\R^7\) as an inner product space with the usual inner product.
(a)
let \(T:\R^7 \to \R^7\) be a linear map with the property that \(\la T(v), v\ra = 0\) for all \(v \in \R^7\text{.}\) Show that \(T\) is not invertible.
Hint.
Calculus tells you that a polynomial of degree 7 and real coefficients has at least one real root.
Solution.
Let \(\chi_T\) be the characteristic polynomial of \(T\text{.}\) Since the degree is odd, the hint says \(\chi_T\) has a real root, that is, \(T\) has a real eigenvalue \(\lambda\text{.}\) Let \(v\) be a (nonzero) eigenvector with \(T(v) = \lambda v\text{.}\) We now consider the requirement that \(\la T(v), v\ra = 0\text{.}\)
Since \(v \ne 0\text{,}\) we cannot have \(\la v,v\ra = 0\text{,}\) so we must have \(\lambda = 0\text{,}\) which says zero is an eigenvalue, and hence the nullspace is nontrivial. This means that \(T\) is not invertible.
(b)
Show by example that there exist linear maps \(T:\R^2 \to
\R^2\) with \(\la T(v), v\ra = 0\) for all \(v \in
\R^2\text{,}\) but with \(T\) invertible. Verify that your \(T\) satisfies the required conditions.
Hint.
If we consider the previous part, the dimension only mattered to produce a real eigenvalue, so that provides a direction to look.
Solution.
Let \([T]_\cE =
\ba{rr}
0\amp 1\\-1\amp 0\ea\) where \(\cE\) is the standard basis for \(\R^2\text{.}\) Then \(\chi_T=x^2+1\) which has no real roots. In particular, 0 is not an eigenvalue which means the nullspace is zero, so \(T\) is invertible. We claim that \(\la T(v),v\ra = 0\) for all \(v \in V\text{.}\) We can read off the action of \(T\) from the matrix:
\begin{equation*}
T(e_1) = -e_2 \text{ and } T(e_2) = e_1, \text{ so } T(ae_1 + be_2 ) = be_1 - a
e_2\text{.}
\end{equation*}
Let \(V\) be a finite-dimensional real inner product space, and \(T: V\to V\) a linear operator satisfying \(T^2 =
T\text{,}\) that is \(T(T(v)) = T(v)\) for all \(v\in V\text{.}\) To eliminate trivial situations, assume that \(T\) is neither the zero transformation, nor the identity operator.
(a)
Show that the only possible eigenvalues of \(T\) are zero and one.
Solution.
Suppose that \(T(v) = \lambda v\) for some nonzero vector \(v\text{.}\) Then
so \((\lambda-1)T(v) = 0\text{,}\) which means either \(\lambda =
1\) (so one is an eigenvalue), or \(T(v) = 0\) which means the nullspace is not zero, hence zero is an eigenvalue.
(b)
Let \(E_\lambda\) denote the \(\lambda\)-eigenspace. Show that \(E_0 = N(T)\text{,}\) the nullspace of \(T\text{,}\) and that \(E_1\) is the image of \(T.\)
Solution.
That \(E_0=N(T)\) is the definition of \(E_0 = \{v\in
V\mid T(v) = 0 = 0v\}.\)
If \(v \in E_1\text{,}\) then \(T(v) = 1\cdot v\text{,}\) but then \(T(v) = v\) which says that \(v \in R(T)\text{.}\) Conversely if \(w = T(v^\prime) \in R(T)\text{,}\) then \(T(w) = T^2(v^\prime) = T(v^\prime) = w\text{,}\) so \(w \in E_1\text{.}\) Thus the image is precisely \(E_1.\)
(c)
Show that \(T\) is diagonalizable.
Solution.
\(\dim E_0\) equals the nullity of \(T\text{,}\) and from above \(\dim E_1\) is the rank, so by rank-nullity, the sum of the sizes of the eigenspaces (which have trivial intersection) is the dimension of the space, so \(V\) has a basis of eigenvectors for \(T.\)
(d)
Let \(W\) be a subspace of \(V,\) and let \(S = \proj_W\) be the orthogonal projection onto the subspace \(W.\) Show that \(S^2 = S,\) so that the orthogonal projection is one linear map satisfying the given property.
Solution.
By definition, we take an orthonormal basis for \(W\) (say having dimension \(r\)), and extend it to an orthonomal basis \(\cB = \{v_i\}\) for \(V\text{.}\) Then \(S(v) = \sum_{i=1}^r \la v,v_i\ra v_i = w\) and by Theorem 3.2.10 we know that \(v = w^\perp + w\) for unique \(w^\perp \in W^\perp\text{.}\) Since \(S(v) = w \in W\) and \(w = w + 0\text{,}\)\(S(w) = w\) (Corollary 3.2.14), that is \(S^2(v)
= S(v)\text{.}\)
In answering the questions below, you may find some of the information below of use. By \(\rref(X)\) we mean the reduced row-echelon form of the matrix \(X\text{.}\)
We see that \(\rref(A|b)\) has a pivot in the augmented column, meaning the system is inconsistent.
(b)
Find a least squares solution to the system \(Ax=b\text{.}\)
Solution.
A least squares solution to \(Ax=b\) is obtained by solving the consistent system \(A^TAx=A^Tb\text{.}\) From the work above, we read off the solution \(x=\ba{r}74\\ -128\\ 71\ea.\)
6.
Suppose a real matrix has SVD given by \(A=U\Sigma V^T\text{:}\)
Using only your knowledge of the SVD (and no compuation), determine \(\rank A\text{.}\)
Solution.
The rank is two since there are precisely two nonzero singular values, \(\sqrt 3\) and \(\sqrt2.\)
(b)
Using only your knowledge of the SVD, give a basis for the kernel (nullspace) of \(A\text{;}\) explain your process.
Solution.
The SVD process begins by finding an orthonormal basis \(\{v_1,
\dots, v_4\}\) for \(A^TA\text{.}\) With \(\sigma_i = \|Av_i\|\) and the rank of \(A\) equaling 2, we know the nullity of \(A\) is also two, and since \(Av_3 = Av_4=0\text{,}\)\(\{v_3, v_3\}\) gives an orthogonal basis for the kernel.
(c)
Using only your knowledge of the SVD, give a basis for the column space of \(A\text{,}\) explaining your process.
Solution.
The column space is spanned by \(\{Av_1, \dots, Av_r\}\) where \(r = \rank A=2\text{,}\) so \(\{Av_1, Av_2\}\) is an (orthogonal) basis for the column space.
7.
Let \(A\) have singular value decomposition
\begin{equation*}
A = U\Sigma V^T = \ba{rr}2/\sqrt5\amp1/\sqrt5\\1/\sqrt5\amp-2/\sqrt5\ea
\ba{rr}8\amp0\\0\amp2\ea \ba{rr}1/\sqrt5\amp2/\sqrt5\\2/\sqrt5\amp-1/\sqrt5\ea.
\end{equation*}
(a)
Prove that \(A\) is invertible.
Solution.
\(A\) is a \(2\times 2\) matrix with two nonzero singular values, so has rank 2, and so is invertible. Alternatively, it is easy to show that \(\det A \ne 0\text{.}\)
(b)
Using the given SVD, find an expression for \(A^{-1}\text{.}\)
Solution.
\(A = U\Sigma V^T\) implies that \(A^{-1} =
(V^T)^{-1}\Sigma^{-1}U^{-1}= V
\ba{rr}
1/8\amp0\\0\amp1/2\ea U^T\) since both \(U\) and \(V\)are orthogonal matrices.
(c)
The goal of this part is to find an SVD for \(A^{-1}\text{.}\) You should express your answer (confidently) as an appropriate product of matrices without multiplying things out, though you should explain why the expression you write represents an SVD for \(A^{-1}\text{.}\) In particular, a couple of warm up exercises will help in this endeavor, and no, the answer in part b is not the correct answer.
First show that the product of two orthogonal matrices in \(M_n(\R)\) is orthogonal.
Next show that the diagonal matrices (with real entries) \(\ba{rr}
\lambda_1\amp0\\0\amp\lambda_2
\ea\) and \(\ba{rr}
\lambda_2\amp0\\0\amp\lambda_1
\ea\) are orthogonally equivalent, i.e., that there exists an orthogonal matrix \(P\) so that
but an explanation would be nice. It should be clear that the standard basis vectors \(e_1, e_2\) for \(\R^2\) are eigenvectors for the matrix. \(\ba{rr}
\lambda_1\amp0\\0\amp\lambda_2
\ea\text{.}\) It follows that the matrix \(P\) with columns \(e_2,
e_1\) is also a matrix of eigenvectors, but which reverses the order of appearance of the eigenvalues.
Now for the main event: The expression for \(A^{-1}\) in the previous part would be an SVD for \(A^{-1}\) but for the fact that the singular values do not satisfy \(\sigma_1>\sigma_2\text{.}\) Fortunately the warm up exercises come to the rescue! We see that
and that \(Q=\ba{rr}
0\amp1\\1\amp0
\ea\) is an orthogonal matrix with \(Q^T = Q\text{,}\) hence by the warm ups, so are the matrices \(QU^T = (UQ^T)^T= (UQ)^T\) and \(VQ\text{.}\) Thus
