In Proposition 3.2.16, we have seen how a complex \(m\times n\) matrix and its conjugate transpose have a natural relation with respect to inner products, and in Subsection 3.2.5 took a first look at the four fundamental subspaces. In this section we develop the corresponding notions for linear maps between inner product spaces.
Subsection3.5.1Basic Properties
Let \(V,W\) be inner product spaces and \(T:V\to W\) be a linear map. We can ask if there exists a linear map \(S:W\to
V\) so that
If \(A\) is an \(m\times n\) complex matrix, then \(T(x) = Ax\) defines a linear transformation \(T:\C^n \to
\C^m\text{.}\) In Proposition 3.2.16, we saw that the linear map \(S:\C^m \to \C^n\) given by \(S(X) = A^* x\) satisfies the requisite property that
Let \(V\) be an inner product space and \(W\) a subspace with orthonormal basis \(\{w_1, \dots, w_r\}.\) As we have seen in Subsection 3.2.4, the orthogonal projection of \(V\) onto \(W\) is given by
By Theorem 3.2.10 and Corollary 3.2.14, the projection map satisfies \(v^\perp := v - \proj_W(v) \in W^\perp\) and \(\proj_W^2 =
\proj_W\text{.}\) We wish to show that the projection is self-adjoint, that is,
Let \(V\) be a finite-dimensional inner product space, \(u\) a unit vector, and \(W\) the hyperplane (through the origin) normal to \(u.\) Geometrically, we want to reflect a vector \(v\) across the hyperplane \(W.\) One way to describe this is to write \(v = \proj_W v + v^\perp\) and define \(H(v) = \proj_W v -
v^\perp\text{,}\) but we recognize that \(v^\perp = \la v,u\ra
u,\) so we can simply write
\begin{equation*}
H(v) = v- 2\la v,u\ra u.
\end{equation*}
The map \(H\) is often called a Householder transformation. We show that it too is self-adjoint.
Proof.
As before, we compute both sides of the desired equality: \(\la Hv,z\ra = \la v,Hz\ra\) and show they are equal.
\begin{equation*}
\la v , (S-S')w\ra = 0
\end{equation*}
for all \(v,w\text{,}\) which implies \(S=S'.\)
We denote the unique adjoint of the linear map \(T\) as \(T^*\text{.}\) As a consequence of uniqueness it is immediate to check that
\begin{equation*}
(\lambda T)^* = \overline\lambda T^*, (S+T)^* =
S^* + T^*. \text{ and } (T^*)^* = T.
\end{equation*}
If \(V\) is a finite-dimensional inner product space, it is easy to show that every linear map \(T:V\to W\) has an adjoint.
Proposition3.5.4.
Let \(V\) be a finite-dimensional inner product space with orthonormal basis \(\{e_1, \dots, e_n\}.\) Then the adjoint map \(T^*:W \to V\) is linear and given by
It follows from this definition and properties of the inner product that \(T^*\) is linear.
As a means of connecting this notion of adjoint with the properties of the conjugate transpose of a matrix given in Proposition 3.2.16, we have the following proposition.
Proposition3.5.5.
Let \(V,W\) be finite-dimensional inner product spaces with orthonormal bases \(\cB_V\) and \(\cB_W.\) Then the matrix of the adjoint \(T^*\) of a linear map \(T:V\to W\) is the conjugate transpose of the matrix of \(T,\) precisely
Let the orthonormal bases be given by \(\cB_V = \{e_1, \dots, e_n\}\) and \(\cB_W =
\{f_1, \dots, f_m\}\text{.}\) If \(A = [T]^{\cB_W}_{\cB_V}\) and \(B=[T^*]_{\cB_W}^{\cB_V}\) then by Theorem 3.2.3, \(B_{ij} = \la T^*(f_j),e_i\ra\) and
Subsection3.5.2A second look at the four fundamental subspaces
In the previous section, we established the existence and uniqueness of the adjoint of a linear map defined on a finite-dimensional inner product space, and connections with the matrix of the linear transformation. Here we look at a few more properties including a second look at the four fundamental subspaces which lie at the heart of the singular value decomposition.
A linear operator \(T:V \to V\) on an inner product space is called self-adjoint or Hermitian if \(T^* =
T.\) We saw that both the Householder transformation and the orthogonal projection were examples of self-adjoint operators.
Proposition3.5.6.
Suppose that \(U,V,W\) are finite-dimensional inner product spaces, and \(S: U \to V\) and \(T:V \to W\) are linear maps. Then
\(\displaystyle S^*T^* = (TS)^*\)
In particular, if \(T\) is invertible, then so is \(T^*\text{,}\) and \((T^*)^{-1} = (T^{-1})^*.\)
Proof.
For all \(u \in U\) and \(w\in W\) we have
\begin{equation*}
\la u, S^*T^*w\ra = \la Su, T^*w\ra = \la TS(u),w\ra = \la
u,(TS)^*w\ra
\end{equation*}
which yields \(S^*T^* = (TS)^*\text{.}\)
For the second, it is immediate that the adjoint of the identity map is again an identity map. As a consequence (and using the first part),
which together with the identity with the operators reversed gives the result. Actually, being operators on finite dimensional vector spaces, one such identity yields the other by rank-nullity.
Another important class of linear map between inner product spaces is the notion of an isometry, a linear map \(T:V\to
W\) which satisfies
\begin{equation*}
\la Tv_1, T v_2\ra_W = \la v_1, v_2\ra_V
\end{equation*}
for all \(v_1, v_2 \in V.\)
Proposition3.5.7.
If \(T:V\to W\) is an isometry, then \(T\) is injective. Moreover, if \(T\) is surjective, then \(T^*=T^{-1}\text{.}\)
Proof.
To show that \(T\) is injective, we note that the kernel is trivial: If \(T(v) = 0\text{,}\) then
so by uniqueness of the adjoint, \(T^* = T^{-1}.\)
The following theorem should be compared to Theorem 3.2.17 and its corollary.
Theorem3.5.8.
Let \(V,W\) be finite dimensional inner product spaces and \(T:V\to W\) a linear map. Then
\(\ker T^* = (\Im T)^\perp\text{,}\)
\(\Im(T^*) = (\ker T)^\perp\text{.}\)
Proof.
Let \(w \in \ker(T^*)\text{.}\) Then \(T^*(w)=0\text{,}\) so \(\la v, T^*(w)\ra = 0 = \la T(v),w\) for all \(v\in V\text{.}\) Thus \(w\) is orthogonal to the image of \(T\text{,}\) i.e., \(\ker(T^*) \subseteq (\Im T)^\perp.\) Conversely, if \(w\in (\Im T)^\perp\text{,}\) then for all \(v\in
V\text{,}\)
In particular, choosing \(v=T^*(w)\) shows that \(T^*(w)=0\text{,}\) hence \((\Im T)^\perp \subseteq \ker T^*,\) giving the first equality.
Since the first statement is true for any linear map and finite-dimensional inner product spaces, we replace \(T\) by \(T^*\) and use \((T^*)^* = T\) to conclude
\begin{equation*}
\ker T = (\Im T^*)^\perp\text{.}
\end{equation*}