LAR Inner Product Spaces

Section 3.1 Inner Product Spaces

While a great deal of linear algebra applies to all vector spaces, by restricting attention to those with some notion of distance and orthogonality, we can go much further.

Subsection 3.1.1 Definitions and examples

Our discussion of inner product spaces will generally restrict to the setting of a vector space over a field \(F\) being either the real or complex numbers.

Recall the axioms of an inner product. They are often paraphrased with higher level concepts. For example, the first two axioms combined says that the inner product is linear in the first variable (with the second variable held constant). What that means is that if we fix a vector \(w\in V\) and define \(T:V\to V\) by \(T(v) = \la v,w\ra\text{,}\) then \(T\) is a linear operator on \(V.\)

Remark 3.1.1.

We note that the third axiom tells us that the inner product is conjugate linear in the second variable (or that the function of two variables,\(\la \cdot,\cdot\ra\text{,}\) is sesquilinear). Using the first three axioms, if we fix \(v\in V\text{,}\) and define \(S:V\to V\) by \(S(w) := \la v,w\ra,\) we observe

\begin{align*} S(u+w) &= \la v, u+w\ra = \overline{\la u+w,v\ra} = \overline{\la u,v\ra + \la w,v\ra}\\ &=\overline{\la u,v\ra} + \overline{\la w,v\ra}= \la v,u\ra + \la v,w\ra = S(u) + S(v), \end{align*}

and

\begin{gather*} S(\lambda u) = \la v, \lambda u\ra = \overline{\la \lambda u,v\ra} = \overline{\lambda\la u,v\ra} = \overline\lambda \la v,u\ra = \overline\lambda S(u), \end{gather*}

hence the term conjugate linear.

Remark 3.1.2.

We also note that if we are dealing with a real inner product space (i.e., \(F=\R\)), then the inner product is linear in both variables leading mathematicians to call it bilinear, that is linear in each variable while holding the other fixed.

Remark 3.1.3.

An inner product on a vector space \(V\) will give us a notion of when two vectors are orthogonal. The positivity condition on an inner product (\(\la v, v\ra >0\) unless \(v=0)\) gives us a notion of length. We define the norm of a vector \(v\in V\) by

\begin{equation*} \|v\|:= \sqrt{\la v,v\ra}. \end{equation*}

First we assemble a collection of inner products, and their norms.

Example 3.1.4. \(V=F^n\).

Let \(v=(a_, \dots, a_n), w=(b_1, \dots, b_n)\in F^n\) (written as row vectors). Define

\begin{equation*} \la v,w\ra := \sum_{i=1}^n a_i \overline {b_i}. \end{equation*}

This inner product is called the standard inner product on \(F^n.\) When \(F=\R,\) this is the usual dot product.

If \(v=(a_, \dots, a_n)\text{,}\) we see that

\begin{equation*} \|v\| = \la v,v\ra = \sqrt {\sum_{i=1}^n a_i \overline a_i} = \sqrt{\sum_{i=1}^n|a_i|^2} \end{equation*}

Example 3.1.5. \(V=M_{m\times n}(\C)\).

Let \(A,B \in V=M_{m\times n}(\C).\) Define the Frobenius inner product of \(A\) and \(B\) by

\begin{equation*} \la A,B\ra := \tr(AB^*) = \tr(B^*A), \end{equation*}

where \(B^*\) is the conjugate transpose of \(B,\) and \(\tr\) is the trace of the matrix.

Here the norm is \(\|A\| = \sqrt{\tr(A^* A)}.\)

Example 3.1.6. \(V=C([0,1])\).

Let \(V=C([0,1])\) be the set of real-valued continuous functions defined on the interval \([0,1]\text{.}\) For \(f,g \in C([0,1])\text{,}\) define their inner product on \(V\) by:

\begin{equation*} \la f, g\ra := \int_0^1 f(t) g(t)\, dt. \end{equation*}

If instead \(f\) and \(g\) are complex-valued, then the inner product becomes:

\begin{equation*} \la f, g\ra := \int_0^1 f(t) \overline{g(t)}\, dt. \end{equation*}

Here the norm is \(\|f\| := \sqrt{\int_0^1 f(t)\overline{f(t)}}=\sqrt{\int_0^1 |f(t)|^2}\text{,}\) where \(|\cdot|\) is the usual absolute value on the complex numbers.

If \((V, \la \cdot, \cdot\ra)\) is an inner product space, we say that

\(u,v \in V\) are orthogonal if \(\la u,v\ra = 0.\)
Two subsets \(S,T \subseteq V\) are orthogonal if \(\la u,v\ra = 0\) for every \(u \in S\) and \(v \in T.\)
\(v \in V\) is a unit vector if \(\|v\| = 1\text{.}\)

Subsection 3.1.2 Basic Properties

We list some basic properties of inner products and their norms which can be found in any of the standard references.

Let \(V\) be an inner product space with inner product \(\la \cdot,\cdot\ra\) and norm \(\|\cdot\|.\)

Theorem 3.1.7.

For all \(u,v,w \in V\) and \(\lambda \in F\)

If \(\la v,u\ra = \la v,w\ra\) for all \(v \in V,\) then \(u=w.\)
\(\| \lambda v\| = |\lambda |\, \|v\|\text{.}\)
\(\|v\| \ge 0\) for all \(v,\) and \(\|v\| = 0\) if and only if \(v=0.\)
(Cauchy-Schwarz Inequality): \(|\,\la u,v\ra\,| \le \|u\|\, \|v\|.\)
(Triangle inequality): \(\| u + v \| \le \|u\| + \|v \|\text{.}\)
(Pythagorean theorem) If \(\la u,v\ra = 0,\) then \(\|u+v\|^2 = \|u\|^2 + \|v\|^2\text{.}\)

Remark 3.1.8. The angle between vectors.

For nonzero vectors \(u,v \in \R^n,\) the Cauchy-Schwarz inequality says that

\begin{equation*} \frac{|\,\la u,v\ra\,|}{ \|u\|\, \|v\|} \le 1, \text{ equivalently} -1\le \frac{ \la u,v\ra }{ \|u\|\, \|v\|} \le 1. \end{equation*}

Thus it makes sense to define a unique angle \(\theta \in [0,\pi]\) with

\begin{equation*} \cos\theta := \frac{\la u,v\ra }{ \|u\|\, \|v\|} \end{equation*}

which we can call the angle between the vectors \(u,v.\) In some statistical interpretations of the vectors, the value of \(\cos \theta\) is called a correlation coefficient.

Prev Top Next