This chapter contains the material that every linear algebra course wants to cover, but which often gets short shrift as time runs short and students strain to keep all the new concepts straight. So a point is made to take time with this material.
It is in this chapter that we find some of the most important applications of linear algebra as well as some of the deepest results, many of which have vast generalizations in the realm of functional analysis.
Starting from basic definitions and properties, we move to the fundamental notion of orthogonality and orthogonal projection. While grounded with geometric intuition, this notion has profound applications to high-dimensional spaces where our geometric intuition fails. Applications include least squares solutions to inconsistent linear systems as well as spectral decompositions for real symmetric and unitary/normal complex matrices. We discuss results over the complex numbers, and note where differences arise with the results over the reals. We state without proof the spectral theorems and leverage them to develop the singular value decomposition of a matrix. We give an application an application to image compression and explore some of the underlying duality.
