Linear Algebra Refresher: Review, Amplification, Examples

Given an \(n\times n\) matrix \(A\) with eigenvalue \(\lambda\text{,}\) the algebraic multiplicity of the eigenvalue is the degree \(d\) to which the term \((x-\lambda)^d\) occurs in the factorization of the characteristic polynomial for \(A.\)

An basis for a vector space is a linearly independent subset of the vector space whose span is the entire space.

The characteristic polynomial of a square matrix \(A\in M_n(F)\) is \(\chi_A = \det(xI_n-A)\text{.}\) One can show that \(\chi_A\) is a monic polynomial of degree \(n\) with coefficients in the field \(F.\)

Note that some authors define the characteristic polynomial as \(\det(A-xI_n)\) in which case the leading coefficient is \((-1)^n,\) but since the interest is only in the factorization of \(\chi_A\) (in particular any roots it may have), it does not really matter which definition one uses.

The column space of an \(m\times n\) matrix \(A\) is the span of the columns of \(A.\) As such, it is a subspace of \(F^m.\)

The dimension of a vector space is the cardinality (size) of any basis for the vector space.

Implicit in the definition of dimension are theorems which prove that every vector space has a basis, and that any two bases for a given vector space have the same cardinality. In other words, the dimension is a well-defined term not depending upon which basis is chosen to consider. When a vector space has a basis with a finite number of elements, it is called finite-dimensional.

An elementary matrix is a matrix obtained by performing a single elementary row (or column) operation to an identity matrix.

Given an \(n\times n\) matrix \(A\) with eigenvalue \(\lambda\text{,}\) the geometric multiplicity of the eigenvalue is the dimension of the eigenspace associated to \(\lambda.\)

A complex matrix \(A\) is called Hermitian if \(A = A^*.\) Necessarily the matrix needs to be square.

A function \(f:X\to Y\) between sets \(X\) and \(Y\) is injective if for every \(x, x' \in X\text{,}\) \(f(x) = f(x')\) implies \(x=x'\text{.}\)

Let \(F\) denote the field of real or complex numbers. For \(z = a + bi \in \C\) (\(a,b \in \R\) and \(i^2 = -1\)), we have the notion of the complex conjugate of \(z\text{,}\) denoted \(\overline z = a-bi.\) Note that when \(z\in \R\text{,}\) that is \(z = a = a + 0i \in \C\text{,}\) we have \(z = \overline z.\) The magnitude (norm, \(absolute value\)) of \(z=a+bi\) is \(|z| = \sqrt{a^2 + b^2}.\)

An inner product space is a vector space \(V\) defined over a field \(F = \R\) or \(\C\) to which is associated an inner product. If \(F=\R\text{,}\) \(V\) is called a real inner product space, and if \(F=\C\text{,}\) then \(V\) is called a complex inner product space.

An isomorphism is a linear map which is bijective (one-to-one and onto; injective and surjective).

A linear combination of vectors \(v_1, \dots, v_r \in V\) is any vector of the form \(a_1v_1 + \cdots + a_rv_r\) for scalars \(a_i \in F.\)

The minimal polynomial of a square matrix \(A\in M_n(F)\) is the monic polynomial, \(\mu_A\text{,}\) of least degree with coefficients in the field \(F\) so that \(\mu_A(A) = 0\text{.}\) The Cayley-Hamilton theorem implies that the minimal polynomial divides the characteristic polynomial.

A matrix \(A\in M_n(\C)\) is normal if it commutes with its conjuate transpose: \(AA^* = A^*A.\)

The nullity of a linear transformation \(T:V\to W\) is the dimension of \(ker(T),\) that is, the dimension of its nullspace.

If \(T:F^n\to F^m\) is given by \(T(x) = Ax\) for an \(m\times n\) matrix \(A\text{,}\) then the nullity of \(T\) is the dimension of the set of solutions of \(Ax=0.\)

If \(T:F^n\to F^m\) is given by \(T(x) = Ax\) for an \(m\times n\) matrix \(A\text{,}\) then the nullspace of \(T\) is often called the nullspace of \(A\), the set of solutions of \(Ax=0.\)

The pivot positions of a matrix are the positions (row,column) which correspond to a leading one in the reduced row-echelon form of the matrix. The pivots are the actual entry of the given matrix at the pivot position.

The pivot columns are the columns of the original matrix corresponding to the columns of the RREF containing a leading one.

The rank of a linear transformation \(T:V\to W\) is the dimension of its image, \(\Im(T).\)

If \(T:F^n\to F^m\) is given by \(T(x) = Ax\) for an \(m\times n\) matrix \(A\text{,}\) then the rank of \(T\) is the dimension of the column space of \(A.\)

By theorem, it is also equal to the dimension of the row space which is the number of nonzero rows in the RREF form of the matrix \(A.\)

The row space of an \(m\times n\) matrix \(A\) is the span of the rows of \(A.\) As such, it is a subspace of \(F^n.\)

Let \(A, B \in M_n(F)\text{.}\) The matrix \(B\) is said to be similar (or conjugate) to \(A\) if there exists an invertible matrix \(P\in M_n(F)\) so that \(B= P^{-1}A P.\) Note that if we put \(Q=P^{-1}\text{,}\) then \(B=QAQ^{-1}\text{,}\) so it does not matter which side carries the inverse. Also note that this is a symmetric relationship, so that \(B\) is similar to \(A\) if and only if \(A\) is similar to \(B.\) Indeed similarity (conjugacy) is an equivalence relation.

Let \(V\) be a vector space over a field \(F,\) and let \(W\subseteq V\text{.}\) \(W\) is called a subspace of \(V\) if \(W\) is itself a vector space with the operations of vector addition and scalar multiplication inherited from \(V.\)

Of course checking all the vector space axioms can be quite tedious, but as a theorem you prove much easier criteria to check. Recall that you already know that \(V\) is a vector space, so many of the axioms (associativity, distributive laws etc) are inherited from \(V.\) Indeed, you prove that to show that \(W\) is a subspace of \(V\text{,}\) it is enough to show that the additive identity of \(V\) is in \(W,\) and that \(W\) is closed under the inherited operations of vector addition and scalar multiplication, i.e, whenever \(w, w' \in W\) and \(\lambda \in F\text{,}\) we must have \(w+w'\in W\text{,}\) and \(\lambda w\in W.\)

A function \(f:X\to Y\) between sets \(X\) and \(Y\) is surjective if for every \(y\in Y\text{,}\) there exists an \(x \in X\) such that \(f(x)=y\text{.}\)

A matrix \(A\) is called symmetric if \(A = A^T.\) Necessarily the matrix needs to be square.