Linear Algebra II HT23, Spectral Theorem
Flashcards
Let $A$ be an $n\times n$ real symmetric matrix. How many roots does $\chi _ A(t)$ have, counted with multiplicity, and hence how many eigenvalues?
When proving that $A \in \mathcal{S}^n$ has a characteristic polynomial with $n$ real roots, you argue by contradiction. How do you rearrange $\overline{\mathbf v} \cdot \mathbf A \overline{ \mathbf v }$, using the fact that $\mathbf A$ is symmetric?
Can you state the spectral theorem for real symmetric matrices?
A real symmetric matrix $A \in \mathcal{S}^n$ has $n$ real eigenvalues and there exists an orthonormal basis for $\mathbb R^n$ consisting of eigenvectors for $A$.
If $A \in \mathcal S^n$ is a symmetric $n \times n$ matrix, then the spectral theorem states that $\mathbb R^n$ has an orthonormal basis consisting of eigenvectors of $A$. If you let the matrix consisting of eigenvectors be $P$, so that $\Lambda = P^{-1}AP$ is diagonal, what is true about $P$, since it contains only orthogonal vectors?
Let $V$ ve a real vector space with inner product $\langle \cdot, \cdot \rangle$. What does it mean for a linear map $T$ to be self-adjoint (or symmetric)?
For all $u, v \in \mathbb R^n$
\[\langle Tu, v \rangle = \langle u, T v \rangle \]Can you state the spectral theorem for self-adjoint operators on a real inner product space?
A self-adjoint map $T$ on a finite dimensional real inner product space $V$ has real eigenvalues and there exists an orthonormal basis for $V$ consisting of eigenvectors of $T$.
When proving that if $\mathbf A \in \mathbb{R}^{n\times n}$ is a symmetric matrix, then it has real eigenvalues, what are the two ways you rearrange
\[(Av)^\intercal \overline{v}\]
when $v$ is an eigenvector with eigenvalue $\lambda$?
On one hand
\[\begin{aligned} (Av)^\intercal \overline{v} &= v^\intercal A^\intercal \overline{v} \\ &= v^\intercal \overline{Av} \\ &=\overline \lambda v^\intercal \overline v \end{aligned}\]but also
\[\begin{aligned} (Av)^\intercal \overline{v} &= \lambda v^\intercal \overline{v} \\ \end{aligned}\]When proving the spectral theorem for a real symmetric matrix $A$, you need to show that
- $A$ has real eigenvalues
- There exists an orthonormal basis of $\mathbb R^n$ consisting of eigenvectors of $A$.
What equivalent condition to (2) do we use when proving the spectral theorem?
There exists an orthogonal matrix $R$ such that $R^{-1} A R$.
When proving the spectral theorem for a real symmetric matrix $A$ in an inner product space $V$, how do you construct an orthonormal basis $v _ 1, \ldots, v _ n$ that you then go on to show actually consists of eigenvectors?
Pick any eigenvalue $\lambda _ 1$ of $A$. Then arbitrarily extend to a basis of $V$ and apply the Gram-Schmidt procedure to get an orthonormal basis.
When proving the spectral theorem for a symmetric matrix $A$ in an inner product space $V$, you can construct an orthonormal basis $v _ 1, \ldots, v _ n$ for $V$ where $v _ 1$ is a unit eigenvector of $A$. How do you define $B$, and why does this help you prove the theorem by induction?
where
\[P = [\pmb v _ 1, \ldots, \pmb v _ n]\]and $C$ is an $(n-1) \times (n-1)$ matrix we wish to show is diagonal. By induction, we can assume that there exists a matrix $Q$ such that $Q^{-1} C Q$ and then use this to construct a matrix $R$ that satisfies the property $R^{-1} A R$ is diagonal.
When proving the spectral theorem for a symmetric matrix $A$, you define $B$ as
\[B = P^\intercal A P = \begin{pmatrix}\lambda _ 1 & 0 \\0 & C\end{pmatrix}\]
where
\[P = [\pmb v _ 1, \ldots, \pmb v _ n] \text{ (eigenvectors of A)}\]
and $C$ is an $(n-1) \times (n-1)$ matrix we wish to show is diagonal. By the induction on spectral theorem, what can we assume about $C$?
There exists an orthonormal matrix $Q$ such that
\[Q^{-1} C Q = D\]where $D$ is diagonal.
When proving the spectral theorem for a symmetric matrix $A$, you define $B$ as
\[B = P^\intercal A P = \begin{pmatrix}\lambda _ 1 & 0 \\0 & C\end{pmatrix}\]
where
\[P = [\pmb v _ 1, \ldots, \pmb v _ n] \text{ (eigenvectors of A)}\]
and $C$ is an $(n-1) \times (n-1)$ matrix we wish to show is diagonal. By the induction on spectral theorem, we can assume that there exists $Q$ such that $Q^{-1} C Q$ is a diagonal matrix. Then how do we construct the final $R$ that is an orthonormal matrix such that that $R^{-1}AR$ is a diagonal matrix and what rwo things do we need to check?
Need to check
- $R^{-1} = R^\intercal$
- $R^{-1} A R$ is diagonal
Say you want to prove the spectral theorem for a real symmetric matrix $A$. You’ve done the step where you show it has real eigenvectors. Then what are the “ingredients” for the rest of the proof?
- $P = [\pmb v _ 1, \cdots, \pmb v _ n]$, (matrix consisting of arbitrary orthonormal extension of eigenvector $\pmb v _ 1$).
- $P^\intercal A P = \begin{pmatrix}\lambda _ 1 & 0 \\ 0 & C\end{pmatrix}$
- $Q^\intercal C Q = D$ (the inductive step)
- $R = P \begin{pmatrix}\lambda _ 1 & 0 \\ 0 & Q\end{pmatrix}$
Proofs
Prove that if $A \in \mathcal{S}^n$ (where $\mathcal S^n$ denotes the set of real $n \times n$ symmetric matrices), then $A$ has $n$ real eigenvalues and there exists an orthonormal basis for $\mathbb R^n$ consisting of eigenvectors for $A$ (the spectral theorem).
Induct on $n$. The case $n = 1$ is trivial: any nonzero vector is an eigenvector and $A = (a _ {11})$ is itself real.
Step 1 (eigenvalues are real). Let $\lambda \in \mathbb C$ be a root of $\chi _ A$ with eigenvector $\mathbf v \in \mathbb C^n$, so $A\mathbf v = \lambda \mathbf v$. Consider $(A\mathbf v)^\intercal \overline{\mathbf v}$. On one hand, using $A^\intercal = A$ and $\overline{A \mathbf v} = A \overline{\mathbf v}$ (since $A$ is real),
\[(A\mathbf v)^\intercal \overline{\mathbf v} = \mathbf v^\intercal A^\intercal \overline{\mathbf v} = \mathbf v^\intercal \overline{A \mathbf v} = \overline\lambda\, \mathbf v^\intercal \overline{\mathbf v}.\]On the other,
\[(A\mathbf v)^\intercal \overline{\mathbf v} = \lambda\, \mathbf v^\intercal \overline{\mathbf v}.\]Subtracting gives $(\lambda - \overline\lambda)\, \mathbf v^\intercal \overline{\mathbf v} = 0$. Since $\mathbf v \ne 0$, $\mathbf v^\intercal \overline{\mathbf v} = \|\mathbf v\|^2 > 0$, so $\lambda = \overline\lambda$, i.e. $\lambda \in \mathbb R$. By the fundamental theorem of algebra applied to $\chi _ A$, $A$ has $n$ real eigenvalues counted with multiplicity.
Step 2 (orthonormal basis with $\mathbf v _ 1$ an eigenvector). Pick a real unit eigenvector $\mathbf v _ 1$ with eigenvalue $\lambda _ 1$ (possible by Step 1). Extend ${\mathbf v _ 1}$ to a basis of $\mathbb R^n$ and apply Gram-Schmidt to obtain an orthonormal basis $\mathbf v _ 1, \ldots, \mathbf v _ n$.
Step 3 (block-diagonalise). Let $P = [\mathbf v _ 1 \mid \cdots \mid \mathbf v _ n]$, so $P$ is orthogonal ($P^\intercal P = I$). Set $B := P^\intercal A P$. The first column of $AP$ is $A \mathbf v _ 1 = \lambda _ 1 \mathbf v _ 1$, so the first column of $B$ is $\lambda _ 1 P^\intercal \mathbf v _ 1 = \lambda _ 1 \mathbf e _ 1$. Since $B^\intercal = P^\intercal A^\intercal P = B$, the first row matches the first column, so
\[B = \begin{pmatrix} \lambda _ 1 & 0 \\ 0 & C \end{pmatrix},\]where $C \in \mathbb R^{(n-1) \times (n-1)}$ inherits symmetry from $B$ (i.e. $C \in \mathcal S^{n-1}$).
Step 4 (inductive hypothesis). Apply the inductive hypothesis to $C$: there exists an orthogonal $Q \in \mathbb R^{(n-1) \times (n-1)}$ with $Q^\intercal C Q = D$ diagonal.
Step 5 (assemble $R$). Define
\[R := P \begin{pmatrix} 1 & 0 \\ 0 & Q \end{pmatrix}.\]$R$ is orthogonal (product of orthogonals), and
\[R^\intercal A R = \begin{pmatrix} 1 & 0 \\ 0 & Q^\intercal \end{pmatrix} B \begin{pmatrix} 1 & 0 \\ 0 & Q \end{pmatrix} = \begin{pmatrix} \lambda _ 1 & 0 \\ 0 & Q^\intercal C Q \end{pmatrix} = \begin{pmatrix} \lambda _ 1 & 0 \\ 0 & D \end{pmatrix},\]which is diagonal. The columns of $R$ are therefore an orthonormal basis of $\mathbb R^n$ consisting of eigenvectors of $A$, with eigenvalues read off the diagonal of $R^\intercal A R$.
Bite-sized
Spectral theorem (real symmetric matrices): a real symmetric matrix $A \in \mathbb R^{n \times n}$ has $n$ real eigenvalues and an orthonormal basis of $\mathbb R^n$ consisting of eigenvectors of $A$. Equivalently, there exists an orthogonal $P$ with $P^\top A P$ diagonal.
What’s the more general operator-theoretic version of the spectral theorem?
A self-adjoint linear map $T$ on a finite-dimensional real (or complex) inner-product space $V$ has all eigenvalues real and an orthonormal basis of $V$ consisting of eigenvectors of $T$. Self-adjoint means $\langle Tu, v \rangle = \langle u, Tv \rangle$ for all $u, v$ — the basis-independent generalisation of “symmetric matrix”.