NLA MT25, Eigenvalue decomposition
Flashcards
@State a theorem that gives any symmetric matrix an eigenvalue decomposition.
Suppose $A \in \mathbb R^{n \times n}$ is a symmetric matrix. Then there is the decomposition
\[A = V \Lambda V^\top\]where:
- $V$ is orthogonal
- $V^\top V = I _ n = V V^\top$
- $\Lambda = \text{diag}(\lambda _ 1, \ldots, \lambda _ n)$ is a diagonal matrix of eigenvalues
@Prove that every symmetric matrix has an eigenvalue decomposition, i.e. that if $A \in \mathbb R^{n \times n}$, then there is a decomposition
\[A = V \Lambda V^\top\]
where:
- $V$ is orthogonal
- $V^\top V = I _ n = V V^\top$
- $\Lambda = \text{diag}(\lambda _ 1, \ldots, \lambda _ n)$
Proof via Schur decomposition:
By the ∆schur-decomposition (over $\mathbb C$), there is a unitary $U \in \mathbb C^{n \times n}$ and upper-triangular $T \in \mathbb C^{n \times n}$ with
\[A = UTU^\ast.\]Since $A^\top = A$ and $A$ is real, $A$ is also Hermitian $(A^\ast = A)$, and hence normal. So by ∆diagonal-schur-iff-normal, $T$ is diagonal. Take $\Lambda = \text{diag}(\lambda _ 1, \ldots, \lambda _ n) =: T$.
Then $\lambda _ j$ are real, since $A = U \Lambda U^\ast$, and $A^\ast = A$ so $\Lambda^\ast = \Lambda$ and each $\lambda _ j = \overline \lambda _ j$.
It remains to replace $U$ with a real orthogonal $V$. Since each $\lambda _ j$ is real, $\text{ker}(A - \lambda _ j I)$ is a real subspace of $\mathbb R^{n}$, and we can pick a real orthonormal basis for it. Doing this on each eigenspace (which are mutually orthogonal because eigenvectors of a symmetric matrix for distinct eigenvalues are orthogonal) yields a real orthogonal $V \in \mathbb R^{n \times n}$ with $V^\top V = I = V V^\top$, so
\[A = V \Lambda V^\top.\]Other proofs:
- Direct proof via induction: Notes - Linear Algebra II HT23, Spectral TheoremU
- Proof via adjoints: ∆spectral-theorem-for-self-adjoint-operators
Worked example
Compute the symmetric eigenvalue decomposition $A = V\Lambda V^\top$ of $A = \begin{bmatrix} 2 & 1 \\ 1 & 2 \end{bmatrix}$ by hand.
Characteristic polynomial: $\det(A - \lambda I) = (2-\lambda)^2 - 1 = (\lambda - 3)(\lambda - 1)$, so $\lambda _ 1 = 3$, $\lambda _ 2 = 1$ (real, as guaranteed for symmetric $A$).
Eigenvectors:
- $\lambda = 3$: $\begin{bmatrix} -1 & 1 \\ 1 & -1 \end{bmatrix} v = 0 \Rightarrow v _ 1 = \tfrac{1}{\sqrt2}(1, 1)^\top$.
- $\lambda = 1$: $\begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix} v = 0 \Rightarrow v _ 2 = \tfrac{1}{\sqrt2}(1, -1)^\top$.
Eigenvectors of distinct eigenvalues of a symmetric matrix are orthogonal, so $V = \tfrac{1}{\sqrt2}\begin{bmatrix} 1 & 1 \\ 1 & -1 \end{bmatrix}$ is orthogonal and
\[A = V \Lambda V^\top, \qquad \Lambda = \mathrm{diag}(3, 1).\]Check: $V \Lambda V^\top = \tfrac12 \begin{bmatrix} 3 & 1 \\ 3 & -1 \end{bmatrix}\begin{bmatrix} 1 & 1 \\ 1 & -1 \end{bmatrix} = \tfrac12\begin{bmatrix} 4 & 2 \\ 2 & 4 \end{bmatrix} = A$ $\checkmark$ (∆symmetric-eigenvalue-decomposition).
Bite-sized
The symmetric eigenvalue decomposition $A = V \Lambda V^\top$ makes two remarkable claims about a real symmetric $A$: the eigenvalues are all real, and eigenvectors of distinct eigenvalues are orthogonal (so $V$ can be chosen real orthogonal).
For what more general matrix class than “symmetric” does a unitary diagonalisation $A = U \Lambda U^*$ exist?
The normal matrices, those satisfying $A^* A = A A^*$. Real symmetric, Hermitian, orthogonal, unitary, and skew-symmetric matrices are all special cases. The eigenvalues are complex in general, but $U$ is always unitary.
When the symmetric $A$ has a repeated eigenvalue $\lambda _ i = \lambda _ j$, the corresponding eigenvectors span a subspace whose dimension equals the multiplicity of $\lambda$; any orthonormal basis of this subspace gives a valid choice of eigenvectors.
Strategy for proving the symmetric eigenvalue decomposition (∆symmetric-eigenvalue-decomposition-proof) via Schur.
- Invoke the Schur decomposition over $\mathbb C$: $A = U T U^\ast$ with $U$ unitary, $T$ upper triangular.
- Observe that real symmetric $A$ is Hermitian, hence normal. By ∆diagonal-schur-iff-normal, $T$ is diagonal.
- Use $A^\ast = A$ to conclude $\Lambda^\ast = \Lambda$, so eigenvalues are real.
- Replace the complex $U$ by a real orthogonal $V$: pick real orthonormal bases of each real eigenspace $\ker(A - \lambda _ j I)$; assemble into $V$.