NLA MT25, Subspaces
Flashcards
In what way does a matrix $V \in \mathbb R^{n \times d}$ represent a subspace $\mathcal S$?
The span of the columns.
@Prove that if:
- $V _ 1 \in \mathbb R^{n \times d _ 1}$
- $V _ 2 \in \mathbb R^{n \times d _ 2}$
- $V _ 1, V _ 2$ both have linearly independent column vectors
- $d _ 1 + d _ 2 > n$
then:
- There is a nonzero intersection between the two subspaces $\mathcal S _ 1 = \text{span}(V _ 1)$ and $\mathcal S _ 2 = \text{span}(V _ 2)$.
Consider the matrix $M := [V _ 1, V _ 2]$, which is of size $n \times (d _ 1 + d _ 2)$. Since $d _ 1 + d _ 2 > n$, this matrix has a right null vector $c \ne 0$ such that $Mc = 0$. Then splitting $c = \begin{bmatrix}c _ 1 \\ -c _ 2\end{bmatrix}$ gives the required result, since $V _ 1 c _ 1 = V _ 2 c _ 2$.
Bite-sized
A $d$-dimensional subspace $\mathcal S \subseteq \mathbb R^n$ can be represented by an $n \times d$ matrix $V$ whose columns are basis vectors; we write $\mathcal S = \mathrm{span}(V) = \{V c : c \in \mathbb R^d\}$.
Lemma 1.1 (the key dimension-count tool): if $\mathcal S _ 1, \mathcal S _ 2 \subseteq \mathbb R^n$ are subspaces with $\dim \mathcal S _ 1 + \dim \mathcal S _ 2 > $ $n$, then $\mathcal S _ 1 \cap \mathcal S _ 2$ contains a nonzero vector. This is the workhorse behind Courant-Fischer, Eckart-Young, and Weyl’s inequality proofs.
Why is it convenient to represent a subspace by an orthonormal matrix $Q$ (with $Q^\top Q = I$) rather than an arbitrary basis matrix $V$?
With $Q^\top Q = I$, projections simplify: the orthogonal projector onto $\mathrm{span}(Q)$ is $QQ^\top$, and the least-squares projection of any $x$ is $QQ^\top x$. With an arbitrary $V$, the projector is $V(V^\top V)^{-1} V^\top$ — a small linear solve is needed. Once the QR factorisation $V = QR$ is computed, $\mathrm{span}(V) = \mathrm{span}(Q)$, so orthonormalising “for free” pays off.