Linear Algebra I MT22, Cauchy-Schwarz Inequality
Flashcards
Can you state the Cauchy-Schwarz Inequality in full?
For $v, w$ in an inner product space $V$, then
\[ \vert \langle v, w\rangle \vert \le \vert \vert v \vert \vert \text{ } \vert \vert w \vert \vert \]The Cauchy-Schwarz inequality states
For $v, w$ in an inner product space $V$, then $ \vert \langle v, w\rangle \vert \le \vert \vert v \vert \vert \text{ } \vert \vert w \vert \vert $.
When does equality hold?
For $v, w$ in an inner product space $V$, then $ \vert \langle v, w\rangle \vert \le \vert \vert v \vert \vert \text{ } \vert \vert w \vert \vert $.
When $v, w$ are linearly dependent, i.e. $v + t _ 0 w = 0$.
In the proof of the Cauchy-Schwarz inequality
For $v, w$ in an inner product space $V$, then $ \vert \langle v, w\rangle \vert \le \vert \vert v \vert \vert \text{ } \vert \vert w \vert \vert $.
Why can you assume $w \ne 0$?
For $v, w$ in an inner product space $V$, then $ \vert \langle v, w\rangle \vert \le \vert \vert v \vert \vert \text{ } \vert \vert w \vert \vert $.
Because if $w = 0$ then the result is immediate.
In the proof of the Cauchy-Schwarz inequality
For $v, w$ in an inner product space $V$, then $ \vert \langle v, w\rangle \vert \le \vert \vert v \vert \vert \text{ } \vert \vert w \vert \vert $.
What statement do you start with that the whole proof follows from?
For $v, w$ in an inner product space $V$, then $ \vert \langle v, w\rangle \vert \le \vert \vert v \vert \vert \text{ } \vert \vert w \vert \vert $.
In the proof of the Cauchy-Schwarz inequality
For $v, w$ in an inner product space $V$, then $ \vert \langle v, w\rangle \vert \le \vert \vert v \vert \vert \text{ } \vert \vert w \vert \vert $.
You get to a stage where you have
\[0 \le \langle v,w \rangle + 2t\langle v,w \rangle + t^2\langle w,w \rangle\]
What do you do next?
For $v, w$ in an inner product space $V$, then $ \vert \langle v, w\rangle \vert \le \vert \vert v \vert \vert \text{ } \vert \vert w \vert \vert $.
Use the fact the discriminant $b^2 - 4ac \le 0$.
In the proof of the Cauchy-Schwarz inequality
For $v, w$ in an inner product space $V$, then $ \vert \langle v, w\rangle \vert \le \vert \vert v \vert \vert \text{ } \vert \vert w \vert \vert $.
You get to a stage where you have
\[\begin{aligned}
0 &\le \vert \vert v+tw \vert \vert ^2 \\\\
&=\langle v,v \rangle + 2t\langle v,w \rangle + t^2\langle w,w \rangle
\end{aligned}\]
If equality holds in Cauchy-Schwarz, i.e. $ \vert \langle v, w\rangle \vert = \vert \vert v \vert \vert \text{ } \vert \vert w \vert \vert $, why are $v, w$ linearly dependent? (Assume $w \ne 0$.)
For $v, w$ in an inner product space $V$, then $ \vert \langle v, w\rangle \vert \le \vert \vert v \vert \vert \text{ } \vert \vert w \vert \vert $.
Equality in Cauchy-Schwarz means the discriminant of the quadratic in $t$ is zero, so there is a (repeated) real root $t = t _ 0$. At this $t _ 0$, $ \vert \vert v + t _ 0 w \vert \vert ^2 = 0$, hence $v + t _ 0 w = 0$ (norms are positive definite), i.e. $v = -t _ 0 w$. So $v, w$ are linearly dependent. (If $w = 0$, dependence is immediate via $0 \cdot v + 1 \cdot w = 0$.)
Bite-sized
Cauchy-Schwarz inequality: $ \vert \langle v, w \rangle \vert \le $ $\ \vert v\ \vert \cdot \ \vert w\ \vert $. Equality holds iff $v, w$ are linearly dependent. Proof uses $0 \le \ \vert v + tw\ \vert ^2$ as a quadratic in $t$ with non-positive discriminant.
Two key consequences of Cauchy-Schwarz in NLA?
(1) Triangle inequality: $\|v + w\| \le \|v\| + \|w\|$ follows by squaring and using Cauchy-Schwarz on the cross term $\langle v, w \rangle$. (2) $x^\top y \le \|x\| _ 2 \|y\| _ 2$ (vector dot product) is the standard inner-product specialisation, used throughout NLA: bounds for matrix-vector products, residual bounds in LS and CG, etc. The matrix version $\|AB\| _ F \le \|A\| _ F \|B\| _ F$ also has Cauchy-Schwarz flavour.