NLA MT25, Vector and matrix norms

Created: October 13, 2025 | Updated: October 14, 2025 | About these notes | View in context | Study these flashcards

Flashcards

Vector norms

@State three useful inequalities involving the $1$-norm, $2$-norm and $\infty$-norm (for vectors).
  • $\frac{1}{\sqrt{n}} \vert \vert x \vert \vert _ 2 \le \vert \vert x \vert \vert _ \infty \le \vert \vert x \vert \vert _ 2$.
  • $\frac{1}{\sqrt n} \vert \vert x \vert \vert _ 1 \le \vert \vert x \vert \vert _ 2 \le \vert \vert x \vert \vert _ 1$.
  • $\frac 1 n \vert \vert x \vert \vert _ 1 \le \vert \vert x \vert \vert _ \infty \le \vert \vert x \vert \vert _ 1$

@State a useful result about comparing $ \vert \vert x \vert \vert _ p$ and $ \vert \vert x \vert \vert _ q$.

For $p > q$, $ \vert \vert x \vert \vert _ p \le \vert \vert x \vert \vert _ q$.

@Prove that if $p > q$, then $ \vert \vert x \vert \vert _ p \le \vert \vert x \vert \vert _ q$.

By homogeneity, note that $ \vert \vert \alpha x \vert \vert _ p = \vert \alpha \vert \cdot \vert \vert x \vert \vert _ p$, so without loss of generality we may assume $ \vert \vert x \vert \vert _ p = 1$. Then

\[1 = \vert \vert x \vert \vert _ p^p = \sum^n _ {i = 1} \vert x _ i \vert ^p \le \sum^n _ {i = 1} \vert x _ i \vert ^q = \vert \vert x \vert \vert ^q _ q \ge 1\]

where the difficult inequality comes from the fact that the entries of $x _ i$ are bounded by $1$.

Matrix norms

@Define the matrix $p$-norm $ \vert \vert A \vert \vert _ p$.
\[\vert \vert A \vert \vert _ p = \max _ x \frac{ \vert \vert Ax \vert \vert _ p}{ \vert \vert x \vert \vert _ p}\]

The matrix $p$-norm $ \vert \vert A \vert \vert _ p$ is normally defined via

\[\vert \vert A \vert \vert _ p = \max _ x \frac{ \vert \vert Ax \vert \vert _ p}{ \vert \vert x \vert \vert _ p}\]

@State three alternative characterisations of the matrix $2$-norm, $1$-norm and $\infty$-norm.
  • $2$-norm: $ \vert \vert A \vert \vert _ 2 = \sigma _ \max(A)$.
  • $1$-norm: $ \vert \vert A \vert \vert _ 1 = \max _ i \sum^m _ {j = 1} \vert A _ {ji} \vert $, i.e. biggest column sum
  • $\infty$-norm: $ \vert \vert A \vert \vert _ \infty = \max _ i \vert \sum^n _ {j = 1} \vert A _ {ij} \vert $, i.e. biggest row sum

@Define the Frobenius norm $ \vert \vert A \vert \vert _ F$.
\[\vert \vert A \vert \vert _ F = \sqrt{\sum _ i \sum _ j \vert A _ {ij} \vert ^2}\]

@Define the trace norm / nuclear norm $ \vert \vert A \vert \vert _ \ast$.
\[\vert \vert A \vert \vert _ \ast = \sum^{\min(m, n)} _ {i = 1} \sigma _ i (A)\]

Which of the following are unitarily invariant matrix norms (i.e. $ \vert \vert A \vert \vert = \vert \vert UAV \vert \vert $)?

  • $2$-norm
  • $1$-norm
  • $\infty$-norm
  • Frobenius norm
  • Trace norm
  • $2$-norm
  • Frobenius norm
  • Trace norm

@Define what it means for:

  • a vector norm $ \vert \vert \cdot \vert \vert $ to be unitarily invariant
  • a matrix norm $ \vert \vert \cdot \vert \vert $ to be unitarily invariant
  • For any $x \in \mathbb C^n$ and any unitary $U$, $ \vert \vert Ux \vert \vert = \vert \vert x \vert \vert $.
  • For any $A \in \mathbb C^{m \times n}$ and unitary matrices $U,V$, $ \vert \vert A \vert \vert = \vert \vert UAV \vert \vert $.

@Define what it means for a matrix norm $ \vert \vert \cdot|$ to be submultiplicative.
\[\vert \vert AB \vert \vert \le \vert \vert A \vert \vert \vert \vert B \vert \vert\]

@State a result about matrix $p$-norms in relation to $ \vert \vert AB \vert \vert _ p$.

Matrix $p$-norms are submultiplicative, i.e.

\[\vert \vert AB \vert \vert _ p \le \vert \vert A \vert \vert _ p \vert \vert B \vert \vert _ p\]

@Prove that matrix $p$-norms are submultiplicative, i.e.

\[\vert \vert AB \vert \vert _ p \le \vert \vert A \vert \vert _ p \vert \vert B \vert \vert _ p\]

Suppose $B = 0$, then

\[\vert \vert AB \vert \vert _ M = \sup _ {x\ne 0} \frac{ \vert \vert ABx \vert \vert }{ \vert \vert x \vert \vert } = 0 = \vert \vert A \vert \vert _ M \vert \vert B \vert \vert _ M\]

as required. If $B \ne 0$, then

\[\begin{aligned} \vert \vert AB \vert \vert _ M &= \sup _ {x \ne 0} \frac{ \vert \vert ABx \vert \vert }{ \vert \vert x \vert \vert } \\ &= \sup _ {Bx \ne 0} \frac{ \vert \vert ABx \vert \vert }{ \vert \vert Bx \vert \vert } \cdot \frac{ \vert \vert Bx \vert \vert }{ \vert \vert x \vert \vert } \\ &\le \sup _ {x \ne 0} \frac{ \vert \vert Ax \vert \vert }{ \vert \vert x \vert \vert } \cdot \sup _ {x \ne 0} \frac{ \vert \vert Bx \vert \vert }{ \vert \vert x \vert \vert } \\ &= \vert \vert A \vert \vert _ M \vert \vert B \vert \vert _ M \end{aligned}\]

@sheets~

@State three useful results about the $1$-norm, $2$-norm and $\infty$-norm (for matrices).
  • $\frac{1}{\sqrt n} \vert \vert A \vert \vert _ \infty \le \vert \vert A \vert \vert _ 2 \le \sqrt m \vert \vert A \vert \vert _ \infty$
  • $\frac 1 {\sqrt m} \vert \vert A \vert \vert _ 1 \le \vert \vert A \vert \vert _ 2 \le \sqrt n \vert \vert A \vert \vert _ 1$
  • $ \vert \vert A \vert \vert _ 2 \le \vert \vert A \vert \vert _ F \le \sqrt{\min(m, n)} \vert \vert A \vert \vert _ 2$


Related posts