Machine Learning MT23, Multivariate Gaussians
Flashcards
Can you define the density function $p(\pmb x)$ for a multivariate Gaussian with covariance $\pmb \Sigma$ and mean $\pmb \mu$?
\[p(\pmb x) = \frac{1}{(2\pi)^{D/2} \vert \pmb \Sigma \vert ^{1/2}\,}\exp\left(-\frac 1 2(\pmb x - \pmb \mu)^T\pmb \Sigma^{-1} (\pmb x - \pmb \mu)\right)\]
The multivariate Gaussian with mean $\pmb \mu$ and covariance matrix $\pmb \Sigma$ is defined as:
\[p(\pmb x) = \frac{1}{(2\pi)^{D/2} \vert \pmb \Sigma \vert ^{1/2}\,}\exp\left(-\frac 1 2(\pmb x - \pmb \mu)^T\pmb \Sigma^{-1} (\pmb x - \pmb \mu)\right)\]
What qualitative relationship is there between the eigenvectors of $\pmb \Sigma$ and the variance?
Order the eigenvalues $\pmb v _ 1, \cdots, \pmb v _ n$ of $\pmb \Sigma$ according to the size of the eigenvalues $ \vert \lambda _ 1 \vert \ge \vert \lambda _ 2 \vert \ge \cdots \ge \vert \lambda _ n \vert $. Then these eigenvectors correspond to the directions of greatest variance in the distribution.