Machine Learning MT23, Covariance and correlation
Flashcards
For a random variable $\pmb x \in \mathbb R^D$, what is the covariance matrix $\mathrm{cov}(\pmb x)$ and what is $(\mathrm{cov}(\pmb x)) _ {ij}$?
\[\mathbb E \left[ (\pmb x - \mathbb E[\pmb x])(\pmb x - \mathbb E[\pmb x])^T \right]\]
and
\[(\mathrm{cov}(\pmb x)) _ {ij} = \mathrm{cov}(X _ i, X _ j)\]Covariance depends on the scale of variables. @Define the correlation $\mathrm{corr}(X, Y)$, which is normalised between $\pm 1$.
\[\mathrm{corr}(X, Y) = \frac{\mathrm{cov}(X, Y)}{\sqrt{\mathrm{var}(X) \mathrm{var}(Y)}\,}\]
Suppose $\pmb \theta$ represents the parameters of some distribution with density function $p$. @Define the likelihood of observing $(x _ 1, \ldots, x _ n)$, i.e. the probability of observing the data with parameter $\theta$.
\[\prod^N _ {i=1} p(x _ i \mid \theta)\]