Machine Learning MT23, Matrix calculus
Flashcards
Suppose:
- $\pmb z \in \mathbb R^n$
- $f : \mathbb R^n \to \mathbb R$
Can you define
\[\frac{\partial f}{\partial \pmb z}\]
?
\[\frac{\partial f}{\partial \pmb z} = \left[ \frac{\partial f}{\partial z _ 1}, \dots, \frac{\partial f}{\partial z _ n} \right]\]
Suppose:
- $\pmb z \in \mathbb R^n$
- $\pmb f : \mathbb R^n \to \mathbb R^m$, $(\pmb f(\pmb z)) _ i = f _ i(\pmb z)$
Can you define
\[J _ {\pmb z}(\pmb f ) = \frac{\partial \pmb f}{\partial \pmb z}\]
?
\[\frac{\partial \pmb f}{\partial \pmb z} =
\begin{bmatrix}
\frac{\partial f _ 1}{\partial z _ 1} & \frac{\partial f _ 1}{\partial z _ 2} & \cdots & \frac{\partial f _ 1}{\partial z _ n} \\\\
\frac{\partial f _ 2}{\partial z _ 1} & \frac{\partial f _ 2}{\partial z _ 2} & \cdots & \frac{\partial f _ 2}{\partial z _ n} \\\\
\vdots & \vdots & \ddots & \vdots \\\\
\frac{\partial f _ m}{\partial z _ 1} & \frac{\partial f _ m}{\partial z _ 2} & \cdots & \frac{\partial f _ m}{\partial z _ n}
\end{bmatrix}\]
Suppose:
- $\pmb z \in \mathbb R^n$
- $f : \mathbb R^n \to \mathbb R$
- $\pmb W \in \mathbb R^{n \times m}$
Can you define
\[\frac{\partial f}{\partial \pmb W}\]
?
\[\frac{\partial f}{\partial \pmb W} =
\begin{bmatrix}
\frac{\partial f}{\partial W _ {11}\\,} & \frac{\partial f}{\partial W _ {12}\\,} & \cdots & \frac{\partial f}{\partial W _ {1m}\\,} \\\\
\frac{\partial f}{\partial W _ {21}\\,} & \frac{\partial f}{\partial W _ {22}\\,} & \cdots & \frac{\partial f}{\partial W _ {2m}\\,} \\\\
\vdots & \vdots & \ddots & \vdots \\\\
\frac{\partial f}{\partial W _ {n1}\\,} & \frac{\partial f}{\partial W _ {n2}\\,} & \cdots & \frac{\partial f}{\partial W _ {nm}\\,}
\end{bmatrix}\]