Geometric Deep Learning HT26, Invariance and equivariance

Created: February 08, 2026 | Read markdown | About these notes | View in context | Study these flashcards |

Flashcards

Invariance and equivariance

Suppose:

  • $G$ is a group
  • $\Omega$ is a domain
  • $\mathcal X (\Omega)$ is the space of signals of $\Omega$
  • $\rho$ is a representation on $G$

@Define what it means for a function

\[f : \mathcal X( \Omega ) \to \mathcal Y\]

to be $G$-invariant.

g-invariant
\[f(\rho(g)x) = f(x)\]

for all $g \in G$ and $x \in \mathcal X(\Omega)$.

Suppose:

  • $G$ is a group
  • $\Omega$ is a domain
  • $\mathcal X (\Omega)$ is the space of signals of $\Omega$
  • $\rho$ is a representation on $G$

@Define what it means for a function

\[f : \mathcal X( \Omega ) \to \mathcal X( \Omega )\]

to be $G$-equivariant.

g-equivariant
\[f(\rho(g)x) = \rho(g) f(x)\]

for all $g \in G$ and $x \in \mathcal X(\Omega)$.

Approximate invariance

Deformation stability

Suppose:

  • $\Omega$ is a domain
  • $\mathcal X (\Omega)$ is a set of signals on this domain
  • $\text{Diff}(\Omega)$ is the group of sufficiently smooth deformations of $\Omega$
  • $G \subset \text{Diff}(\Omega)$ is a subgroup of stronger symmetries, such as translations
  • $c : \text{Diff}(\Omega) \to \mathbb R$ is a complexity measure of symmetries so that $c(\tau) = 0$ whenever $\tau \in G$

@Define what it means for a function $f$ to be geometrically stable with respect to this complexity measure. @Visualise this in the context of diffeomorphisms of images

d75648e40d9c44a0b1b7650a5ed80e4e
\[ \vert \vert f(\rho(\tau) x) - f(x) \vert \vert \le Cc(\tau) \vert \vert x \vert \vert \]

where $C$ is some constant independent of this signal.

Domain deformations

Suppose:

  • $\mathcal D$ is a space of possible domains (e.g. all graphs)
  • $d _ {\mathcal D}$ is some measure of distances between domains
  • $\mathcal X(\mathcal D) = \{ (\mathcal X(\Omega), \Omega) \mid \Omega \in \mathcal D \}$ is a set of possible input signals over these domains

@Define what it means for a function $f : \mathcal X(\mathcal D) \to \mathcal Y$ to be stable to domain deformations.

a18152538869493790620dccb7568ac0
\[ \vert \vert f(x, \Omega) - f(\tilde x, \tilde \Omega) \vert \vert \le C \vert \vert x \vert \vert d _ {\mathcal D}(\Omega, \tilde \Omega)\]

Triviality of linear invariants

Suppose:

  • $f : \mathcal X(\Omega) \to \mathcal Y$ is a linear function
  • $f$ is $G$-invariant

@Justify that linear invariants are trivial in some sense.

9c5b1b97602748f5bea38e9262bf01b1
\[\begin{aligned} f(x) &= f(x) \frac{\mu(G)}{\mu(G)} \\ &= f(x) \cdot \frac{1}{\mu(G)} \int _ G d \mu (g) \\ &= \frac{1}{\mu (G))} \int _ G f(x) \text d \mu(g) \\ &= \frac{1}{\mu (G)} \int _ G f(g\cdot x) \text d \mu(g) \\ &= f\left( \frac{1}{\mu(G)} \int _ G (g \cdot x) \text d \mu(g) \right) \end{aligned}\]

So $f$ can only depend on $x$ through a group averaging operation. In the case of translations of images, this corresponds to the average RGB colour.

@State a result about how you can create a family of rich and stable features by composition of linear equivariants.

55cce39aa4eb473dac002ce27ece27c5

Suppose

  • $\Omega$ is some domain
  • $\mathcal X(\Omega, \mathcal C)$ and $\mathcal X(\Omega, \mathcal C')$ are signals over channels $\mathcal C, \mathcal C'$
  • $G$ is a group of symmetries of $\Omega$
  • $B : \mathcal X(\Omega, \mathcal C) \to \mathcal X(\Omega, \mathcal C')$ is $G$-equivariant satisfying $B(g \cdot x) = g \cdot B(x)$ for all $x \in \mathcal X$, $g \in G$
  • $\sigma : \mathcal C' \to \mathcal C''$ is an arbitrary (non-linear) map
  • $\pmb \sigma : \mathcal X(\Omega, \mathcal C') \to \mathcal X(\Omega, \mathcal C'')$ is the elementwise instantiation of $\sigma$ given as $(\pmb \sigma(x))(u) = \sigma(x(u))$

Then:

\[U := (\pmb \sigma \circ B) : \mathcal X(\Omega, \mathcal C) \to \mathcal X(\Omega, \mathcal C'')\]

is also $G$-equivariant. This yields a general family of $G$-invariants by composing $U$ with group averages $A \circ U : \mathcal X(\Omega, \mathcal C) \to \mathcal C'')$.



Related posts