Computer Vision MT25, Image restoration
Flashcards
Summarise the difference between image enhancement and image restoration.
In image restoration, you model a “true” image which has been degraded.
@State (and @visualise) the general setup of image restoration.
We have:
- A true image $f$
- A degraded image $g$
- Some noise $n$
- A “impulse response” or “point spread” function $d$
- The degraded image comes from a convolution with $d$ and added noise $n$
Suppose we have:
- A true image $f$
- A degraded image $g$
- A “impulse response” or “point spread” function $d$
- The degraded image comes from a convolution with $d$ and added noise $n$
\[g(x, y) = \iint d(x - u, y - v) f(u, v) \text du \text dv\]
How could you naïvely recover an estimate $\hat f$ of the true image?
Suppose we have:
- A true image $f$
- A degraded image $g$
- A “impulse response” or “point spread” function $d$
- The degraded image comes from a convolution with $d$ and added noise $n$
\[g(x, y) = \iint d(x - u, y - v) f(u, v) \text du \text dv\]
One way of naïvely recovering an estimate $\hat f$ of the true image is to compute:
\[\hat f = \mathcal F^{-1} \left( \frac{\mathcal F\{g\}}{\mathcal F\{d\}} \right)\]
What’s the problem with this?
$d$ is often very close to zero in high-frequency regions, so you end up amplifying the noise present in the original image.
@visualise~
Wiener filter
Suppose we have:
- A true image $f$
- A degraded image $g$
- A “impulse response” or “point spread” function $d$
- The degraded image comes from a convolution with $d$ and added noise $n$
\[g(x, y) = \iint d(x - u, y - v) f(u, v) \text du \text dv\]
One way of naïvely recovering an estimate $\hat f$ of the true image is to compute:
\[\hat f = \mathcal F^{-1} \left( \frac{\mathcal F\{g\}}{\mathcal F\{d\}} \right)\]
The problem with this is that $d$ often very close to zero in high-frequency regions, so you end up amplifying the noise present in the original image. @Define the Wiener filter and @state the additional assumption that it is derived from.
We add the constraint that we wish to minimise $\mathbb E( \vert g - \hat f \vert ^2)$. The Wiener filter works in the frequency domain as
\[\hat F(u, v) = W(u, v) G(u, v)\]where
\[W(u, v) = \frac{D^\ast(u, v)}{ \vert D(u, v) \vert ^2 S(u, v) + K(u, v)}\]and:
- $S(u, v) = \mathbb E( \vert F(u, v) \vert ^2)$ is the mean power spectral density of the original signal, typically $1$
- $K(u, v) = \mathbb E( \vert N(u, v) \vert ^2)$ is the mean power spectral density of the noise, typically a small constant
The Wiener filter works in the frequency domain as
\[\hat F(u, v) = W(u, v) G(u, v)\]
where
\[W(u, v) = \frac{D^\ast(u, v)}{ \vert D(u, v) \vert ^2 S(u, v) + K(u, v)}\]
and:
- $S(u, v) = \mathbb E( \vert F(u, v) \vert ^2)$ is the mean power spectral density of the original signal, typically $1$
- $K(u, v) = \mathbb E( \vert N(u, v) \vert ^2)$ is the mean power spectral density of the noise, typically a small constant
Can you give an alternative formulation that intuitively explains how the Wiener filter prevents amplifying noise?
where $\text{SNR}$ is the signal to noise ratio.
Hence we invert the filter but scale up inversely with the expected noise.
@Visualise how the Wiener filter can be used to deblur an image.
Motion blur
What’s one way of modelling motion blur?
Convolution with a line segment filter.
@Visualise how you can model convolution with a line segment filter.
Intuitively, the convolution is averaging the pixels along a particular line.
Give an @algorithm for removing motion blur.
- Rotate the image so that the motion blur is horizontal.
- Estimate the length of the blur.
- Construct a line segment filter.
- Compute and apply Wiener filter.
Generative models of degradation
@State the generative model of image degradation.
We have:
- A true image $f$
- A degraded image $g$
- An $N \times N$ matrix $A$
- Some noise $n \in \mathbb R^{N}$
and model $g$ as
\[g = Af + n\]In the generative model of image degradation, we have:
- A true image $f$
- A degraded image $g$
- An $N \times N$ matrix $A$
- Some noise $n \in \mathbb R^{N}$
and model $g$ as
\[g = Af + n\]
Given $g$ and $A$, how is an estimate $\hat f$ for the original image found?
where $p(f)$ is a regulariser.
In the generative model of image degradation, we have:
- A true image $f$
- A degraded image $g$
- An $N \times N$ matrix $A$
- Some noise $n \in \mathbb R^{N}$
and model $g$ as
\[g = Af + n\]
Given $g$ and $A$, we find an estimate $\hat f$ for the original image via
\[\hat f = \text{argmin} _ f[(g - Af)^2 + \lambda p (f)]\]
where $p(f)$ is a regulariser. Give an @example choice for the regulariser, and explain why this is a good choice.
(recall here that $f$ is a “vectorised” version of the image, i.e. a column vector of pixels).
This means that there’s no drastic changes in the restored image.
In the generative model of image degradation, we have:
- A true image $f$
- A degraded image $g$
- An $N \times N$ matrix $A$
- Some noise $n \in \mathbb R^{N}$
and model $g$ as
\[g = Af + n\]
Given $g$ and $A$, we find an estimate $\hat f$ for the original image via
\[\hat f = \text{argmin} _ f[(g - Af)^2 + \lambda p (f)]\]
where $p(f)$ is a regulariser. @State two ways that you could estimate $A$.
- Super resolution: Given multiple images of the same scene, you can estimate $A$.
- Blind deblurring: Instead consider the optimisation problem $\hat f, \hat h = \text{argmin} _ {f, h} [(g - A(h) f)^2 + \lambda _ f p _ f(f) + \lambda _ h p _ h(h)]$.