Computer Vision MT25, Visual tracking
Flashcards
Suppose we have the following template and image:
@State the energy function we would want to minimise in a template tracking context, and @visuallise what the energy might look like after it has been calculated.
One approach to template tracking is to take a template $T$ and an image $I$, and calculate an energy function
\[E(u,v)
= \sum _ {(x,y)\in\left[-\frac{w}{2},\,\frac{w}{2}\right]\times\left[-\frac{h}{2},\,\frac{h}{2}\right]}
\left( I(u+x,\,v+y) - T\!\left(x+\frac{w}{2},\,y+\frac{h}{2}\right) \right)^{2}\]
What is the main problem with this approach?
It is very slow as you have to do many comparisons over many points in the image.
In template tracking, we have an image $I$ and a template $T$ and wish to minimise the energy
\[E(u, v) = \sum _ {x, y} \left( I(u+x, v+y) - T(x, y) \right)^2\]
Minimising this exactly is typically very expensive since you have to iterate over all the pixels in the image. Assuming we have some point $(t _ x, t _ y)$, how does Lucas-Kanade template tracking reformulate this energy function relative to an update $(\delta _ x, \delta _ y)$?
What is the primary problem with visual trackers relying too heavily on the spatial relationships between pixels? What’s one way of partially addressing this problem, and what’s the issue with this fix?
- They are prone to break due to partial occlusion and orientation changes in the scene.
- One fix is to update the template from frame to frame, but this can lead to drift.
A lot of approaches to visual tracking of many points are based on updating points using some estimate of local optical flow, but this has issues with occlusions. What’s one general approach for improving our estimates?
Predict the global changes.