AIMA - Probabilistic Reasoning Over Time

In which we try to interpret the present, understand the past, and perhaps predict the future, even when very little is crystal clear.

The changing state of the world is handled by using a set of random variables to represent the state at each point in time.
Representations can be designed to (roughly) satisfy the Markov property, so that the future is independent of the past given the present. Combined with the assumption that the process is _time-homogenous, this greatly simplifies the representation.
A temporal probability model can be thought of as containing a transition model describing the state evolution and a sensor model describing the observation process.
The principle inference tasks in temporal models are filtering (state estimation), prediction, smoothing and computing the most likely explanation. Each of these tasks can be achieved using simple, recursive algorithms whose run time is linear in the length of the sequence.
Three families of temporal models were studies in more depth: hidden Markov models, Kalman filters, and dynamic Bayesian networks (which include the other two as special cases).
Unless special assumptions are made, as in Kalman filters, exact inference with many state variables is intractable. In practice, the particle filtering algorithm and its descendants are an effective family of approximation algorithms.