AIMA: Quantifying Uncertainty
In which we see how to tame uncertainty with numeric degrees of belief.
Flashcards
Why probability and decision theory
Why do we need probability theory?
Because it provides a reasonable framework for dealing with uncertainty.
What is the decision theory “equation”?
What is the principle of maximum expected utility (MEU)?
An agent is rational if and only if it chooses the action that yields the highest expected utility, averaged over all possible outcomes of the action.
If logical assertions say whether a possible world exists or not, what do probabilistic assertions say?
How likely a given world is.
Prior and posterior probability
What is a prior or unconditional probability?
Degrees of belief in an event without any other information.
What is a posterior or conditional probability?
Degrees of belief in an event with additional information given.
Is
\[P(\text{Total} = 11 \vert \text{Die} _ 1 = 6)\]
a prior or posterior probability?
Posterior.
How do you pronounce
\[P(A \vert B)\]
?
“The probability of $A$ _ given _ $B$”
The product rule and distributions
What is the difference between
\[\pmb{P}(\pmb{x})\]
and
\[P(x)\]
?
The former is a probability distribution showing the different probabilities for different values of a vector $\pmb{x}$ and the latter is the probability of $x$ occurring.
What is a joint probability distribution?
A way of describing the probability for different combinations of variables.
What is the full joint probability distribution?
A way of describing the probability for every different combinations of random variables under consideration.
How can you reduce the size of a full joint probability distribution?
Use absolute and conditional independence to factor it down.
Bayes’ rule
What is Bayes’ rule, for determing the probability of a $\text{Cause}$ given its potential $\text{Effect}$?
Why is Bayes’ rule sometimes written as
\[\pmb{P}(A \vert B) = \alpha \pmb{P}(B \vert A) \pmb{P}(B)\]
?
Because $\alpha$ is a normalisation factor that makes the values of the probability distribution sum to one.
Conditional independence and naive Bayes
What is conditional independence of $A$ and $B$ given $C$?
Where $A$ and $B$ are only independent after observing $C$ as it is the direct cause of both of them.
How could you state the conditional independence of $A$ and $B$ given $C$ in probability notation?
What is a naive Bayes model?
A technique for estimating probabilities in a situation where you have a single cause with conditional independence of all effect variables.
What is an example cause and effect in a naive Bayes model?
- Cause: The category of a piece of text
- Effect: Certain keywords appearing in the text