© 26 January 2023 by Michael A. Kohn
Link to the pdf if this article
Introduction | Coin Problem | Bayes’s Rule: History | Bayes’s Rule: Derivation | Billiards Problem | Endnotes | References
Coin Problem
Many years ago, I was asked in a job interview to solve the following problem:
A bag contains three coins: one fair coin, one 2-headed coin, and one 2-tailed coin. One of the three coins is selected and flipped. It shows heads. What is the probability that it is the 2-headed coin?
In this problem, the hypothesis is that the selected coin is 2-headed. The observation is that it comes up heads on a single toss. To solve the problem, we calculate one unknown probability from three known probabilities.
The unknown probability is the probability of having selected the 2-headed coin given that it comes up heads on a single toss:
$$P(\text{2-headed}\,|\,\text{heads}) .$$
The three known probabilities are:
1) the probability that it comes up heads given that it is the 2-headed coin, $$P(\text{heads} \:|\: \text{2-headed}) = 1 ,$$ 2) the probability of selecting the 2-headed coin, $$P(\text{2-headed}) = \frac{1}{3} ,$$ 3) the overall probability of heads,
\begin{align*} P(\text{heads}) & = P(\text{heads}\,|\,\text{fair})\!\times\! \frac{1}{3} + P(\text{heads}\,|\,\text{2-headed})\!\times\! \frac{1}{3} + P(\text{heads}\,|\,\text{2-tailed})\!\times\! \frac{1}{3}\\ P(\text{heads}) &=(\frac{1}{2}\times \frac{1}{3}) + (1 \times \frac{1}{3}) + (0 \times \frac{1}{3}) = \frac{1}{2} . \end{align*}The formula used for this calculation is universally known as “Bayes’s Theorem” or “Bayes’s Rule”. (If you think I should be punctuating the possessive in some way other than “Bayes’s”, see Endnote #1.) I will give a general version of Bayes’s Rule in the next section. For this problem, it is:
\begin{align*} P(\text{2-headed} \,|\,\text{heads}) &= \frac{P(\text{heads}\,|\,\text{2-headed})\times P(\text{2-headed)}}{P(\text{heads})}\\ &= \frac{1 \times \frac{1}{3}}{\frac{1}{2}} \end{align*}
In the interview, I didn’t simplify the answer to \(\frac{2}{3}\), but the interviewer passed me to the next level anyway.