Bayesian v. Frequentist — Coin Problem

© 26 January 2023 by Michael A. Kohn

Link to the pdf if this article

Introduction | Coin Problem | Bayes’s Rule: History | Bayes’s Rule: Derivation | Billiards Problem | Endnotes | References

Coin Problem

Many years ago, I was asked in a job interview to solve the following problem:

A bag contains three coins: one fair coin, one 2-headed coin, and one 2-tailed coin. One of the three coins is selected and flipped. It shows heads. What is the probability that it is the 2-headed coin?

In this problem, the hypothesis is that the selected coin is 2-headed. The observation is that it comes up heads on a single toss. To solve the problem, we calculate one unknown probability from three known probabilities.

The unknown probability is the probability of having selected the 2-headed coin given that it comes up heads on a single toss:

$$P(\text{2-headed}\,|\,\text{heads}) .$$

The three known probabilities are:

1) the probability that it comes up heads given that it is the 2-headed coin, $$P(\text{heads} \:|\: \text{2-headed}) = 1 ,$$ 2) the probability of selecting the 2-headed coin, $$P(\text{2-headed}) = \frac{1}{3} ,$$ 3) the overall probability of heads,

$$\begin{align*} P(\text{heads}) & = P(\text{heads}\,|\,\text{fair})\!\times\! \frac{1}{3} + P(\text{heads}\,|\,\text{2-headed})\!\times\! \frac{1}{3} + P(\text{heads}\,|\,\text{2-tailed})\!\times\! \frac{1}{3}\\ P(\text{heads}) &=(\frac{1}{2}\times \frac{1}{3}) + (1 \times \frac{1}{3}) + (0 \times \frac{1}{3}) = \frac{1}{2} . \end{align*}$$

The formula used for this calculation is universally known as “Bayes’s Theorem” or “Bayes’s Rule”. (If you think I should be punctuating the possessive in some way other than “Bayes’s”, see Endnote #1.)   I will give a general version of Bayes’s Rule in the next section. For this problem, it is:

$$\begin{align*} P(\text{2-headed} \,|\,\text{heads}) &= \frac{P(\text{heads}\,|\,\text{2-headed})\times P(\text{2-headed)}}{P(\text{heads})}\\ &= \frac{1 \times \frac{1}{3}}{\frac{1}{2}} \end{align*}$$

In the interview, I didn’t simplify the answer to $\frac{2}{3}$, but the interviewer passed me to the next level anyway.

(next) Bayes’s Rule: History

Introduction | Coin Problem | Bayes’s Rule: History | Bayes’s Rule: Derivation | Billiards Problem | Endnotes | References