© 26 January 2023 by Michael A. Kohn
Link to the pdf if this article
Introduction | Coin Problem | Bayes’s Rule: History | Bayes’s Rule: Derivation | Billiards Problem | Endnotes | References
Bayes’s Rule: History
Bayes’s Rule calculates the probability of A given B from
1) the probability of B given A,
2) the probability of A before observing B, and
3) the overall probability of B.
$$P(A|B) = \frac{P(B|A) \times P(A)}{P(B)} $$
But Bayes never wrote it.
Thomas Bayes (1701-1761) was a Presbyterian minister, amateur mathematician, and member of the Royal Society of London who lived in Tunbridge Wells, England. He is famous for “An Essay Towards Solving A Problem in the Doctrine of Chances”, which was published in the Royal Society’s Philosophical Transactions on 23 December 1763, more than two and a half years after his death (Bayes 1763; Barnard 1958). His friend Richard Price (1723-1791) found the essay among Bayes’s papers and sent it to the Royal Society along with an introductory letter, footnotes, an abridgement of the latter part of the essay, and an appendix containing numerical examples (Stigler 2018).
The essay as originally published is 49 pages — 24 pages written by Bayes and 25 by Price. It is difficult to read today because, to us, the 18th century English seems stilted and the mathematical notation is unfamiliar. The closest Bayes gets to stating the rule that now bears his name is Proposition 5 of 7 in an introductory section on “the general laws of chance”.
Original text
If there be two subsequent events, the probability of the 2nd b/N and the probability of both together P/N, and it being first discovered that the 2nd event has happened, from hence I guess that the 1st event has also happened, the probability I am in the right is P/b.
Bayes, T. (1763). An essay towards solving a problem in the doctrine of chances. Phil Trans R Soc. 1763 Dec 31; 53: p. 381.
Modern Equivalent
$$P(A|B) = \frac{P(B \& A)}{P(B)}$$If A and B are two events , then
In many textbooks, this is presented as the definition of conditional probability (Blitzstein, 2019, page 46). Again, it is just one of seven propositions preparatory to discussing the main problem, which we will get to shortly.