What Did Bayes Really Say? — References

Introduction | Problem and Definitions | Propositions 1 – 7 | Bayes’s Billiards | Endnotes | References

References

Bayes, T. An essay towards solving a problem in the doctrine of chances. Phil Trans R Soc. 1763 Dec 31;53:370?418.

The essay as originally published. Available at https://royalsocietypublishing.org/doi/pdf/10.1098/rstl.1763.0053. But don’t try to read this; read the next version instead.

Barnard GA. Studies in the History of Probability and Statistics: IX. Thomas Bayes’s Essay Towards Solving a Problem in the Doctrine of Chances: Reproduced with the permission of the Council of the Royal Society from The Philosophical Transactions (1763), 53, 370-418. Biometrika. 1958;45(3-4):293-5.

This is better than the original essay as published because “the notation has been modernized, some of the archaisms have been removed, and what seem to be obvious printer’s errors have been corrected.”

Bernoulli, Daniel. 1738 “Exposition of a New Theory on the Measurement of Risk.” Papers of the Imperial Academy of Sciences in St. Petersburg. Translated from the Latin by Louise Sommer in Econometrica, Vol 22, 1954, pp. 23-36.

Daniel’s uncle, Jacob Bernoulli, wrote Ars Conjectandi, published posthumously in 1713. Daniel distinguishes between the monetary value of an event and its utility. Bayes isn’t concerned with this distinction, but he clearly means to define the probability of an event as the ratio of its expected utility to the utility realized if it occurs.

Blitzstein JK, Hwang J. Introduction to probability. Second edition. Boca Raton: CRC Press; 2019.

My favorite probability textbook. Look for Joseph Blitzstein’s Stat 110 lectures on YouTube. This book and the other probability textbooks on my shelf (see list below) present the probability axioms, basic rules, and definitions — what Price calls “the general laws of chance” — in roughly the same order that Bayes does in Section 1 of this essay. Except for Jaynes, they all use the notation of set theory.

The following books all identify $P(A \cap B) = P(A)P(B|A)$ as the “multiplication rule”:

Berry, Donald A. Statistics: A Bayesian Perspective. Duxbury; 1996. p. 133

Freund JE, Walpole RE. Mathematical Statistics. Third edition. Prentice-Hall; 1980. p. 53.

Hogg RV, Craig AT. Introduction to Mathematical Statistics. Fourth edition. MacMillan; 1978. p. 63.

Ross, Sheldon. A First Course in Probability. 10th Edition. Pearson; 2020. p. 73.

Dale AI. Bayes or Laplace? An Examination of the Origin and Early Applications of Bayes? Theorem. Archive for History of Exact Sciences 1982;27:23?47.

As we will see, Bayes couldn’t find a good approximate solution to his problem when the number of successes and failures are both large (>15). Laplace found one 20 years later. Dale shows that Price, building on Bayes’s analysis, got close.

Jaynes ET, Bretthorst GL. Probability theory: the logic of science. Cambridge, UK; New York, NY: Cambridge University Press; 2003. 727 p.

Jaynes presents probability as an extension of logic. In other words, he extends reasoning about the truth of a proposition to reasoning about its plausibility. His approach is more general than the Kolmogorov system of probability with its use of set notation, but since that is the approach taken by Blitzstein and the other textbooks on my shelf, I will use set notation in my comments. Jaynes refers to propositions that are true or false; the other books (and Bayes) refer to events that either occur or fail to occur.

Keynes, J. M., A Treatise on Probability. Macmillan \& Co., London, 1921.

Before his “General Theory of Employment, Interest and Money”, Keynes published this book on probability theory, which according to Dale (see above), presents the odds form of Bayes’s Rule.

Komologorov AN, Bharucha-Reid AT, Morrison N. Foundations of the theory of probability. Second English edition, Dover edition. Muneola, New York: Dover Publications, Inc; 1956. 84 p.

The original German monograph appeared in 1933 with a Russian translation in 1936. This English translation by Nathan Morrison was published in 1956. Bharucha-Reid added a bibliography. This is the origin of the now standard textbook exposition of a sample space $\Omega$ of possible events and the axioms: 1) $P(\Omega) = 1$ and 2) $P(A \cup B) = P(A) + P(B)$ for disjoint $A$ and $B$ . Jaynes (2003) says that replacing logical operations on propositions $A$ , $B$ , … by set operations on the set $\Omega$ does not change the abstract structure of the theory, but makes it less general in respects that can matter in applications.” I see no reason to talk in terms of “events” instead of “propositions”, but in order for my presentation of the properties of probability to parallel those presented in the textbooks, I will use the symbol “ $\cap$ ” for the logical ”AND” and the symbol “ $\cup$ ” for the logical “OR”.

Laplace, Pierre Simon. 1814. “A Philosophical Essay on Probabilities” Blackmore Dennett. Kindle Edition. Published 2019.

This is Laplace’s write-up of lectures that he delivered in 1795 to “the normal schools” where he had been called by the national convention as a professor of mathematics. It covers the same material as his “Analytical Theory of Probabilities”, but without the equations. This is the 1902 English translation by F.W. Truscott and F.L.Emory.

Stigler SM. The history of statistics: the measurement of uncertainty before 1900. Cambridge, Mass: Belknap Press of Harvard University Press; 1986.

See Chapter 3: Inverse Probability. Stigler differs from some others on the nature of Bayes’s key insight.

Stigler SM. Richard Price, the first Bayesian. Statistical Science. 2018 Feb;33(1):117-25.

An interesting article about Richard Price and especially his appendix to Bayes’s essay.