Statistics, Data Mining, and Machine Learning in Astronomy 5 Bayesian Statistical Inference “The Bayesian approach is the numerical realization of common sense ” (Bayesians) We have already addressed[.]
5 Bayesian Statistical Inference “The Bayesian approach is the numerical realization of common sense.” (Bayesians) e have already addressed the main philosophical differences between classical and Bayesian statistical inferences in §4.1 In this chapter, we introduce the most important aspects of Bayesian statistical inference and techniques for performing such calculations in practice We first review the basic steps in Bayesian inference in §5.1–5.4, and then illustrate them with several examples in §5.6–5.7 Numerical techniques for solving complex problems are discussed in §5.8, and the last section provides a summary of pros and cons for classical and Bayesian methods Let us briefly note a few historical facts The Reverend Thomas Bayes (1702– 1761) was a British amateur mathematician who wrote a manuscript on how to combine an initial belief with new data to arrive at an improved belief The manuscript was published posthumously in 1763 and gave rise to the name Bayesian statistics However, the first renowned mathematician to popularize Bayesian methodology was Pierre Simon Laplace, who rediscovered (1774) and greatly clarified Bayes’ principle He applied the principle to a variety of contemporary problems in astronomy, physics, population statistics, and even jurisprudence One of the most famous results is his estimate of the mass of Saturn and its uncertainty, which remain consistent with the best measurements of today Despite Laplace’s fame, Bayesian analysis did not secure a permanent place in science Instead, classical frequentist statistics was adopted as the norm (this could be at least in part due to the practical difficulties of performing full Bayesian calculations without the aid of computers) Much of Laplace’s Bayesian analysis was ignored until the early twentieth century when Harold Jeffreys reinterpreted Laplace’s work with much clarity Yet, even Jeffreys’ work was not fully comprehended until around 1960, when it took off thanks to vocal proponents such as de Finetti, Savage, Wald, and Jaynes, and of course, the advent of computing technology Today, a vast amount of literature exists on various Bayesian topics, including the two books by Jaynes and Gregory listed in §1.3 For a very informative popular book about the resurgence of Bayesian methods, see [26] W