3. Bayesian methods for scientific modeling

In Part I we introduce the basics of Bayesian statistics. We’ve divided the material into chapters on:

• Inference and PDFs, with an introduction to Bayes’ theorem and other ingredients of Bayesian statistical analysis, including our first look at parameter estimation;

• Bayesian posteriors, with explorations of probability density (or distribution) functions (PDFs);

• Updating via Bayes’ rule, with interactive guides to the updating of PDFs when additional knowledge is acquired;

• Error propagation, with a detailed look at three aspects of error propagation: nuisance parameters and marginalization, changing variables, and Gaussian approximations to propagated errors.

• Bayes in practice, with a summary of the advantages of the Bayesian framework, elaboration of the Bayesian workflow, and a detailed example on Bayesian linear regression that puts together all of the ingredients.

• Exercises for Part I, which includes exercises to build intuition or gain practice in the basics of Bayesian inference, starting from the rules of plausible inference, focusing particularly on parameter estimation, and building on those examples to provide a first exposure to Monte Carlo sampling.

Good supplementary references for this material that are particularly physicist-friendly are Chapters 1 and 2 in Sivia [SS06]; Chapter 1 from Gregory [Gre05]; and the article by Trotta [Tro08].

Many of the examples in Sivia have been implemented in Jupyter notebooks that you should work through as we proceed. You are highly encouraged to answer the questions in the notebooks, play with different values for the parameters, and extend the examples.

Additional material on scientific modeling can be found in Appendix A. See Overview of scientific modeling material.