18. Details of MCMC

Here we provide formal and detailed discussions on stochastic processes in general, including random walk and Gaussian process examples, and then on Markov Chain Monte Carlo and the Metropolis Hasting algorithm.

• Stochastic processes introduces the concept of a stochastic process, with an example.

• Definition of a stochastic process gives a formal definition, with a code examples and several exercises.

• Example: Random walk is a basic example.

• Special case: Gaussian process introduces the Gaussian process, which we will revisit in detail in Overview of Gaussian processes.

• Markov chains has a detailed introduction to Markov chains, with examples.

• The Metropolis-Hastings algorithm provides a complete definition and some exercises.