19. Details of MCMC

Here we provide formal and detailed discussions on stochastic processes in general, including random walk and Gaussian process examples, and then on Markov Chain Monte Carlo and the Metropolis Hasting algorithm.

• Stochastic processes introduces the concept of a stochastic process, with code examples and several exercises.

• Example: Random walk provides a basic example.

• Special case: Gaussian process introduces the Gaussian process, which we will revisit in detail in Overview of Gaussian processes.

• Markov chains has a detailed introduction to Markov chains, with examples.

• Derivation of the Metropolis-Hastings algorithm provides a complete definition and some exercises.