I just wanted to put up a few animations of HMC and slice samplers that I have been playing around with.
I started coding up the slice sampler and the Hamiltonian Monte-Carlo (HMC) after reading the chapter in MacKay’s book. Around the same time, Colin Carol published is minimc package for HMC. This package so nicely written so I just decided to play around with it instead. I’ll do another post with some HMC experiments.
The code for the HMC animation (uses minimc) can be found here. The code for slice sampler and the file to run the animation can be found here.
Hamlitonian Monte Carlo
I would recommend heading over to Colin’s blog and reading the introduction to HMC he has posted up there and then come back and watch these videos. Note the Metropolis proposals where the sampler jumps in momentum space to a new energy level. Then it takes a bunch of leapfrog steps and then takes a sample.
1-d slice sampler
I basically implemented Mackay’s algorithm from his book (Ch 29, page 375):
Main algorithm
Stepping out
Shrinking
And here’s the video:
multi-dimensional sampler
Toward the end of his lecture on slice sampling, MacKay answers a student question about how you’d do a multi-dimensional slice sampling. That makes a hell of lot more sense to me that his one paragraph in the book. Basically, each time we pick a random direction for the vector we step out in. Rest is pretty much the same.