Scalable Monte Carlo Simulations of Millions of Particles on Thousands of GPUs

We develop a scalable parallel algorithm for Metropolis Monte Carlo simulations of particles with short-range interactions. We use a checkerboard decomposition to allow massively parallel trial moves on GPUs. A second level of domain decomposition enables scaling to systems of millions of particles on thousands of GPUs. A CPU code path efficiently scales to thousands of CPU cores, with as few as one hundred particles per rank. We implement this algorithm for hard anisotropic particles as a plugin to our open-source particle simulation toolkit HOOMD-blue (http://codeblue.umich.edu/hoomd-blue). Our implementation is production-ready with all the features needed for general use in a large research group. It supports a variety of shapes (spheres, ellipsoids, convex (sphero)polygons, simple polygons, convex (sphero)polyhedra, and general polyhedra), NVT and NPT ensembles, pressure measurement, and free energy calculations along with all the file I/O and scripting capabilities already present in HOOMD-blue.