This post is a quick book review of

The Art of Multiprocessor Programming
by Maurice Herlihy and Nir Shavit

The Art of Multiprocessor Programming is one of the books I used to learn about parallel programming.   I think it is a very good book for getting started with parallel programming. The first section discusses theoretical aspects of parallel programming.  This information is important because it describes how to think about parallel execution, which is an important skill when actually programming in parallel.  My early posts were inspired by these chapters and if you liked that content, the first few chapters of The Art of Multiprocessor Programming describes it in much greater detail.  I never found this type of information online.  There is also a good description of parallel programming using atomic operations, which is another topic I briefly covered on this blog.  They discuss the various atomic instructions and their relative computational power and then describe various data structures and algorithms that can be implemented using atomic operations.  This has been very useful for our work parallelizing the kernel.

The Art of Multiprocessor Programming reads more like a university textbook than a programming guide, but that's what I prefer.  They present many algorithms and data structures, but not examples of using those tools to solve real world problems.  I've also noticed a few bugs in the code they provide, so be careful if you are implementing some of their algorithms (and make sure to download the errata).  However even with those issues, I think this is a very good book to get up to speed on parallel programming.

Thanks to Google books you can preview pretty much the entire book online.