Computers with multiple processors have been around for a long time and people have been studying parallel programming techniques for just as along. However only in the last few years have multi-core processors and parallel programming become truly mainstream. What changed?

Here are some definitions for terms used in this post:

• core: the part of a processor responsible for executing a single series of instructions at a time.
• processor: the physical chip that plugs into a motherboard. A computer can have multiple processors, and each processor can have multiple cores
• process: a running instance of a program. A process's memory is usually protected from access by other processes.
• thread: a running instance of a process's code. A single process can have multiple threads, and multiple threads can be executing at the same on multiple cores
• parallel: the ability to utilize more than one processor at a time to solve problems more quickly, usually by being multi-threaded.

For years, processors designers had been able to increase the performance of processors by increasing their clock speeds. However a few years ago they ran into a few serious problems. RAM access speeds were not able to keep up with the increased speed of processors, causing processors to waste clock cycles waiting for data. The speed at which electrons can flow through wires is limited, leading to delays within the chip itself. Finally, increasing a processor's clock speed also increases its power requirements. Increased power requirements leads to the processor generating more heat (which is why overclockers come up with such ridiculous cooling solutions). All of these issues meant that is was getting harder and harder to continue to increase clock speeds.  The designers realized that instead of increasing the core's clock speed, they could keep the clock speed fairly constant, but put more cores on the chip. Thus was born the multi-core revolution.

Unfortunately, the shift to multi-core processors has lead to some serious issues on the software side. Before this change, the clock speed of processors was doubling about every 18 months. Thus a piece of software would get "faster" over time simply because newer processors were faster than the older ones. With multi-core processors, this speed up no longer occurs, clock speeds have settled around the 2 to 3 GHz range. New processors may be slightly faster for non-parallel (single threaded) applications (usually due to architectural changes, as opposed to clock speed increases) however their real increase in power is due to multiple cores. Thus a single threaded application will never be able to utilize the increase in power provided by multiple cores. You could get a processor that is 4, 8 or 16 times as powerful, but a single threaded application will not show a corresponding speed up. Thus if an application is to get faster as newer processors are released, it must be parallel and it must be able to scale to the number of available processors.

For a more in-depth look at the software issues, take a look at The Free Lunch Is Over by Herb Sutter, which originally appeared in Dr. Dobb's Journal, 30(3), March 2005.

An important aspect to recognize is that I am talking about parallelizing for performance reasons. There are certain types or parts of programs that are traditionally implemented using multiple threads, graphical user interfaces (GUIs) for example. However these applications generally use threads to allow them to handle multiple inputs at the same time or to reduce the latency of certain interactions. However the overall performance of such applications is often not improved by having access to multiple cores. Thus even things like GUIs will need to parallelize their time consuming algorithms if they are to take advantage of (and thus get faster on) multiple core machines.

Hopefully I've convinced you that parallelizing applications is not just important, but necessary. In the next article I am going to talk about how parallel programming is different that sequential programming, and why that makes it harder.