In my previous post, I tried to convince you that going parallel is necessary for high performance applications. In this post, I am going to show what makes parallel programming different, and why that makes it harder.
Here are some definitions used in this post:
- process: the operating system level representation of a running executable. Each process has memory that is protected from other processes.
- thread: within a single process each thread represents an independent, concurrent path of execution. Threads within a process share memory.
We think of a function as a series of discrete steps. Lets say a function f, is composed of steps f1, f2, f3... e.g.