Seeing concurrency

Modern computer programming makes extensive use of concurrency. Concurrency means two or more paths of code executing at the same time on the same set of resources. Ever since Windows 3.11 allowed us to Alt-Tab between windows, we have all been "multi-tasking".

Concurrency is a powerful way to maximize the use of a system. While one process or thread is waiting on data from a disk drive, the operating system automatically switches to another thread ready to process data from the Internet. A typical system runs with dozens, even hundreds, of processes or threads.

But programming concurrent systems is difficult. The interaction of these various processes and threads creates bugs including dead locks, live locks, and race conditions. Modern debugging tools can help a bit, but it usually boils down to brute force to root out these issues. I've spent countless hours figuring out why the whole system is coming unglued only to discover one synchronization lock out of place.

How is it that we can make extensive us of concurrent programming, but yet it so difficult? How can a topic which is taught in the first year of any computer science program still be baffling to a career software engineer? I believe it's because we lack the ability to see concurrency directly. We know about concurrency. We can explain what it is. But sometimes the interaction of just two threads can surprise us, let alone the interaction of hundreds.

This reminds me of Immanuel Kant's view of human perception. Now, just in case it's not obvious, I don't know squat about Kant. I've read a bit about him, and I've read a bit of him, but just enough to know that I don't actually know anything about Kant. That said, he believed that the certainty we feel in the realms of mathematics and science, in the accuracy of our perception of space and time, is due to our sense perception being space-and-time oriented. "The sensible world and its phenomena are not entirely independent of the human mind, which contributes its basic structure." (http://plato.stanford.edu/entries/kant/).

I think this is why multi-threaded and multi-process programming is difficult. We can't see it. Our natural sense perception is to see time as a single, linear stream. This is how we experience life day to day. So this is how we understand computer code: a single, simple flow of instructions starting at the top and working its way to the bottom.

But we can imagine time in multiple, cyclical streams. I've been a big fan of Doctor Who since Tom Baker days, as he careened though space and time in a British police box. You probably have your favorite time travel story. I think time travel in pop culture and concurrency in computing have a lot in common. But imagining something is not the same as really knowing it.

So how are we able to make such extensive use of concurrency? It's our ability to abstract. Abstraction is the creation of a higher-level representation which hides lower-level details. Abstraction allows us to take a oily, noisy car engine and stick it inside of a shiny, clean auto-body and drive around town with ease. Abstraction allows anyone to play music on an iPod with a touch of a finger, even thought that single touch requires millions of transistors to operate at the speed of light with logical perfection.

So it is with concurrency: we create abstractions which hide the concurrency so that the vast majority of code works in a simple, linear flow of instructions. We isolate the difficulties of synchronizing threads, solve the concurrency problems, then spend the vast majority of time programming within the abstraction of a single-threaded environment.

This understanding of how concurrency is managed reflects the humility and glory of man. We have an incredible ability to create powerful, wonderful electronics. And at the same time we lack the ability to see the interaction of two simple threads.