How to explain why multi-threading is difficult

Question

I am a fairly good programmer, my boss is also a fairly good programmer. Though he seems to underestimate some tasks such as multi-threading and how difficult it can be (I find it very difficult for anything more than running a few threads, waiting for all to finish, then return results).

The moment you start having to worry about deadlocks and race conditions, I find it very difficult, but the boss doesn't seem to appreciate this - I don't think he has ever come across this. Just slap a lock on it is pretty much the attitude.

So how can I introduce him, or explain why he might be underestimating the complexities of concurrency, parallelism, and multi-threading? Or maybe I am wrong?

Edit: Just a bit on what he has done - loop through a list, for each item in that list create a thread that executes a database update command based on the information in that item. I'm not sure how he controlled how many threads executed at once, I guess he must have added them to a queue if there were too many running (he wouldn't have used a semaphore).

Answer 1

Multi-threading is simple. Coding an application for multi-threading is very, very easy.

There's a simple trick, and this is to use a well-designed message queue (do not roll your own) to pass data among threads.

The hard part is trying to have multiple threads magically update a shared object in some way. That's when it gets error-prone because folks don't pay attention to the race conditions that are present.

Many folks don't use message queues and try to update shared objects and create problems for themselves.

What becomes difficult is designing an algorithm that works well when passing data among several queues. That's difficult. But the mechanics of co-existing threads (via shared queues) is easy.

Also, note that threads share I/O resources. An I/O bound program (i.e., network connections, file operations or database operations) is unlikely to go any faster with lots of threads.

If you want to illustrate the shared object update problem, that's simple. Sit across the table with a bunch of paper cards. Write down a simple set of calculations -- 4 or 6 simple formulas -- with lots of room down the page.

Here's the game. You each read a formula, write an answer and put a card down with the answer.

Each of you will do half the work, right? You're done in half the time, right?

If your boss doesn't think much and just starts, you'll wind up conflicting in some way and both writing answers to the same formula. That didn't work because there's an inherent race condition between both of you reading before writing. Nothing stops you from both reading the same formula and overwriting each other's answers.

There are many, many ways to create race conditions with badly- or un-locked resources.

If you want to avoid all conflicts, you cut the paper up into a stack of formulas. You take one off the queue, write the answer down, and post the answers. No conflicts because you both read from a one-reader-only message queue.

Answer 2

1. If you can count on any mathematical experience, illustrate how a normal execution flow that is essentially deterministic becomes not just nondeterministic with several threads, but exponentially complex, because you have to make sure every possible interleaving of machine instructions will still do the right thing. A simple example of a lost update or dirty read situation is often an eye-opener.

2. "Slap a lock on it" is the trivial solution... it solves all your problems if you're not concerned about performance. Try to illustrate how much of a performance hit it would be if, for instance, Amazon had to lock the entire east coast whenever someone in Atlanta orders one book!

Answer 3

Multi-threaded programming is probably the most difficult solution to concurrency. It basically is quite a low level abstraction of what the machine actually does.

There's a number of approaches, such as the actor model or (software) transactional memory, that are much easier. Or working with immutable data structures (such as lists and trees).

Generally, a proper separation of concerns makes multi-threading easier. Something, that is all to often forgotten, when people spawn 20 threads, all attempting to process the same buffer. Use reactors where you need synchronisation and generally pass data between different workers with message queues.
If you have a lock in your application logic, you did something wrong.

So yes, technically, multi-threading is difficult.
"Slap a lock on it" is pretty much the least scalable solution to concurrency problems, and actually defeats the whole purpose of multi-threading. What it does is to revert a problem back to a non-concurrent execution model. The more you do it, the more it is likely, that you have only one thread running at the time (or 0 in a deadlock). It defeats the whole purpose.
This is like saying "Solving the problems of the 3rd world is easy. Just throw a bomb on it." Just because there is a trivial solution, this doesn't render the problem trivial, since you care for the quality of the result.

But in practice, solving these problems is just as hard as any other programming problems and is best done with appropriate abstractions. Which makes it quite easy in fact.

Answer 4

I think there's a non technical angle to this question - IMO it's an issue of trust. We commonly get asked to reproduce complex apps like - oh, I don't know - Facebook for example. I have come to the conclusion that if you are having to explain the complexity of a task to the uninitiated/management - then something's rotten in Denmark.

Even if other ninja programmers could do the task in 5 minutes, your estimates are based on your personal ability. Your interlocutor should either learn to trust your opinion on the matter or hire someone whose word they are willing to accept.

The challenge is not in relaying the technical implications, which people either tend to ignore or are unable to grasp through conversation, but in establishing a relationship of mutual respect.

Answer 5

One simple thought experiment to understand deadlocks is the "dining philosopher" problem. One of the examples I tend to use to describe how bad race conditions can be is the Therac 25 situation.

"Just slapping a lock on it" is the mentality of someone who hasn't come across difficult bugs with multi-threading. And it is possible that he thinks you are overstating the seriousness of the situation (I don't - it is possible to blow stuff up or kill people with race condition bugs, especially with embedded software that ends up in cars).

Answer 6

Short answer in two words: OBSERVABLE NONDETERMINISM

Long answer: It depends on which approach to concurrent programming you use given your problem. In the book Concepts, Techniques, and Models of Computer Programming, authors clearly explain four main practical approaches to writing concurrent programs:

• Sequential programming: a baseline approach that has no concurrency;
• Declarative concurrency: usable when there is no observable nondeterminism;
• Message-passing concurrency: concurrent message passing between lots of entities, where each entity internally process the message sequentially;
• Shared state concurrency: thread updating shared passive objects by means of coarse grained atomic actions, e.g. locks, monitors and transactions;

Now the easiest of these four approaches apart from obvious sequential programming is declarative concurrency, because the programs written using this approach have no observable nondeterminism. In other words, there are no race conditions, as race condition is just an observable nondeterministic behaviour.

But the lack of observable nondeterminism means, that there are some problems we can not tackle using declarative concurrency. Here is where the last two not so easy approaches come into play. The not so easy part is a consequence of observable nondeterminism. Now they both fall under stateful concurrent model and are also equivalent in expressiveness. But because of the ever increasing number of cores per CPU, it seems the industry has taken more interest recently in message passing concurrency, as can be seen in the rise of message passing libraries (e.g. Akka for JVM) or programming languages (e.g. Erlang).

The previously mentioned Akka library, which is backed by a theoretical Actor model simplifies building concurrent applications, as you don't have to deal any more with locks, monitors or transactions. On the other hand it requires different approach to designing solution, i.e. thinking in a way of how to hierarchically composite actors. One could say that it requires a totally different mindset, which again can be in the end even harder than using plain state shared concurrency.

Concurrent programming is hard because of observable nondeterminism, but when using the right approach for the given problem and the right library that supports that approach, then a lot of issues can be avoided.

Answer 7

Concurrent applications are not deterministic. With the exceptionally small amount of overall code the programmer has recognized as vulnerable, you do not control when a part of a thread/process executes in relation to any part of another thread. Testing is harder, takes longer, and is unlikely to find all concurrency related defects. Defects, if found, are ofhen subtle cannot be consistently reproduced, hence fixing is difficult.

Therefore the only correct concurrent application is one that is provably correct, something not often practiced in software development. As a result, the answer by S.Lot is the best general advice, as message passing is relatively easy to prove correct.

Answer 8

I was first taught that it could bring up issues by seeing a simple program that started 2 threads and had them both print to the console at the same time from 1-100. Instead of:

1
1
2
2
3
3
...

You get something more like this:

1
2
1
3
2
3
...

Run it again and you may get totally different results.

Most of us have been trained to assume that our code will execute sequentially. With most multi-threading we cannot take this for granted "out of the box".

Answer 9

Try using several hammers to mash in a bunch of closely spaced nails at once without some communication between those holding the hammers... (assume that they're blindfolded).

Escalate this to building a house.

Nowtry to sleep at night imagning you're the architect. :)

Answer 10

Easy part: use multithreading with contemporary features of frameworks, operating systems and hardware, like semaphores, queues, interlocked counters, atomic boxed types etc.

Hard part: implement the features themself using no features in first place, may be except few very limited capabilities of hardware, relying say only on clocking coherence guarantees across multiple cores.