Performance: Is There a Reason to Use Processes over Threads?

Question

This is a general question but I work in the .NET world so I'd like to know if there any specific quirks about the .NET Framework / Core platforms that I should be concerned about here.

I think it's safe to say that as a general rule, breaking long-running jobs in to threads could be slightly faster than breaking jobs in long-running processes. However, the amount of performance difference would be negligible. I haven't done any benchmarks on this, but this answer seems to indicate that there is an consensus on the matter at least as far as .NET is concerned.

However, I still hear the argument "It was broken up in to separate processes for performance reasons" often. I'm puzzled by this and I wonder if this argument ever holds weight. Please remember, this discussion has nothing to do with maintainability. The question I am asking: are multiple jobs running on a single process each necessarily faster than multiple jobs running on a single thread each inside one process?

Intuitively, I would have to guess that the answer is no. For example, 10 long-running jobs running on 10 threads should run with roughly the same performance as 10 long-running jobs on 10 different processes. But, I see people making the design choice of breaking services up in to smaller parts purely for the purpose of performance.

What I suspect is going on is that shoddy code has created scenarios where starvation is occurring. I think that what is happening is that parallelism is being overused and availability of CPUs is not being honoured so that the benefits of multiple cores are being eroded because the jobs are fighting for CPU power. Still, this doesn't really explain why breaking a job out in to separate processes would improve performance.

So, how can I prove that either a) performance is not improved by breaking jobs out in to separate processes, or b) the improvements made by breaking the code out in to separate processes is only because of poor code design in the first place?

Answer 1

as a general rule, breaking long-running jobs in to threads could be slightly faster than breaking jobs in long-running processes

There was a time when I believed this as well. But I have first-hand, real-world experience that this can be very wrong - specifically with .NET. I have observed several scenarios where the extra isolation by processes gained a speed-up by a factor almost equivalent to the number of CPUs, whilst thread parallelization only gained a speedup of a factor 2 at maximum. Though I did not always understood the reasons behind this in full, I can tell you two (possible) causes for such behaviour:

• Shared memory pool between all threads: not sure how well the garbage collector handles the objects created by different threads, but I am pretty sure this was one of the reasons where the speedup was far below the factor we expected

• Usage of third-party libs which support multithreading, but do not support it very efficiently. An example for this is ADODB for accessing MS Access databases. If one uses thread parallelization to open one connection per thread, each one connecting to a different database file, it seems the different threads disturb each other - still working correctly, but way slower than one could expect.

Hence my recommendation is to try it out: if there is a task to solve where you would expect thread parallelization to bring you a speed factor of "N", where N = "no of CPUs", but you only observe a factor much smaller than N, then try "process parallelization". Of course, this makes only sense if the extra effort for implementing this is acceptable.

I would not be astonished if you make the same observation as me.

Answer 2

I think you are misconstruing people's motivation for using processes. In general, people don't choose processes for speed, they choose processes for the extra isolation, then measure the speed to make sure they aren't sacrificing too much speed.

There are some languages where you usually need processes to gain true parallelism. JavaScript is famously single-threaded, and Python has the Global interpreter lock that will cause serialization within a single process. Also, separate processes are obviously required to be able to scale across multiple physical nodes. In general, though, threads are preferred for speed, and even beyond that, lightweight threads/fibers/greenlets/goroutines/whatever are going to be faster than threads for many use cases.

Answer 3

There is not a real performance benefit for using processes over threads.

The real reason why you hear this is a tiny “white lie”. Thing is: writing multithreaded software is hard and it can get real complex real fast. You have to deal with race conditions, locking, asynchronous calls and a lot more, which make it very error-prone.

So you do the poor man’s multithreading: multiple processes of sequentially coded programs. You still have to deal with some race conditions but all in all, it becomes magnitudes easier to code.

But then when someone ask you why you chose processes over threads, are you going to say: “I’m too bad of a programmer to use threading” or mutter something about performance?

Answer 4

As far as I remember it depends on the OS. If the OS supports only ULT (user level threads) and if your process will do a lot of blocking I/O on different ULTs then it will probably be faster to split it into multiple processes and let the OS schedule them so that every time one of the processes block the OS will schedule one of the other processes and the overall performance will be better, especially in a multiprocessor environment.

Answer 5

There is positive proof there is no difference: Windows only runs threads. Windows does not "run" processes. Processes are a security boundary and memory is managed per-process.