How does immutability remove the need for locks when two threads are trying to update the shared state?

Question

Okay so I read through this:

Does immutability entirely eliminate the need for locks in multi-processor programming?

And this was the main takeaway for me:

Now, what does it get you? Immutability gets you one thing: you can read the immutable object freely, without worrying about its state changing underneath you

But that was only regarding reading.

What happens when two threads are trying to generate a new shared state? Lets say they're both reading some immutable number N, and want to increment it. They can't mutate it directly so the both generate two completely new values at the same time both of which are just N + 1.

How do you reconcile this problem so that the shared state becomes N + 2? Or am I missing something and that's not how it works?

Answer 1

So I think that we need to eliminate the term "shared state" from your question, because shared state is almost diametrically opposed to the notion of using immutability to avoid locking.

In your example, you basically said that both read some value "N" and both create a new object with a value "N+1".

The key is that you wouldn't necessarily save the value "N+1". Rather, you would save the values "N" and "+1" inside both threads 1 and 2. In other words, you would save a reference to the original value you read as well as the modification that you made to it.

Now, the "shared state" should instead be a 3rd thread that reconciles the two (very often this 3rd thread is the thread that originally created the first 2). When reconciling the two "N+1" values, it should see that both started with "N" and both did a modification of "+1". The final result is "N+2".

It is important to recognize that "N+2", when saved, will also be a new immutable object that cannot be changed. It is this lack of this "shared state" that allows you to avoid the need for locks.

Answer 2

What happens when two threads are trying to generate a new shared state?

Let's be clear about what I understand your question to be:

You have some mutable variable of immutable state. Let's use an int for simplicity:

int x = 42;

Then you want two threads to both try to increment x by 1.

Then you get to synchronize them. Immutability provides little value here.

All immutability guarantees is that the variable you're reading from isn't in some half-way state when read. Since x is atomic, this doesn't make much sense. Any read you do will get the whole value.

But since you're mutating the variable x, you need synchronization even though the value is immutable. Each thread is making 2 atomic operations:

y = read x;
write x with y+1;

If both threads read before write, then both threads will see x, not x+1. So you need to synchronize things

But what if x was a pair?

Pair x = {2,4};

Then having immutable Pairs will guarantee that both values are changed at the same time.

But the threads have 4 atomic operations with mutable Pairs:

y = read x.x;
write x.x with y+1;
y = read x.y;
write x.y with y+1;

Every one of them can be interrupted depending on the concurrency of the threads. With immutable Pairs, it forces you to do something like this:

tmp = x;
y = new Pair(tmp.x+1, tmp.y+1);
x = y;

You still need synchronization, because you have the mutable variable x, but the reference copy to tmp is atomic. Since tmp is local, and you know it's not being modified, it doesn't need synchronized even though you're doing two different operations (to read tmp.x and tmp.y).

If you think about how objects are used, most of the time, you just want some snapshot and do some operations on it. If you weren't updating x above, you wouldn't need synchronization. The copy to tmp will happen automatically when you pass x into some function.

But since you were asking specifically about updating a mutable reference, you don't get the immutability benefits. If the reference was also read only, then there would be no need to synchronize anything since nothing can change.

Answer 3

The problem that you have is that you're looking at simplified example, which is so far simplified that you've entirely removed the immutable state from it and left only a single (albeit atomic) mutable value. That's not a realistic example of an immutable system.

To bring some immutability back into it, consider a different example. Two threads are generating objects somehow and inserting them into a shared map structure. The map is implemented as an immutable map, I.e. it has an insert operation that returns a modified copy of the map, leaving the original intact. In this case, what happens is that each thread, when it wants to add something, has a few different options:

1. It could use a lock, create an updated copy and replace it, then unlock it for other threads. This is a simple approach, and is similar to how you'd perform the operation with a non-immutable map, other than that reader threads don't need to acquire the lock. But we can do better:

2. It can grab the reference to the current map into private storage, make a new map by inserting into it, and then performing an atomic test and set operation to change the reference. If the value has changed while it has been working, it must redo the operation, but because the operation is a pure function of immutable state we know that it has had no side-effects so can be repeated as many times as necessary to make the update work. If too many threads are trying to update the same state it can become a bottleneck, so falling back on locking if too much contention is detected is a good idea. In many scenarios this can perform better than locking

3. Passing off the operations to be performed on mutable state to a third thread that can serialise them so that the generator threads do not need to worry about concurrency. This offers the highest throughput of all options, but comes at the expense of system resources for maintaining the serialisation thread.

In an environment where we can package up pure mutation operations like this you can write a library that manages all of this automatically, so you just need to pass it a function for updating the value and it can decide which strategy to take. Many languages have libraries that implement "software transactional memory" (which is the name of the technique number 2 above) - some of them will also perform locking and/or serialisation as and when they feel it appropriate. But all of this is made possible because the objects in use are immutable and there are only a small number of mutable references to them.

Answer 4

Aren't locks only needed if you're changing state?

There is a subtlety here. Locks are needed not only if the current thread wants to modify the state, but if any any other thread might modify the state. This means that you can only safely elide the object if you know that no other part of the system will modify it. In other words, you can only elide the lock if the object is immutable.

Saying that locks aren't needed for immutable objects is just the same as saying that locks aren't needed if you only read the object.

But additionally, it also means that by restructuring your code to use immutable objects, we can get rid of locks. Let's consider a simple case, implemented using mutable objects:

List list = new List();
void worker() {
   for(...) {
      synchronized(list) {
         list.add(...)
      }
   }
}

for (...) {
    start(worker)
}

for (...) {
   waitForWorkerToFinish();
}

This will work, but not very well because all the threads will be fighting over the lock. Here's the immutable solution:

List worker() {
   List list;
   for(...) {
      list.add(...)
   }
   return list;
}

for (...) {
    start(worker)
}

List list;
for (...) {
   list.addAll(waitForWorkerToFinish());
}

Notice how there are no locks in the second example. (Well, there are probably locks implementing start and waitForWorkerToFinish, but worker doesn't have to acquire locks). Because it has gotten rid of the lock, it will be faster.

Answer 5

The answer is pretty easy, with immutability, you don't change state rather you create new state(s).

So you have 2 processes both getting input from the same state. What you end up with is 3 things: first the original state, and 2 new states which is the output from the 2 processes.

What you need now is a third process dedicated to putting those states from the other two processes together and come up with a final answer.

Answer 6

That's not how it works.

If you have two threads reading and incrementing a shared variable, it's not immutable. When you mark something as immutable, you're telling everyone (and yourself) that the value isn't going to change.

Answer 7

If you'll forgive me for using a more complex example with hefty work worth parallelizing and not just atomic operations to a numeric variable, consider a video game loop which has to apply artificial intelligence, then physics, then rendering in that order:

Here even mutable concurrent data structures for the game state won't let us effectively evaluate this as a true pipeline without just locking and bottlenecking and causing everything to still wait on each other, because artificial intelligence needs to be applied completely before physics and make its changes visible to it before it can be applied, and both need to be applied completely before rendering a frame to avoid artifacts. We might be able to use multiple threads to speed up one of these in their implementation to prevent the other two from waiting as long, but we cannot evaluate it in parallel.

But what happens if our game state becomes immutable? In that case we no longer have "shared state". We have a pipeline focusing on inputting a former game state and outputting a new game state:

And while the first pass would require that physics wait for the first output from AI, and for the renderer to wait for the first output from physics, at this stage the parts of the pipeline can now be running in parallel cranking out new outputs as a true, parallel pipeline rather than a serial executor.

And this can potentially be done in a lock-free way with guaranteed system-wise throughput. So this is just one example but it's one very familiar to me, and one where I benefited quite a bit (with huge improvements to frame rates, not just in terms of faster frame rates but more consistent and smoother frame rates) by making each stage of the pipeline pure (free of external, observable side effects), for which immutability also helped to enforce.

If you can't help but make thread B depend on thread A to finish and see its modified output with that sort of order dependency, then you can still at least turn that into a true pipeline where the two can be running in parallel (thread B can be focused on cranking out a new output for thread C, perhaps, while thread A is simultaneously working on cranking out a new output for thread B), and all without locks, provided we are not mutating shared state.

With that said, whether the state is mutable or not is actually not so relevant to achieving this type of concurrency and lock-free thread-safety, and I hope I don't upset the functional purists out there by saying that (I'm coming at this pursuing purity in imperatives languages like C and C++ where practicality forces me to take a relaxed view about immutability). There are lots of benefits to immutability that help enforce purity and help simplify maintaining invariants, but the ultimate thing that helps here is to make the state no longer shared among threads. Immutability enforces that very strictly, but you can achieve this kind of pipeline with a mutable interface design for your game state provided it's not shared among threads, with each thread inputting a copy and outputting a new, modified version.

How do you reconcile this problem so that the shared state becomes N + 2? Or am I missing something and that's not how it works?

That normally becomes a matter of atomic operations. But I imagine for a more complex example and what you're really after, if one thread must make its output visible to the other conceptually, then you still don't need to lock on shared state because you can apply the pipeline approach mentioned above. The second thread will initially need to wait (it doesn't need to lock necessarily) on the first thread to feed it the initial output, but then the first thread can start churning out new outputs in parallel right after while the second thread is doing its thing. You can at least achieve a true, parallel pipeline in these cases by avoiding shared state and favoring a pure I/O nature (as opposed to a mutating nature causing external side effects) to the design.

Answer 8

There's an immutable object with a value of N. Two threads each want to create an immutable object with a value of N+1. They can do that without any locking just fine.

What you didn't mention, is that quite likely there is somewhere a shared mutable reference to the first object, and two threads want to change that shared mutable reference to point to a new immutable object with a higher value. Changing that mutable reference is where you will need a lock. If you never store a reference to the new objects in a shared place, you don't need locks.