0

Let's say there is a user table and has field called coins to represent the coins available for that user in the system. At this stage a certain user has 50 coins in his account. Same user do the following requests almost instantly, but in the following order.

  1. Buy something for 20 coins
  2. Insert a promo code that doubles the coins

If the 1st request fails, in a properly written monolithic application, the user will end up with 100 coins.

But in a microservices based application where users are maintained in the separate microservice with a separate database and orders are maintained in a separate microservice with a separate database which utilize SAGA pattern to handle transactions, the following scenario can happen.

  1. At some stage of the SAGA, 20 coins will be removed from the account. Now user has 30 coins.
  2. Promo code will be applied. Now user has 60 coins.
  3. At later stage of the SAGA, buying the item fails. So it initiate a rollback.
  4. User will be given 20 coins at a stage of the rollback.

At the end, user will own only 80 coins, which is wrong.

Is there any way to fix this? Any suggestion?

Ramesh-X
  • 109

1 Answers1

1

You have to operations/events:

  • (A)20 coins were removed from an account and then added back
  • (B) balance of an account was doubled

And in your monolith application you guaranteed to execute them in some order - either A,B or B,A.

In either order of events, the outcome is 100. The property to help you to do this is serializability - transactions gets executed in some order - but one by one, not at the same time.

Clearly you run in a problem if you do truly execute two transactions at the same time. This problem may even arise is a monolith application if an improper isolation level is enabled b/n transactions.

So what do we need to "fix" in a distributed system to get same behaviour? There are several options. The one I would use in your case is locking. A system which operates on a user's balance has to get a lock to execute. In that case the order will be either A,B or B,A but the outcome will be the same.

If we take your example to a bit higher level, we can say that there is causal dependency between A and B, since an outcome of A is needed for B to work correctly. Before doubling coins (B) you have to know if purchase (A) succeeded. This means that your operations are not concurrent, where concurrency is not a time thing, but a part of concept "happens before".

So to resolve your specific problem in practice, you could use zookeeper to manage distributed lock on account balance. Zookeeper is actually pretty known for being suitable for these types of tasks.

We could take the solution to the next level and introduce a consensus based system, but that will be much more complicated to understand and implement.

AndrewR
  • 196