CQRS sequential numbering of business documents

Question

I am currently building a microservice-based system to learn CQRS/ES, Docker, AMQP and all the other goodies that goes with it.

I have never asked a question online before as I am usually pretty good at finding answers to my queries by reading what that others have asked.

This time, I am stumped by what seems to be very simple.

Business documents such as invoices, purchase order, credit memos, etc. usually have an identifying number. (i.e. Invoice No.: 5707)

My question is how would I produce a sequential number for each of these types of documents in an event-sourced system?

I'm concerned about race conditions where an invoice number may inadvertently be duplicated or skipped. What is the best practice for this?

Thank you for your time.

EDIT: Here is what I tried:

I have an aggregate root (AR) called Invoice that has an associated saga called newInvoiceInitialization. The Invoice receives a command CreateNewInvoice. The Invoice AR processes this command and emits an event called NewInvoiceWasCreated.

The newInvoiceInitialization saga handles the NewInvoiceWasCreated event and starts the saga to initialize the invoice AR. The newInvoiceInitialization saga in-turn sends a CheckoutNextInvoiceNumber command to another AR called SequenceNumberGenerator with value objects: CorrelationId and ResourceTypeName.

The SequenceNumberGenerator AR works with a saga called SequenceNumberCheckoutSaga. These two will "check-out" the next InvoiceNumber in the sequence and supply that to the Invoice AR via a SequenceNumberWasCheckoutOut event.

The newInvoiceInitialization saga receives the SequenceNumberWasCheckoutOut event and sends the InvoiceNumber to the Invoice AR with a AssignInvoiceNumberToInvoice command.

When the Invoice AR completes the newInvoiceInitialization saga successfully, it emits the event InvoiceNumberWasAssignedToInvoice. This event triggers the SequenceNumberCheckoutSaga to offers the command FinalizeNumberCheckout to the SequenceNumberGenerator AR which ends the SequenceNumberCheckoutSaga and finalizes the usage of that number.

If the Invoice fails to accept the InvoiceNumber or has other problems it sends the InvoiceNumberAssignmentFailed event which causes the SequenceNumberCheckoutSaga to command the SequenceNumberGenerator AR with a ResetNumberCheckout command to roll-back the checkout-out sequence number and ends the SequenceNumberCheckoutSaga.

This all just seems overly complicated to produce invoices with numbers that aren't missed or duplicated. I am probably missing something.

Answer 1

The TL;DR is: yes, there are technical challenges in preventing collisions, but it's really something you should ask your business department.

---

Best practice from a business perspective isn't a sequential number, but one that contains some structure. If the structure is chosen well, it may help avoid most race conditions.

One example is to use timestamps for numbering, usually in reverse order to make simple file name sorting provide a good oder: YYYYMMDD_document_title.

Another thing I've seen in use is to number documents according to cases and business years. A case number might be a customer number: <case-or-customer>/<year>/<sequence number>.

Any sequence may additionally be prefixed by a namespace identifier, i.e. one might see invoice numbers for a customer be numbered independently from correspondence.

Each business department chooses the system they work best with - so it's really up to them to tell you how to create this number.

---

That said, even if you do create numbers in such a way, there could still be collisions. You mention REDIS as part of your solution; REDIS can be used to create atomically incrementing counters.

I suspect that the best pattern for you would be to:

• Choose a prefix generation method as I outline above.
• Use INCR to generate unique counter values.
• Concatenate your document name from those components and the title, e.g.: <prefix>-<counter>-<title>.

Answer 2

I want to strongly discourage directly using DB id's as user readable identifiers of any kind.

Every unique identifier should only be beholden to one system. If you publish your DB id's where users see them they'll find some reason to mess with your system. Steve Jobs famously insisted on a having a different employee id number when he learned he was number 2. Rather than take number 1 from Waz he demanded to be employee number 0. Just don't let users see these numbers and this noise stays out of the low levels of your system. A simple mapping could have solved this problem and kept DB id's from being beholden to users.

An invoice number sounds like it could just be a DB id. Here's why not: you need to understand that uniqueness is only meaningful within a scope. A DB id is unique to a DB table. To a DB. But an invoice is unique to a business. Some businesses have more than one DB. Some get other DB's as they expand and grow. Some get other DB's as they buy other companies.

Don't even pretend that you can predict this kind of change. Maintain the uniqueness of your table by hiding the ID from anything that doesn't have to have a direct relationship with your table. Make the other things like invoices go through something that understands what those other things are. That translation is a business rule. Keep that business rule out of your DB.

This way your DB will work if you buy other companies or if other companies buy you because your DB doesn't care.

If you feel like so far I've just dodged the problem you're right. I have. That was the point. But if you want me to lay out a plan for the business rule consider an invoice number that has the ability to grow as more factors need to be added to ensure uniqueness. Then, much like a composite key, as conflicts arise they can be deconflicted by adding tags that place the rest of the invoice in it's previous scope. This is wildly different then trying to predict what these conflicts will be because no structure is imposed on what these new tags will be until we know what they are. All we've done is recognize that the invoice number may be required to expand. This may be as simple as using letters for former companies, allowing a delimiter like a dash, or a new field like "group" (that's what some insurance companies call it).

If you're thinking: Oh so I just expose the structure of the company you're really missing the point. You don't have to show how anything works under the covers. You just have to allow what you show to change as your needs change even while what's under the covers doesn't change.

So long as your design allows for this expansion, it doesn't have to predict it. Predicting what form change will take and getting it wrong does more damage than having left it alone. Just keep your direct exposure minimized and the work change causes will be minimized as well.