This blogpost is part of a larger blog post series. Get the overview of the content here.
Never assume messages will arrive in the order you sent them. If it works fine on your machine that doesn’t mean it will do the same in your production environment. Especially under heavy load the queuing system might decide to throttle throughput. By just using one thread and a queuing system that guarantees order might help you on infrastructure level. But communication frameworks like NServiceBus manage exceptions by retrying a few times. During the retry phase a message could be moved to the error queue. The message can be retried even days after it arrived to its original destination. So at application level messages can still be processed out of order. Write explicit code which deals with that fact and behaves consistently in these situation. Let us consider the following scenario. Three parties are involved: namely sales, billing and shipping. Sales publishes an order accepted event which is subscribed by the billing and shipping. Billing will bill the order and as soon as the bill is covered it will raise an order billed event. Shipping will create shipping information as soon as an order gets accepted. Imagine a race condition where the order billed event arrives before the order accepted event in shipping. In this case shipping knows nothing about that particular order. Retrieving of shipping information correlated by the order identification will fail with an exception. But that is perfectly fine! Just consider retrying messages later in those cases. This is usually a huge mind shift. By relying on retrying mechanism and the fact that messages may arrive out of order you also expect exceptions to be happening during processing. But in certain business scenarios there is even a smarter solution available. Instead of relying on retries you can simply design your message more intelligently. What does that mean? Well you can answer that yourself by thinking about the following problem:
Here is a short exercise for you: A sender sends a lot of price updates for products. A receiver needs to process them correctly. How can you design your messages so that out of order processing doesn’t screw your system up?
I cannot stress this enough. Messaging is not the answer to every problem! Sometimes classical approaches like Remote Procedure calls or HTTP APIs are just fine. As always it is about tradeoffs. There is a cost involved when switching to a messaging infrastructure. Even the business needs to rethink in terms of business requirements. A big bang approach is almost certainly doomed to fail. Do small changes around the edges. Do not apply messaging to the whole architecture. Use the edges on which messaging might give the most benefits. Let the business get accustomed to messaging by using tiny little steps. But messaging alone is not enough. We will cover more in the next post.