Let’s save our Thanksgiving sales with Azure Service Bus

As we saw in the last post, naive receiving of messages with Azure Service Bus destroys our Thanksgiving sales. But how can we do better to satisfy our bosses and first and foremost our precious customers? As Sean Feldman rightfully pointed out in the comments section, our sample was using MaxConcurrentCalls set to one.

How silly! Who wrote that… Oh, it was me 😉

So how about we correct my mistake and set to 100 to like the following

receiveClient.OnMessageAsync(async message => {...},
new OnMessageOptions { AutoComplete = false, MaxConcurrentCalls = 100 })

With this tiny change, we can suddenly receive 100 messages concurrently as well as complete and abandon (if necessary) them concurrently. Let’s do the math again. Our 120 K orders we received ca now be processed in 20 minutes instead of 33 hours. On a lucky Thanksgiving rush, we’d be able to process the 1.2 million orders we received in about 200 minutes or 3 hours and 20 minutes. That is a significant improvement for such a tiny change!

The hard part is to pick a decent concurrency number. From an Azure SDK perspective setting the MaxConcurrentCalls to a higher number doesn’t necessarily mean the SDK will use a significant amount of more resources like Threads. Internally the SDK is almost completely asynchronous using the old APM style. The concurrency is limited by using an asynchronous semaphore* which is acquired before each asynchronous receive and released as soon as the callback delegate is executed (that is a bit of a simplification but let’s stick to that analogy). So potentially we could set it to a much higher number.

But it depends on what we are executing inside the callback delegate. For example, if we’d be executing operations against a database like SQL Azure for argument’s sake we’d be needing dedicated connections for each concurrent access to the database. SqlConnections are pooled, and by default the pool, size is set to 100. So if we’d set the concurrency to a higher number, we would eventually be exhausting the connection pool, and the throughput would suffer significantly. So let’s stick for now with 100 for MaxConcurrentCalls.

We went from Snail orders to orders that are processed at least as fast as my son can drive his bobby car. So I think your Thanksgiving sales is almost saved and we can enjoy our turkey. Or can we do even better? Of course, we can! But that is a topic for another post.

 

* A semaphore is like a nightclub: it has a certain capacity, enforced by a bouncer. Once it’s full, no more people can enter, and a queue builds up outside. Then, for each person that leaves, one person enters from the head of the queue. […] Albahari