Azure Service Bus .NET SDK Deep Dive – Transfer DLQ

Explains where messages go when they can’t be transferred to the destination, for more posts in this series go to Contents.

A while ago, I introduced the concept of deadlettering messages in which I talked about where messages go in case of message expiry, reaching the maximum delivery count, or they were forcefully dead lettered. In the posts about atomic sends as well as send via, I showed how a transaction scope can be used to group send operations together into atomic units that either succeed or fail together. But what happens to the messages when in the case of SendVia the broker cannot deliver the messages to the destination (i.ex. the destination queue no longer exists or is temporarily disabled)?

Azure Service Bus doesn’t disappoint you when it comes to reliable message delivery. Transfer queues, as well as deadletter queues are managed as system queues under the scope of an existing message queue. For example, messages that are being transferred reside in senderQueue/$transfer. Messages that failed to be transferred will be moved to senderQueue/$transfer/$deadletterqueue.

To demonstrate this I created a message receiver that creates a transaction scope and within the transaction scope it sends a message to a destinationQueue.

await using var transactionalClient = new ServiceBusClient(connectionString, new ServiceBusClientOptions
{
    EnableCrossEntityTransactions = true,
});

var receiver = transactionalClient.CreateProcessor(inputQueue);
var sender = transactionalClient.CreateSender(destinationQueue);
receiver.ProcessMessageAsync += async processMessageEventArgs =>
{
    var message = processMessageEventArgs.Message;

    using (var scope = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
    {
        await sender.SendMessageAsync(new ServiceBusMessage("Will not leak"));

        await Prepare.Hazard(connectionString, destinationQueue);

        scope.Complete();
    }
};

You might have noticed the Prepare.Hazard method on line number 16. The method uses a management client to disable the destinationQueue by disallowing send operations to it.

public static async Task Hazard(string connectionString, string destinationQueue) {
    var client = new ManagementClient(connectionString);
    if (await client.QueueExistsAsync(destinationQueue)) {
        var description = new QueueDescription(destinationQueue)
        {
            Status = EntityStatus.SendDisabled
        };
        await client.UpdateQueueAsync(description);
    }

    await client.CloseAsync();
}

Let’s see the code in action.

I do believe this example demonstrates clearly that even if you are not using SendVia it is important to know what kind of deadlettering scenarios can occur and setup monitoring as well as alerts to make sure you don’t miss the fact that messages got moved to the deadletter queues. It will prevent you from running into big surprises like “where the hell is the message gone!” plus makes you sleep way better during the night ;).

Updated: 2021-03-26 to use the new SDK