Azure Service Bus .NET SDK Deep Dive – Deadlettering

Explains where messages got when certain conditions are met, for more posts in this series go to Contents.

If we are using queuing, we certainly don’t want to lose messages. After all, those queues decouple the sender from the receiver by persisting the business intent messages and allowing them to handle them whenever the receiver has time. But what if:

• The receiver doesn’t pick up the message before the message’s time to live is expired (Expiry)
• The receiver could not successfully handle the message due to processing errors for multiple times (MaxDeliveryCount)
• The receiver forcefully deadletters a message (Deadletter)

Where should those messages go? At the end of the day, it is probably a business decision what you want to do with those messages, and the answer might vary depending on the message type or the input queue. The good news is though that Azure Service Bus offers dead lettering capabilities. Dead letter queues are dedicated queues that allow messages to be parked there for a period of time until we have a way to deal with those, or we ultimately decide to purge them. The important piece though, is that putting messages to the deadletter queue means they are no longer in the input queue and that is a good thing because they no longer block the receiver from processing other messages in the queue.

Dead letter queues are sub-queues hidden behind a special name called queueName/$DeadLetterQueue. There are more sub-queues per queue, but we will talk about them at a later point in this article series.

To enable deadlettering of messages that expired it is required to opt-in for that when creating the queue by specifying DeadLetteringOnMessageExpiration.

var client = new ServiceBusAdministrationClient(connectionString);
var description = new CreateQueueOptions(destination)
{
    DeadLetteringOnMessageExpiration = true,
    MaxDeliveryCount = 1
};
await client.CreateQueueAsync(description);

Let’s have a closer look at the three scenarios. When sending three message to a destination queue like the following

// Scenario: Expiry
var message = new ServiceBusMessage("Half life")
{
    TimeToLive = TimeSpan.FromSeconds(1)
};
await sender.SendMessageAsync(message);
// Scenario: MaxDeliveryCount
var message = new ServiceBusMessage("Delivery Count");
await sender.SendMessageAsync(message);
// Scenario: Deadletter
var message = new ServiceBusMessage("Poor Soul");
message.ApplicationProperties.Add("Yehaa", "Why so happy?");
await sender.SendMessageAsync(message);

and then receiving them

 receiver.ProcessMessageAsync += async processMessageEventArgs =>
 {
     var message = processMessageEventArgs.Message;
     switch (UTF8.GetString(message.Body)) {
         case "Half life":
             await processMessageEventArgs.AbandonMessageAsync(message);
             break;
 
         case "Delivery Count":
             throw new InvalidOperationException();
 
         case "Poor Soul":
             await processMessageEventArgs.DeadLetterMessageAsync(message,
                 new Dictionary<string, object>
                 {
                     {"Reason", "Because we can!"},
                     {"When", DateTimeOffset.UtcNow}
                 });
             break;
     }
 };

The receiver in our case, does the following:

• When a message with content “Half life” is received it abandons the message so the message will be retried until the expiry time is reached
• When a message with content “Delivery Count” is received, an InvalidOperationException is thrown, which causes the broker to automatically redeliver the message again and again until the MaxDeliveryCount is reached (by default ten retries). If all attempts are failed the message is moved to the deadletter queue
• When a message with content “Poor Soul” is received the client is used to directly deadletter the message with additional metadata to provide a deadletter reason

Let’s explore the state of the deadletter queue once the receiver has done its job.

The above video shows the three messages in the deadletter queue and the first message that was expired. Next up let’s inspect the message which exceeded the delivery count (for the demo I set the MaxDeliveryCount on the queue to one)

and last but not least the message that was explicitly deadlettered

As we can see in the above video, not only the custom dead letter properties have been transferred but also the initial user properties on the message.

Updated: 2021-03-23 to use the new SDK