Azure Service Bus Topology migration with NServiceBus

  1. Azure Service Bus Topologies
  2. Azure Service Bus Topologies with Forwarding
  3. Azure Service Bus Topologies of NServiceBus
  4. Azure Service Bus Topology migration with NServiceBus (current)

In the last post I explained the complexity we introduced for our customers when we created multiple topologies without providing a smooth migration path. In this post I’ll walk you through the online migration path that we created to allow our customers to transition their existing NServiceBus endpoints using the EndpointorientedTopology to the ForwardingTopology piece by piece.

At Particular Software, we strive as much as possible to support our customers and allow them to migrate existing code, patterns or systems as smooth as possible to newer versions of our components. Whenever we can we favor an online migration approach. Let’s face it our customers have mission-critical systems and big-bang solutions where everything has to be taken offline, migrated and redeployed is probably not going to make our customers happy. So when we started brainstorming a solution for migrating customers running on the EndpointorientedTopology to the ForwardingTopology we had three main requirements:

  • Migration should not be a big-bang and thus be aligned with whatever cadence our customers will deploy into production
  • Migration of NServiceBus endpoints should be possible in any order to avoid unnecessary complexity of deployment to production. By any order, we wanted to make sure that it did not matter whether the subscribers were migrated first or the publishers were migrated first
  • Subscribers should be safeguarded from duplicates during the migration period

To fulfill the above requirements we came up with a hybrid topology that bridges the gap between the EndpointorientedTopology and the ForwardingTopology by running a hybrid mode of those two topologies. Since our customers move from the EndpointorientedTopology to the ForwardingTopology the “migration mode” is enabled on the EndpointorientedTopology. The migration mode only affects events. It does not change how events are published (that would break backwards compatibility) but changes how events are subscribed to. Subscribers running in migration mode will no longer fetch messages from subscriptions. Instead, they will perform the following:

  1. Make sure the forwarding topology specific parts are in place in case it isn’t yet by creating; a bundle topic bundle-1 for all events, a subscription for the subscriber endpoint and a filter rule per event type. The subscription will auto-forward the events to the endpoint’s input queue.
  2. Safeguard against duplicated items by creating a migration topic named migration if not present, that performs de-duplication and auto-forwards events to the bundle-1 topic.
  3. Auto-forward all events arriving at <subscriber_endpoint_name>.<event_type_name> subscriptions from the endpoint specific topic <publishing_endpoint_name>.events to the migration topic.

Below is an example of a publisher with two events, EventA and EventB, and a subscriber consuming those events. When an endpoint is running in migration mode, the topology is as follows:

Why is the migration topic required? When an instance of a publisher is been migrated to the forwarding topology, and another instance is still in migration, all events published by the endpoint still in migration need to be de-duplicated. The migration topic ensures events are de-duplicated regardless of which instance published them.

By introducing this migration mode our customers can now smoothly transition to the newly introduced topology without doing a big bang or even worse losing messages that would be already enqueued on the subscriptions (remember they behave like queues).

This wraps up this series on Azure ServiceBus topologies. As a final step, I would like to give a shout out to the ASB team. Their support continues to be outstanding. During the design and implementation phase, we discovered a bug on the broker side that would have made our migration approach not possible. The team triaged the issue and came up with a broker side fix that was rolled out globally so that our customers can benefit from our migration approach. Fantastic!

About the author

Daniel Marbach

2 comments

By Daniel Marbach

Recent Posts