In my last post we lost billions of dollars from a VIP customer. Let’s not do that again, shall we? In the meantime you should have bought a bigger wallet, I hope you did your homework as well. So the question is how can we improve the situation? Let do a quick recap what went wrong last time:
We were using async void to fire & forget the send operation inside the enlistment implementation. This allowed us to asynchronously dispatch the send operation without blocking the client code. The major drawback of this approach was that because of the nature of fire & forget we would not get any exceptions captured and “marshalled” back to the client. So in the case something goes wrong inside the send operation the whole operation would be lost without us getting notified.
We can address the same way unit testing frameworks are supporting async void methods. Let’s read again the guidance around async void:
Async void methods have different error-handling semantics. When an exception is thrown out of an async Task or async Task<T> method, that exception is captured and placed on the Task object. With async void methods, there is no Task object, so any exceptions thrown out of an async void method will be raised directly on the SynchronizationContext that was active when the async void method started.
https://msdn.microsoft.com/en-us/magazine/jj991977.aspx
Reading the quote above again we notice that the exception is raised on the SynchronizationContext which was active when the async void method started. So if we could somehow hook into that SynchronizationContext we would essentially be capable of intercepting the exception. There is an excellent blog post by Stephen Toub which describes the concept of an async pump.
The magic of the async pump lies in the custom SynchronizationContext and this helper method:
public static void Run(Func<Task> func) { var prevCtx = SynchronizationContext.Current; try { var syncCtx = new SingleThreadSynchronizationContext(); SynchronizationContext.SetSynchronizationContext(syncCtx); var t = func(); t.ContinueWith( delegate { syncCtx.Complete(); }, TaskScheduler.Default); syncCtx.RunOnCurrentThread(); t.GetAwaiter().GetResult(); } finally { SynchronizationContext.SetSynchronizationContext(prevCtx); } }
Let me depict the code a bit for you. It stores the current SynchronizationContext, sets the SingleThreadSynchronizationContext, invokes the Func<Taks> and adds a continuation to the returned task which will call the Complete() method on the SynchronizationContext. As long as the underlying BlockingCollection is not marked as complete (adding) the RunOnCurrentThread call block. When the RunOnCurrentThread method comes back the result is acquired from the task which throws the exception to the caller in the case the task is faulted. So what does this bring to the table you might think? Well we can use the concept of the async pump inside our volatile enlistment like that.
public void Commit(Enlistment enlistment) { AsyncPump.Run(this.onCommit); enlistment.Done(); }
This seems like a minor change. Let’s see if this works and execute our test again
[Test] public async Task VIPCustomerSendingBillionsOfDollars() { var vipCustomer = new AsyncTransactionalMessageSender(); using (var tx = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled)) { await vipCustomer.SendAsync(new Message("First Billion")); await vipCustomer.SendAsync(new Message("Second Billion")); await vipCustomer.SendAsync(new Message("Third Billion")); tx.Complete(); } Debug.WriteLine("But boss we used a TransactionScope and it completed successful!"); }
Now everything works as expected.
The exception raised in the SendInternal method is correctly raised to the caller. Literally we just saved billions of dollars because we now could initiate our retry mechanism or anything and our system wouldn’t assume everything went fine when in reality it didn’t. If this would be a brothers Grimm story I would now say: “And they lived happily ever after”. Unfortunately this isn’t a brothers Grimm story.
We now have a working solution for our volatile transaction enlistment. But we now have another problem introduced. Can you spot it? Stay tuned to the next post in this series.