Machine.Specifications – The alternative nunit

In my last post, I explored xunit as an alternative to MSpec. In this blog post, I’m going to do the same with NUnit. Most people underestimate the power of NUnit. It is a great testing framework that has been around since the early days of .NET and Open Source and is still actively maintained. When it comes to tool integration, NUnit has one major advantage: The Resharper plugin is developed by JetBrains and shipped together with every Resharper release. So everybody which uses NUnit never has to worry about if there will be a Resharper integration. Furthermore, they’ll never worry about the plugin no longer being maintained (like in other One-Man-Show-Plugins). In fact there is even a 3.0 Beta release currently available of NUnit, all development is done on Github. Be sure to check out the release notes! Yes, yes nitpickers, it uses attributes and such but who cares?

So to reimplement the basic functionality of Machine.Specifications you need two things

• The naming class
• The catch class
• Basic infrastructure from NUnit.Specifications

So let’s get started with the basic piece of infrastructure:

The code below as said is from NUnit.Specifications. The code here is for illustration purposes. I suggest you use NUnit.Specifications directly. If you copy paste this code make sure you follow the license defined by Derek Greer.

    using CategoryAttribute = NUnit.Framework.CategoryAttribute;

    // [DebuggerStepThrough]
    [TestFixture]
    public abstract class it
    {
        protected Exception Exception;

        [TestFixtureSetUp]
        public void TestFixtureSetUp()
        {
            InvokeEstablish();
            InvokeBecause();
        }

        [TestFixtureTearDown]
        public void TestFixtureTearDown()
        {
            InvokeCleanup();
        }

        void InvokeEstablish()
        {
            var types = new Stack<Type>();
            Type type = GetType();

            do
            {
                types.Push(type);
                type = type.BaseType;
            } while (type != typeof(it));

            foreach (Type t in types)
            {
                FieldInfo establishFieldInfo =
                    t.GetFields(BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.FlattenHierarchy)
                        .SingleOrDefault(x => x.FieldType.GetCustomAttribute<SetupDelegateAttribute>() != null);

                Delegate establish = null;

                if (establishFieldInfo != null) establish = establishFieldInfo.GetValue(this) as Delegate;
                if (establish != null) Exception = Catch.Exception(() => establish.DynamicInvoke(null));
            }
        }

        // [EditorBrowsable(EditorBrowsableState.Never)]
        void InvokeBecause()
        {
            Type t = GetType();

            FieldInfo becauseFieldInfo =
                t.GetFields(BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.FlattenHierarchy)
                    .SingleOrDefault(x => x.FieldType.GetCustomAttribute<ActDelegateAttribute>() != null);

            Delegate because = null;

            if (becauseFieldInfo != null) because = becauseFieldInfo.GetValue(this) as Delegate;
            if (because != null) Exception = Catch.Exception(() => because.DynamicInvoke(null));
        }

        // [EditorBrowsable(EditorBrowsableState.Never)]
        private void InvokeCleanup()
        {
            try
            {
                Type t = GetType();

                FieldInfo cleanupFieldInfo =
                    t.GetFields(BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.FlattenHierarchy)
                        .SingleOrDefault(x => x.FieldType.GetCustomAttribute<CleanupDelegateAttribute>() != null);

                Delegate cleanup = null;

                if (cleanupFieldInfo != null) cleanup = cleanupFieldInfo.GetValue(this) as Delegate;
                if (cleanup != null) cleanup.DynamicInvoke(null);
            }
            catch (Exception e)
            {
                Console.WriteLine(e);
            }
        }

        public IEnumerable GetObservations()
        {
            Type t = GetType();

            var category = (CategoryAttribute)t.GetCustomAttributes(typeof(CategoryAttribute), true).FirstOrDefault();
            string categoryName = null;

            if (category != null)
                categoryName = category.Name;

            IEnumerable<FieldInfo> itFieldInfos =
                t.GetFields(BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.FlattenHierarchy)
                    .Where(x => x.FieldType.GetCustomAttribute<AssertDelegateAttribute>() != null);

            return itFieldInfos
                .Select(fieldInfo => new TestCaseData(fieldInfo.GetValue(this))
                .SetDescription("Jehaha")
                    .SetName(fieldInfo.Name.ToFormat())
                    .SetCategory(categoryName));
        }

        [Test, TestCaseSource("GetObservations")]

        public void should(Delegate observation)
        {
            if (Exception != null)
                throw Exception;

            observation.DynamicInvoke();
        }
    }

I know this is a large piece of code. But it is fairly simple if you understand how it works. Let me quickly walk through it. To simulate the same behavior like Machine.Specifications we are using the TestFixtureSetup and Teardown methods. In the TestFixtureSetup, we are reflecting over all fields in the current hierarchy of type Establish and Because. Then we are first executing all Establish delegates by starting with the Establish delegate highest in the inheritance hierarchy and then the one Because delegate in the hierarchy. Exceptions are catched and rethrown accordingly. When the setup phase is over the TestCase in the method should is executed. As TestCaseSource, the method GetObservations is used. The method reflects over all It delegates and dynamically creates TestCases with the appropriate Category and Name by using the Name formatting class from Machine.Specifications. Because all the reflection code used in the base class uses the newly introduced delegate decoration attributes this should automatically work with custom delegate types.

All you have to do then is to tweak your specs so that your base specs use the it class. Which then looks like the following

    [Subject(typeof(Account), "Funds transfer")]
    [Tags("failure")]
    public class when_transferring_an_amount_larger_than_the_balance_of_the_from_account
      : AccountSpecs
    {
        static Exception exception;

        Establish context = () =>
        {
            fromAccount = new Account { Balance = 1m };
            toAccount = new Account { Balance = 1m };
        };

        Because of =
          () => exception = Catch.Exception(() => fromAccount.Transfer(2m, toAccount));

        It should_not_allow_the_transfer =
          () => exception.Should().BeOfType<Exception>();
    }

    public abstract class AccountSpecs : it // notice here!
    {
        protected static Account fromAccount;
        protected static Account toAccount;
    }

Everything else remains unchanged. As long as you are not using Because_of, IAssemblyContext, ICleanupAfterEveryContextInAssembly or a custom result provider you are fine with this approach. Furthermore, if you would like the same HTML output report as with Machine.Specifications you would need to roll up your sleeves and do some fancy XSLT transformation of the standard NUnit report.

And here two screenshots to show how it looks like

and the executed tests

As you can see above there is no longer an execution “bubble” on the It (which doesn’t make sense anyway how Machine.Specifications is currently implemented) and because the base class is called It and the test case method called should it automatically reads nice. Of course if you’d be using the Given, When, Then syntax with custom delegates you would need to rename the base class and the test case method accordingly.

There is one last drawback left over. The node right under the category has still the underscores in it. Ugly? Yeah. But who cares? If you do then here is a pointer to a possible workaround.

You could attempt to write a custom NUnit addin which overrides the test names

[NUnitAddin(Description = "Machine.Specs", Name = "Machine.Specs", Type = ExtensionType.Core)]
    public class NUnitAddinWhichDoesCrazyThings : IAddin
    {
        public bool Install(IExtensionHost host)
        {
            IExtensionPoint testDecorators = host.GetExtensionPoint("TestDecorators");
            testDecorators.Install(new Decorator());
            return true;
        }
    }

    public class Decorator : ITestDecorator
    {
        public Test Decorate(Test test, MemberInfo member)
        {
            test.TestName.Name = Naming.ToFormat(test.TestName.Name);
            test.TestName.FullName = Naming.ToFormat(test.TestName.FullName);
            return test;
        }
    }

The last time I tried this with Resharper the tree nodes weren’t updated properly. But I might not have tried hard enough 😉 The next and last post in this series will be a quick summary. Stay tuned.