Tests are Documentation, or are they?

Yesterday evening, I gave a workshop titled “To test, or not to test” at the Software Crafters Zürich Meetup. In the workshop, we gathered reasons to write tests: being confident that the code works, being confident that regressions can be prevented, helping to drive the implementation, and having documentation of the system. Interestingly, when I prepared the workshop, I forgot about the documentation aspect of the tests. Here is why and why it matters.

We used tests as documentation in the past, too.

A couple of years earlier, I wouldn’t have forgotten to add “documentation” to the list of test benefits. We went to take a look at the tests regularly to get information about how the system works. The tests showed us how the system behaves, and listed all the side-effects to be expected when executing a command (the assert part of the test). All the things that need to be around for a command or query to be successful (the arrange part of the test), and how to call the system (the act part of the test).

But that changed once we switched from C# to F#.

Now, we use the production code as documentation.

After a while of writing our code mostly with F#, we noted that when we were wondering how a certain piece of our software worked, we didn’t first go to the test anymore—we just jumped directly into the implementation. After some reflection, we came to the conclusion that well-written F# production code is even more concise than the code testing the behavior. Reading the production code directly to understand how it works became faster than reading the corresponding tests. The production code shows what is going on in a very concise way, whereas the tests are “cluttered” with test data. Don’t get me wrong; our tests are well-written and nice. However, looking directly at the production code became the fastest way to understand it.

Why this difference?

One day, we discussed this insight in the team and looked for reasons for our changed behaviour.

Overall, we recognised that the F# code was much more focused on the business case, with fewer distractions regarding composition, error handling, and general clutter in the code (e.g. obvious type annotations, or additional syntax elements in C#).

Composing functions (or static methods) is much easier than composing instance methods because they can just be called without the need to instantiate or get an object first. Also, pipes make it easy to orchestrate multiple calls in a very concise and easy-to-read way (left to right, top to bottom; not inside-out or with lots of local variables).

public string GetNameOfCustomer(int id)
{
    return GetName(LoadCustomer(id));
}

public string GetNameOfCustomer(int id)
{
    var customer = GetCustomer(id);
    return GetNameOfCustomer(customer);
}

let getNameOfCustomerFromId id =
    id
    |> loadCustomer
    |> getNameOfCustomer

(Business) error handling bloated our C# code massively, compared to F#, where we can hide the mechanics of error handling elegantly behind computation expressions.

public async Task<(string Name, int Amount)?> 
    GetCustomerNameAndAmount(
        int customerId, int dataId)
{
    var customer = await LoadCustomer(customerId);
    if (customer == null) { return null; }

    var data = await LoadData(dataId);
    if (data == null) { return null; }

    var name = GetNameOfCustomer(customer);
    if (name == null) { return null; }

    return (name, data.Amount);
}

let getCustomerNameAndAmount customerId dataId =
    asyncResult {
        let! customer = loadCustomer customerId
        let! data = loadData dataId
        let! name = 
            customer
            |> getNameOfCustomer
            |> Result.requireSome "customer has no name"
        return name, data.Amount
  }

Although the example is a bit of a stretch, it describes the typical reality in our code very well (for both our C# code and F#). The examples above show why I call programming in F# “happy path coding”.

The F# code feels like looking at the happy path only, while the mechanics of synchronicity and error handling are hidden away in computation expressions like asyncResult.

Conclusions

F# gives us the tools (computation expressions, discriminated unions, lightweight composition) to make the production code so expressive that it is easy to read and understand. It is focused on the business job to get done, not the mechanics of infrastructure concerns. We now prefer to look directly at our production code to understand how a feature works rather than first looking at the test code, as we did when programming in C#.

We still write tests to ensure our code behaves correctly, but we no longer use them as documentation.

Happy coding!