Today’s random F# code: Using Verify to prevent breaking changes in stored data

We use event sourcing in many sub-systems of our application to store data. We serialize the data of the events as JSON and store the JSON in the database. To ensure that we don’t change the structure of these event data and by this introduce backward compatibility issues, we use Verify to snapshot test the event data.

Snapshot testing

From the Verify website:

Verify is a snapshot tool that simplifies the assertion of complex data models and documents.

Verify is called on the test result during the assertion phase. It serializes that result and stores it in a file that matches the test name. On the next test execution, the result is again serialized and compared to the existing file. The test will fail if the two snapshots do not match: either the change is unexpected, or the reference snapshot needs to be updated to the new result.

Wrapper functions around Verify

First, we need some simple wrapper functions to make Verify easier to use from F#:

verify is for normal tests, verifyUsingParameters is for tests with multiple test cases (Theory in xUnit, for example).

Verify Settings

Second, we need some Verify settings (in our case, the default settings can’t be used):

let verifySettings =
    let settings = VerifySettings ()
    settings.UseDirectory "EventData"
    settings.ModifySerialization (fun s ->
        s.DontScrubGuids ()
        s.DontScrubDateTimes ())
    settings

The test

And finally, we need a test:

The test uses MemberData to get all the event data we need to check. The key is a string representation to identify the specific event data that will be written into the file, which is generated by Verify. instance is an instance of the specific event data that we want to check.

First, we serialize the instance. We need a little helper so that the correct type is used for serialisation because this test uses Thoth for serialisation, and Thoth uses the static type, not the dynamic type:

type DataEncoder =
    static member Encode<'a>(value : obj) =
        let casted : 'a = downcast value
        JsonConversion.Database.encodePretty casted

Then, we check the serialized instance against what we approved earlier.

If we accidentally change the event data of an event, the test will alert us!

Getting the test data

The last mission piece is to get the test data. Not the nicest code, but it gets the job done 🙂

We use some reflection to find all events in our codebase – they all end with Event and contain a field named Data. Data is always a discriminated union in our code, so we get all their cases and create sample values for them. Finally, we return a descriptive key and the example value. The example generator generates always the same values for any type. Otherwise, we couldn’t execute the snapshot test on them because the test would always fail due to different sample values.

Conclusions

Verify is a great tool to prevent changes in our data structures. We also use it to verify all data we send over our API.

Additionally, Verify provides a plugin to diff actual and verify values inside our IDE easily and a Windows tray app to quickly verify lots of files when, for example, running the tests for the first time.