Unquote is a great library for writing test assertions in F#. In our acceptance tests, we frequently check data structures for equality. When these data structures get bigger, reading test failure messages gets harder. That’s why I implemented an F# data structure differ around Unquote that lets me quickly see the differences.
Update: Find a newer version here.
The following example shows a simple test sing Unquote and its output:
[<Fact>]
let ``simple example`` () =
let a = [ 1 ; 2; 3 ]
let b = [ 1 ; 3; 4 ]
test <@ a = b @>[1; 2; 3] = [1; 3; 4]
falseAlternatively, a simple comparison can be written as:
[<Fact>]
let ``simple example`` () =
let a = [ 1 ; 2; 3 ]
let b = [ 1 ; 3; 4 ]
a =! bThe real power of Unqote shows when using quoted expressions:
[<Fact>]
let ``expression example`` () =
let a = [ 1 ; 2; 3 ; 4 ; 5]
let b = [ 2 ; 4 ; 6 ]
test <@ (a |> List.filter (fun x -> x % 2 = 0) = b) @>The error message contains the individual steps to get to the result:
a |> List.filter (fun x -> x % 2 = 0) = b
[1; 2; 3; 4; 5] |> List.filter (fun x -> x % 2 = 0) = [2; 4; 6]
[2; 4] = [2; 4; 6]
falseThis helps us a lot when we do some transformations on the data that we want to check because we can see the original data before and during the transformations. For example, when we get a Result from a function and we want to do a Result.map on the data before comparing it. If the test fails, we see the original data.
The problem with bigger data structures
Some of our tests work on rather big data structures like trees or nested lists of records containing lists. With such data structures, it quickly gets tricky to spot what actually is different.
Look at this example:
type Leaf = { Name : string }
and Node = { Name : string ; Children : Tree list }
and Tree =
| Leaf of Leaf
| Node of Node
[<Fact>]
let ``tree example`` () =
let a =
Node {
Name = "a"
Children =
[
Leaf { Name = "leaf1" }
Node { Name = "b"
Children =
[
Leaf { Name = "leaf2" }
]
}
Leaf { Name = "leaf3" }
]
}
let b =
Node {
Name = "a"
Children =
[
Leaf { Name = "leaf1" }
Node { Name = "c"
Children =
[
Leaf { Name = "leaf2" }
]
}
Leaf { Name = "leaf3" }
]
}
a =! bThe result of running the above test:
Node
{ Name = "a"
Children =
[Leaf { Name = "leaf1" };
Node { Name = "b"
Children = [Leaf { Name = "leaf2" }] };
Leaf { Name = "leaf3" }] } = Node
{ Name = "a"
Children =
[Leaf { Name = "leaf1" };
Node { Name = "c"
Children = [Leaf { Name = "leaf2" }] };
Leaf { Name = "leaf3" }] }
falseI kept the sample short for the sake of a blog post, but I hope you can see why it gets difficult to see subtle differences quickly.
Let’s pimp it!
Let’s change the test to this (only a = is added so that we use the ==! operator instead of =!):
[<Fact>]
let ``pimped tree example`` () =
let a =
Node {
Name = "a"
Children =
[
Leaf { Name = "leaf1" }
Node { Name = "b"
Children =
[
Leaf { Name = "leaf2" }
]
}
Leaf { Name = "leaf3" }
]
}
let b =
Node {
Name = "a"
Children =
[
Leaf { Name = "leaf1" }
Node { Name = "c"
Children =
[
Leaf { Name = "leaf2" }
]
}
Leaf { Name = "leaf3" }
]
}
a ==! bWhen we run this test, the output looks as follows:
Delta:
Node/Children[1]Node/Name b = c
Unquote Message:
Node
{ Name = "a"
Children =
[Leaf { Name = "leaf1" };
Node { Name = "b"
Children = [Leaf { Name = "leaf2" }] };
Leaf { Name = "leaf3" }] } = Node
{ Name = "a"
Children =
[Leaf { Name = "leaf1" };
Node { Name = "c"
Children = [Leaf { Name = "leaf2" }] };
Leaf { Name = "leaf3" }] }
falseThe difference is easy to spot because the differences are listed directly at the top of the error message – followed by the Unquote error message. Woohoo!
Implementing the differ
First, we need a custom operator:
[<AutoOpen>]
module UnquoteOperators
let inline (==!) a b = UnquoteExtensions.diff a bThe custom operator simply delegates to the diff function:
module UnquoteExtensions
open Microsoft.FSharp.Reflection
open Xunit.Sdk
open Swensen.Unquote
open FSharpx
type ListIterator =
static member GetValues<'a>(value : obj) =
let casted : 'a list = downcast value
casted |> List.toArray |> Array.map (fun v -> v :> obj)
let rec getDifferences (a : obj) (b : obj) breadcrumb =
let t = a.GetType()
if a = b then
""
elif (t.IsGenericType && t.GetGenericTypeDefinition() = typedefof<FSharp.Collections.list<_>>) then
let method = typeof<ListIterator>.GetMethod("GetValues").MakeGenericMethod(t.GetGenericArguments() |> Array.exactlyOne)
let a_values : obj[] = downcast method.Invoke(null, [| a |])
let b_values : obj[] = downcast method.Invoke(null, [| b |])
if a_values.Length <> b_values.Length then
let toString = Array.map (fun v -> $"{v}") >> String.concat ";"
$"{breadcrumb} [{a_values |> toString }] = {b_values |> toString}"
else
Array.zip a_values b_values
|> Array.mapi (fun i (x,y) ->
if (x <> y) then
getDifferences x y (breadcrumb + $"[{i}]")
else
"")
|> Array.filter (fun x -> x.Length > 0)
|> String.concat "\n"
elif (FSharpType.IsUnion t) then
if (FSharpType.GetUnionCases(t).Length = 1) then
$"{breadcrumb} {a} = {b}"
else
let a_case = FSharpValue.GetUnionFields(a, t)
let b_case = FSharpValue.GetUnionFields(b, t)
if (a_case = b_case) then
""
else
let aa = a_case |> fst
let bb = b_case |> fst
if (aa = bb) then
Array.zip (snd a_case) (snd b_case)
|> Array.map (fun (x,y) ->
getDifferences x y $"{breadcrumb}{aa.Name}")
|> Array.filter (fun x -> x.Length > 0)
|> String.concat ", "
else
$"{breadcrumb} {aa} = {bb}"
elif t = typeof<string> then
$"{breadcrumb} {a} = {b}"
else
let properties = t.GetProperties()
if (properties.Length = 0) then
if a <> b then $"{breadcrumb} {a} = {b}" else ""
else
let allValues =
properties
|> Array.map (fun p -> p.Name, p.GetValue(a), p.GetValue(b))
let mismatches =
allValues
|> Array.filter (fun (_, pa,pb) -> not(pa = pb))
mismatches
|> Array.map (fun (name, pa,pb) -> getDifferences pa pb (breadcrumb + $"/{name}"))
|> String.concat "\n"
let diff a b =
try
a =! b
with
| :? TrueException as e ->
let message = getDifferences a b ""
raise (TrueException("Delta:\n" + message + "\n-----------------------------------------------\nUnquote Message:\n" + e.Message, false))The diff function first uses normal Unquote (line 80). When Unquote throws an exception, we catch it and execute getDifferences (line 83).
getDifferences does some reflection magic to walk through the value and compare field per field, list item per list item and so on.
Please note that the implementation above is not complete – it currently just covers what we need in our code. But I hope you get the idea to extend it for your code if needed.
The breadcrumb is used to keep track of where we are in the structure and to provide an error message that points to the difference.
That’s all for this time – happy coding!
