Type-safety across .Net and TypeScript – TypeScript-friendly JSON serialization of F# types

This is part 6 of how we generate types from our .Net backend to be used in our TypeScript client.

1. Why do we even bother?
2. Generating TypeScript constants from .Net constants
3. Finding the types used in communication between the .Net backend and the TypeScript client
4. Generating TypeScript classes from .Net types
5. Generating Angular Services from .Net WebApi Controllers
6. TypeScript-friendly JSON serialization of F# types
7. Testing JSON serialization and deserialization
8. Putting all the parts together

None of the existing JSON libraries can serialize F# unions in a way that is easy to use in TypeScript. Therefore, we changed the library Thoth so that the serialized JSON matches our needs.

Why Thoth?

We used Thoth as a starting point because it is a JSON serialization/deserialization library written in F# for F#. It handles F# records and unions out of the box – just not exactly how we need it so that it is easy to use together with TypeScript.

The problem is how discriminated unions are serialized. Normally, a discriminated union like type DU = A of int | B of string*int is serialized like ["B","hello",17]. An array containing the case as the first element, followed by the values for the case fields. As we have seen in the post about how we generate TypeScript types, we generate a type for DU that looks like this:

export type DU = {
    "A"?: number
    "B"?: [string, number]
};

That means, we need a JSON that looks like this: {"B":["the answer",42]}. An object containing the case as a property and the values in an array.

There are some other things we changed as well. They are explained below when I show all the changes that we made to the code. Please note that Thoth’s code on Github has changed a bit in the meantime.

To achieve our desired behaviour, we copied the source code of Thoth and made the following changes.

Changes to record encoding

In the function autoEncodeRecordsAndUnions we changed the handling of records from original to

if FSharpType.IsRecord(t, allowAccessToPrivateRepresentation=true) then
    let setters =
        FSharpType.GetRecordFields(t, allowAccessToPrivateRepresentation=true)
        |> Array.map (fun fi ->
            let targetKey = Util.Casing.convert caseStrategy fi.Name
            let encoder = autoEncoder extra caseStrategy skipNullField fi.PropertyType
            fun (source: obj) (target: JObject) ->
                let value = FSharpValue.GetRecordField(source, fi)
                if not skipNullField || (skipNullField && not (isNull value)) then // Discard null fields
                    target.[targetKey] <- encoder.Encode value
                target)
    boxEncoder(fun (source: obj) ->
        //--> changed
        let getInitialJObject () =
            let cutGenericType typeName =
                let index = typeName |> FSharpx.String.indexOfChar '`'
                if index > 0 then
                    typeName |> FSharpx.String.substring' 0 index
                else
                    typeName

            let o = JObject()
            o.["$type"] <- JValue.CreateString(cutGenericType t.Name)
            o
        (getInitialJObject (), setters) ||> Seq.fold (fun target set -> set source target) :> JsonValue)
        //--> end changed

Line 13-26: We add a property named $type for every record that holds the name of the type (without generic type parts, if any). We need that for records that implement interfaces to have a discriminator to be used in a switch statement in TypeScript.

A record like type R = { A : int } is serialized as {"$type":"R","a":42}.

It is simpler to do that for all records, not just the ones that actually implement an interface.

Changes to discriminated union encoding

We changed the original to this:

elif FSharpType.IsUnion(t, allowAccessToPrivateRepresentation=true) then
    boxEncoder(fun (value: obj) ->
        //--> changed
        let unionCasesInfo = FSharpType.GetUnionCases(t, allowAccessToPrivateRepresentation = true)
        let info, fields = FSharpValue.GetUnionFields(value, t, allowAccessToPrivateRepresentation=true)

        let isEnumDU =
            unionCasesInfo
            |> Seq.forall (fun x -> x.GetFields().Length = 0)

        if isEnumDU
        then
            string info.Name
        else
            match unionCasesInfo.Length with
            | 1 ->
                match fields.Length with
                | 0 ->
                    string info.Name
                | 1 ->
                    let fieldTypes = info.GetFields()
                    let encoder = autoEncoder extra caseStrategy skipNullField fieldTypes.[0].PropertyType
                    encoder.Encode(fields.[0])
                | len ->
                    let fieldTypes = info.GetFields()
                    let target = Array.zeroCreate<JsonValue> len
                    for i = 0 to len-1 do
                        let encoder = autoEncoder extra caseStrategy skipNullField fieldTypes.[i].PropertyType
                        target.[i] <- encoder.Encode(fields.[i])
                    array target
            | _ ->
                match fields.Length with
                | 0 ->
                    let typeName =
                        info.Name
                    let result = JObject()
                    result.Add(typeName, (bool true))
                    result :> JsonValue
                | length ->
                    let result =
                        let fieldTypeInfos = info.GetFields()
                        if fields.Length = 1
                        then
                            // wert
                            let fieldInfo = fieldTypeInfos.[0]
                            let field = fields.[0]
                            let fieldEncoder = autoEncoder extra caseStrategy skipNullField fieldInfo.PropertyType
                            fieldEncoder.Encode field
                        else
                            // array
                            let result = JArray()
                            for i in 0..(length-1) do
                                let fieldInfo = fieldTypeInfos.[i]
                                let field = fields.[i]
                                let fieldEncoder = autoEncoder extra caseStrategy skipNullField fieldInfo.PropertyType
                                let fieldValue = fieldEncoder.Encode field
                                result.Add(fieldValue)
                            result :> JsonValue

                    let wrapper = JObject()
                    wrapper.Add(info.Name, result)
                    wrapper :> JsonValue)
    //--> end changed

Line 4, 5: We get all cases of this union and their fields.

Line 7-13: We check whether it is a DU that is used as a “simple” enum. If so, we just write the case name to JSON.

Line 15: Depending on the number of cases, we deal differently with this DU. Single case DUs are handled specially. If a case is later added to this DU, then we would serialize it differently. This is, however, not a problem for us because the JSON we generate with this pimped Thoth version is never persisted. For persisted JSON, we use the normal version of Thoth.

Line 18: A single case DU without a field is serialized by writing the case’s name. E.g. type A = A results in A.

Line 20: A single case DU with a single field is serialized by encoding the field’s value. The DU itself is erased. E.g. type A = A of int result in 42.

Line 24: A single case DU with multiple fields is serialized by putting all the serialized fields’ values into an array. E.g. type A = A of int*string results in [42,"hello"].

Line 33: For a value of a multi-case DU without any fields like A in type D = A | B of int, we create an object with a property A of type bool. The bool is irrelevant, but this is a way to match the type that we generate for such a DU: export type D = { A?: bool; B?:int }. We just don’t want to use null or 0 because that could lead to trouble when comparing with == in TypeScript. And it’s a nicer JSON 😉

Line 39: Here we handle a multi case DU and the value has at least one field. If there is only one field then we serialize that, otherwise we serialize all the fields and put them into an array.

Line 60: We put the serialized values for the content of the DU into an object with the case name as a property. If we have for example the following types:

type R = { A : int }
type DU = A of R | B

And we serialize the value let value = DU.A { A = 42 }, we get {"A":{"$type":"R","a":42}}. And that matches our generated TypeScript type.

Changes to record deserialization

We didn’t have to change anything in record deserialization.

Changes to discriminated union deserialization

We changed the original to this:

Changes to elif FSharpType.IsUnion(t, allowAccessToPrivateRepresentation=true) then
    boxDecoder(fun path (value: JsonValue) ->
        //--> added
        let unionCasesInfo = FSharpType.GetUnionCases(t, allowAccessToPrivateRepresentation = true)

        match unionCasesInfo.Length with
        | 1 ->
            let unionCaseInfo = unionCasesInfo.[0]
            let fields = unionCaseInfo.GetFields()

            match fields.Length with
            | 0 ->
                let name = unionCaseInfo.Name
                makeUnion extra caseStrategy t name path [||]
            | 1 ->
                let name = unionCaseInfo.Name
                makeUnion extra caseStrategy t name path [| value |]
            | _ ->
                let jsons = value.AsJEnumerable()

                let values =
                    fields
                    |> Seq.zip jsons
                    |> Seq.map (fun (v, p) ->
                        let decoder = autoDecoder extra caseStrategy false p.PropertyType
                        decoder.Decode (path, v))
                    |> Seq.toList
                    |> FsToolkit.ErrorHandling.List.sequenceResultM

                values
                |> Result.map(fun v ->
                    FSharpValue.MakeUnion(unionCaseInfo, v |> List.toArray, allowAccessToPrivateRepresentation = true))
        //--> added ended (including the following | _ ->
        | _ ->
            if Helpers.isString(value) then
                let name = Helpers.asString value
                makeUnion extra caseStrategy t name path [||]
            elif Helpers.isArray(value) then
                let values = Helpers.asArray value
                let name = Helpers.asString values.[0]
                makeUnion extra caseStrategy t name path values.[1..]
            //--> added
            elif Helpers.isObject(value) then
                let dict = value.Value<JObject>() :> System.Collections.Generic.IDictionary<string, JToken>
                let uciOption = unionCasesInfo |> Array.tryFind (fun uci -> dict.ContainsKey uci.Name)
                match uciOption with
                    | Some uci ->
                        let content = dict.[uci.Name]
                        if Helpers.isBool content then
                            makeUnion extra caseStrategy uci.DeclaringType uci.Name path [||]
                        else if Helpers.isArray content then
                            let contentArray = Helpers.asArray content
                            makeUnion extra caseStrategy uci.DeclaringType uci.Name path contentArray
                        else
                            makeUnion extra caseStrategy uci.DeclaringType uci.Name path [| content |]
                    | None -> ("", BadPrimitive("Type not found in Union Cases", value)) |> Error
            //--> added end
            else (path, BadPrimitive("a string or array", value)) |> Error)record deserialization

We check the same cases as when encoding. Then we inverse the process. I think most is self-explanatory at this stage. Just some quick hints:

Line 19-28: We decode the values from they array and if there is an error we stop.

Line 43: We deserialize the JObject and try to find the corresponding union case. Remember that the bool property is a trick for cases without any fields.

The rest

The rest of Thoth’s code is unchanged. We only put it into a different namespace so that we can use this pimped version for serialization and deserialization when we send data to the client or receive data from the client; and the normal version for all cases when we want to persist JSON. As already written above, the pimped version cannot be used in cases when the JSON is persisted because when a discriminated unions is changed from being single case to multi case, the JSON cannot be deserialized anymore. The normal version of Thoth can handle that because the JSON does not handle single case DUs in a special way.

So now that we can serialize and deserialize F# records and discriminated unions in a way that matches out generated TypeScript types, it is time to show you, how we test that our types are serialized correctly. Stay tuned for the next post.

This blog post is made possible with the support of Time Rocket, the product this blog post is about. Take a look (German only).