We return to bi-temporal event sourcing in part twelve of my series on event sourcing. This time, we look at set-and-remove-based bi-temporal events – the second variant of bi-temporal events in our system. You’ll see why they are needed, and what additional problems this kind of event brings to the table.
In earlier posts in this series, I showed you bi-temporal events. We look at an example showing how organisational charts (forms and units) change over time. We call that kind of bi-temporal event stream the lifetime (or create-update-delete-based) kind because a thing is created, updated, and maybe deleted. The events represent the thing’s lifetime. We used two time axes (effective and application) to specify when the event takes effect (e.g., when the organisational chart takes effect) and when the data entered the system (the application). To project the organisational chart, we sorted the events by their effective timestamps to project the timeline. The application timestamp is only relevant when multiple events share the same effective timestamp, so we can, for example, override an update with a newer one.
Set-and-remove-based bi-temporal events
We also have a different kind of timeline: set-and-remove-based.
Examples are:
- The assignment (and removal) of a calendar to an organisational unit from a given date.
- The rules to check when validating workdays (whether a user’s entered data results in a meaningful workday).
- Stating that an employee starts or stops working on a specific project at a given time.
- The settings to use when calculating project activities from the above start/stop information.
These events do not reflect the lifetime of a thing but rather a timeline that specifies which data is relevant at any given time. So, if we need to calculate activities based on start/stop information, we look up the settings for the relevant workday(s).
Override or Insert?
When an event (with a later application) is added to the event stream with an earlier effective timestamp than an existing event, we have a problem. The following picture shows that we first add event A, then B, and finally C. Horizontally, we see the application time axis, so the events are in order of addition: A, B, C. Vertically, we see the effective time axis. And the order of the events, by their effective timestamps, is A, C, B.
When projecting the events into a timeline, we have two options. Either event C overrides event B, resulting in the right projection (override). Or we keep both events so that the value of C is only valid until the effective timestamp of event B, resulting in the left projection (insert).
For example, when a user specifies the rules that need to be run to validate a workday, they specify a set of rules and a date from which that set is effective (= operation 1). Now, what should happen when the user specifies a new set of rules for an earlier effective date (= operation 2)? Should the new set be used for dates after the effective date of operation 1 – operation 2 overrides operation 1 – or should the old rules be effective from the date that operation 1 specified?
Both scenarios can be valid. Therefore, we need to be able to choose the insert/override behaviour when we project events.
Summary
Set-and-remove-based timelines introduce the problem that events can be projected into a timeline in two ways. Either a later-added event with an earlier effective date is inserted or overrides later events.
There is also another problem: how to reverse the effect of an event in such timelines. That is the topic of the next post in this series.
Deep Dive
Let’s take a look at organisational calendars. A unit in an organisational chart can have a calendar in our application. Every member of the organisational unit automatically sees the unit’s events in their calendar. An easy way to define company- or department-wide holidays or parties.
A calendar can be added to an organisational unit per an effective date, and it can also be removed per an effective date:
Then, we define that the added event should be treated as a set event (Sets (effective, value)), and the removed event should be treated as a remove event (Removes effective) when projecting a timeline.
Finally, we need to specify the projection behaviour. First, we need to specify that we want to use a set-and-remove-based projection (ProjectionActionConfiguration = SetRemove). Second, we specify the override behaviour. In this example, we use OverrideBehavior.Override. So, when there are already set events, and we add an earlier (effective) event, the new event overrides all later (effective) events. This happens when a user decides that a calendar should be available earlier than already specified.
