Knowledge, Reality and Reconciling the Difference (Part 1)

Values often need to be tracked as they change over time. Imagine we’re building a payroll system. A foundational requirement would be to store an employee’s pay rate. In all but the most basic of systems, saving this rate as a simple value is insufficient (“Joe’s wage = $20/hour”); rather, a history of the rate over time needs to be maintained (“Joe’s wage = $15/hour as of 10/15/13; Joe’s wage = $20/hour as of 2/1/14”). We do this by giving pay rate a time dimension.

What our system knows about a given moment in time and what really was at that same moment can differ. Suppose Joe’s manager signs paperwork on January 20 giving Joe a pay raise but leaves for vacation before turning in the paperwork. At close of business on January 20, the payroll system knows Joe’s rate to be one amount when, in reality, it’s a different amount. Joe has been given a raise but the system doesn’t yet know about it.

The possibility of such differences can make using a single time dimension insufficient. When Joe’s manager returns from vacation on February 1, he finds the duly-signed pay raise form sitting on his desk and sends it on to the payroll department. How should payroll date the raise when entering it into their system? Backdating the singe time dimension to the raise’s effective date (January 20) will create a discrepancy between Joe’s pay rate history (which the backdating will revise) and paycheck history (which will still shows paycheck amounts between January 20 and February 1 computed using the pre-raise pay rate). Dating the pay rate change to when payroll learned of it (February 1) is also unacceptable. Doing so ignores the fact that the raise took effect two weeks prior.

The solution? Simple: ignore the raise altogether. Just kidding! Rather, the pay rate needs to be stored with two dimensions of time: the time the change took effect (actual time or effective time) and the time the system found out about the change (recorded time). After expanding time to two dimensions, the necessary details can be captured without sacrificing clarity or creating confusion:  “Joe’s wage = $20/hour effective 1/20/14; recorded 2/1/14.”

Backdating opens the opportunity to revise the system’s knowledge of the past. If the system took action based on its past knowledge and then that knowledge is revised, remedial action may be required. Joe’s paychecks for the time between his raise and when payroll found out about the raise were less than they should have been. After his raise is entered into the payroll system, rectification is necessary. Perhaps a supplemental paycheck should be issued, the amount owed could be added to the next regular paycheck or a flag could be set alerting a human that manual intervention is needed.

Designing systems that track values as they change over time is often complex. Let’s spend several posts building a system which both works with time in two dimensions and automatically takes corrective action when discrepancies between what was known and what really was are discovered.

Our upcoming code-based exploration will focus on modeling objects for a specific scenario. To prepare for our journey, a broad, conceptual overview of temporal-related design patterns will be helpful. Martin Fowler’s Temporal Patterns is recommended reading in preparation for what is to come.

Reference: Temporal Patterns by Martin Fowler
For additional reading on this topic, see the “Other Readings” listed at the bottom of his article.

2 thoughts on “Knowledge, Reality and Reconciling the Difference (Part 1)

  1. Tom

    Hey Ben – This is an important topic. Management of temporal objects can be tricky. Particularly if times are important. For example, does the effective date mean at the beginning of the day, or the end of the day?

    Since one of the purposes of tracking both the effective date and the recorded date could be to create a supplemental payment, you are going to have to know what the previous rate too. In that case, it might be more important to record the effective end date of the previous rate, with the assumption the current rate to affect the same day.

    Each of the rates would be a separate record. If following rails standards, you would typically have a created_at and updated_at on each record, which would serve as your recorded date.

    1. Ben Gribaudo Post author

      Thanks for these comments, Tom! I tend to think that an effective timestamp should apply starting at the beginning of the time period specified. So, an effective date of December 15 would represent something taking effect starting at the beginning of December 15 and an effective time of 13:15:23 UTC would mean the associated data applies beginning at 13:15:23.0000000 UTC.


Leave a Reply

Your email address will not be published. Required fields are marked *