Time series and forecasting · insight-1201
Freeze the event and schedule calendar as it stood at each origin before using it as a feature
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- Data scientist, ML engineer
- Industries
- aviation, logistics, retail
Summary: situation, action, reason, boundary
Situation
A route-demand model uses an event table (strikes, airport works, schedule changes) that operations kept editing after the fact; the backtest reads the current table.
Action
Snapshot the calendar table with an as-of timestamp. For each origin, build features from the version that existed then, and keep only entries announced before the origin. Fixed public holidays can stay.
Reason
A calendar is a series with revisions. Known-in-advance dates such as national holidays are safe; entries created after the event are labels in disguise.
Boundary
Disruption, promotion, maintenance and schedule flags maintained by people rather than computed from the date.
Symptom or risk
Events recorded after they happened, or dates corrected later, give the backtest knowledge no forecaster had, so accuracy on disrupted days is flattered.
Exceptions
Pure date arithmetic (weekday, month, fixed holidays) has no vintage problem.
Trade-offs
Needs audit columns (created_at, effective_from); history before they existed cannot be replayed exactly.
Pitfalls
Joining the live table on date and trusting a created_at column that was overwritten on edit.
Example
Hypothetical: a strike day is flagged retroactively on 12 March; any origin before 12 March must see no flag for that day.