S20 · code review simulation
A holiday-aware rewrite of a demand baseline and its golden cases
People, data and outputs in this scenario are simulated. Each stage is recorded when you commit; the reveal comes after.
- 5stages
- 40 minabout
- 0stages committed
The brief
The synthetic airline's planning job forecasts passengers for each flight-day with a baseline called 4wk-same-weekday:
the mean of the same flight's passengers on the four previous same weekdays (d-7, d-14, d-21, d-28). It is version v1
and has run for a year.
A teammate proposes v2, described as a "refactor that handles holidays": when a day in the four-week history is a public holiday or an event day, skip it and walk further back until four clean same-weekday values are found. Nothing else changes. The code is in review and you are asked whether it is safe to merge.
You maintain the golden cases for this job (a small, versioned set of frozen inputs with expected behaviour). All data
are the synthetic airline sample (flight_day, 2022-2024); flights are ZX101-ZX119, and the 2024 evaluation covers
6,954 flight-days. Numbers below were recomputed from the sample database.
Stage 1 of 5
The pull request description reports one number, computed over all 6,954 flight-days of 2024 for which both versions produce a forecast:
| Version | WAPE (sum of absolute error / sum of passengers) |
|---|---|
| v1 | 14.62% |
| v2 | 14.67% |
The author writes: "Metrics are flat, so the refactor is safe to merge."
How do you read this evidence?