Practical insights
- 1000published
- 25in review
- 8topic families
Results
- Use many rolling origins that span the seasons you care about, not one final holdout
A forecast model is chosen from a single holdout of the last eight weeks before peak season.
easydocumented fact - Fit normalisation statistics at each cutoff, never on the full time series
A notebook standardises the whole series with its mean and standard deviation, then runs a rolling-origin backtest.
easydocumented fact - In direct models, keep every lag at least as long as the horizon it predicts
A direct model for 14-days-ahead demand uses lag 7 sales as a feature.
easypractitioner heuristic - Shift before rolling so window features exclude the target period
A 7-day rolling mean of sales is computed and joined to the same row as the target.
easydocumented fact - End each backtest origin on a complete aggregation period
Daily bookings are rolled up to weeks, and the cutoff falls on a Wednesday; the training week's total contains three days and the test week begins mid-week.
easypractitioner heuristic - Build the naive benchmark at the production horizon from the origin value, not from the previous day
A 28-day-ahead model is compared with a naive baseline computed as yesterday's actual for every target day.
easydocumented fact - Report how many series the new model beats the benchmark on, next to the average improvement
A candidate model improves average MASE over 2,000 series by 4% and the team proposes a full cutover.
easypractitioner heuristic - Forecast a new route or product with a global model on its attributes instead of waiting for local history
A new route launches with three weeks of data, so a local seasonal model has nothing to estimate seasonal indices from.
easypractitioner heuristic - Give a boosted tree the same-weekday and same-season lags explicitly
A daily boosted model uses lags 1 to 7 and a rolling 7-day mean, and its Monday forecasts keep missing the Monday level after a holiday week.
easypractitioner heuristic