Time series and forecasting · insight-0206
Report error by horizon and evaluate at the horizon the decision uses
- Evidence type
- documented fact
- Difficulty
- easy
- Roles
- Data scientist, Data analyst
- Industries
- aviation, retail, logistics
Summary: situation, action, reason, boundary
Situation
A dashboard shows one pooled error for a 1-to-90-day forecast that feeds pricing decisions made 21 days before departure.
Action
Compute error separately for each horizon, or for decision-relevant horizon buckets, and compare against a baseline at each one. Select models on the horizons that change actions.
Reason
Forecast error usually grows with horizon, and model rankings can differ between short and long horizons.
Boundary
Revenue management, inventory replenishment with lead times, staffing plans.
Symptom or risk
Pooled error is dominated by easy short horizons and hides that the 21-day forecast, the one that drives the decision, may be no better than seasonal naive.
Exceptions
If every horizon drives an equally costly decision, a pooled metric with explicit horizon weights is acceptable.
Trade-offs
Per-horizon tables are noisier; group horizons into buckets when data are thin.
Pitfalls
Comparing a model's 1-step error with a colleague's 28-step error.
Lessons and concepts
Claims and their sources
The FPP3 time-series cross-validation example shows forecast RMSE increasing as the forecast horizon increases.