Time series and forecasting · insight-0252
Try a simple average of diverse forecasts before building a model-selection layer
- Evidence type
- empirical finding
- Difficulty
- easy
- Roles
- Data scientist
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
ETS, ARIMA and a gradient-boosting model each win on different series, and the team plans a meta-model to pick a winner per series.
Action
Benchmark an equal-weight average of the candidate forecasts against per-series selection and against the best single model, in the same rolling-origin backtest.
Reason
Combining forecasts often improves accuracy, and simple averages have proven hard to beat by more complex weighting schemes.
Boundary
Portfolios of series with heterogeneous behaviour and several reasonable models.
Symptom or risk
Per-series selection on noisy backtests often picks the luckiest model, and the selection layer adds complexity and instability.
Exceptions
If one model is clearly misspecified for a segment (for example ignores intermittency), exclude it rather than averaging it in.
Trade-offs
Averages are harder to explain per series and need all component models maintained.
Pitfalls
Averaging quantiles from different models as if that produced a calibrated quantile; check coverage.
Lessons and concepts
Claims and their sources
Bates and Granger showed combining forecasts can improve accuracy.
FPP3 notes that a simple average of forecasts has proven hard to beat by more complicated combination methods.