Time series and forecasting · insight-1254
Feed a statistical model's out-of-sample forecast to the ML model as a feature, rebuilt at every origin
- Evidence type
- practitioner heuristic
- Difficulty
- hard
- Roles
- Data scientist, ML engineer
- Industries
- retail, energy, e-commerce
Summary: situation, action, reason, boundary
Situation
A boosted-tree forecaster is worse than ETS at trend and seasonal shape but better at promotions, and the team wants both strengths.
Action
At every training origin, fit the statistical model on data before that origin, store its forecast for the target time, and use it, or the ratio of the target to it, as a feature or fit the ML model to the residual. Evaluate in nested rolling origins and compare with a simple average of the two.
Reason
A statistical forecast supplies structure and the ML model supplies covariate effects, but the feature is valid only if it is computed from information available at each origin.
Boundary
Retail, demand and energy series with strong trend or seasonal structure and many covariates.
Symptom or risk
Using the statistical model's fitted values from one fit on the full history leaks the future into features, and the stacked model looks excellent in the backtest and weak in production.
Exceptions
If a simple average of the two models already matches the stack, the extra pipeline is not worth it.
Trade-offs
A two-stage pipeline doubles the fitting cost, and errors in the first stage propagate into the second.
Pitfalls
Using in-sample fitted values of the statistical model as the feature.