Time series and forecasting · insight-0239
Backtest recursive forecasts with their own predicted lags, as production will run
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- ML engineer, Data scientist
- Industries
- retail, energy
Summary: situation, action, reason, boundary
Situation
A recursive LightGBM forecast reports similar error at day 1 and day 14.
Action
In each backtest origin, generate the full horizon by feeding predictions back into lag and rolling features exactly as the serving code does, ideally by calling the same function.
Reason
Error accumulation is the main weakness of recursive forecasting, and only a faithful simulation reveals it.
Boundary
Recursive ML and neural forecasting pipelines.
Symptom or risk
The backtest filled lag features with actual values at every step, so it measured a stack of one-step forecasts, not the multi-step forecast deployed.
Exceptions
Direct models with lags at least as long as the horizon do not feed predictions back.
Trade-offs
Faithful simulation is slower; vectorise across series rather than shortening the test.
Pitfalls
Separate training and serving feature code, which drifts apart over time.