Neural networks, deep learning, NLP · insight-1473
Tune the look-back window of a neural forecaster and check it against a linear model on the same windows
- Evidence type
- empirical finding
- Difficulty
- medium
- Roles
- Data scientist, ML engineer
- Industries
- energy, retail
Summary: situation, action, reason, boundary
Situation
A Transformer forecaster is given a much longer history in the hope of better accuracy.
Action
Backtest several look-back lengths (for example 1, 2, 4 and 8 seasonal cycles) for the neural model and for a linear model on identical windows, and choose the shortest window within noise of the best.
Reason
Zeng et al., Section 5.3, found that adding historical input helped their linear models but not the Transformer-based forecasters, which they read as overfitting to noise.
Boundary
Neural multi-step forecasters on long, seasonal series.
Symptom or risk
Extra history can add noise: on their long-horizon benchmarks Zeng et al. report Transformer forecasters stagnating or getting worse as the window grows while linear models improved.
Exceptions
Findings come from specific long-term benchmarks; your series may differ.
Trade-offs
More backtest runs.
Pitfalls
Changing window and horizon together; comparing models on different training window counts.
Lessons and concepts
Claims and their sources
Zeng et al. (Section 5.3) report that LTSF-Linear models improve substantially with longer look-back windows while Transformer-based forecasters stagnate or get worse, read as overfitting to noise.