Neural networks, deep learning, NLP · insight-1474

Ensemble neural forecasters over seeds, look-back windows and losses instead of trusting one run

Situation
A neural forecaster's backtest error changes visibly from one random seed to the next.
Action
Train several members varying seed, look-back window and loss, aggregate by median or mean, report both the spread across members and the ensemble error, and stop adding members when the gain is within noise.
Reason
The N-BEATS paper builds a 180-model ensemble varying loss, input window length (2H to 7H) and random initialisation, aggregates by median, and attributes the gain to the diversity this creates.
Boundary
Neural forecasters on tens of series or more, where training is cheap enough to repeat.

Symptom or risk

A single run decides whether it beats the baseline, and picking the best seed after seeing the backtest bakes luck into the claim.

Strict latency or cost limits; very few series where members cannot be validated.

Several times the training and serving cost.

Members that share seed and window; selecting members by test score.

The N-BEATS paper's test-set ensemble has 180 models varying three losses (sMAPE, MASE, MAPE), input window lengths of 2H to 7H, and random initialisations, with the median as ensemble output.
N-BEATS: Neural basis expansion analysis for interpretable time series forecasting