Time series and forecasting · insight-1203

Split a panel of series by time for every series together, not by holding out whole series

Situation
A global model is trained on 200 routes; the backtest holds out 20 routes entirely and trains on the other 180 across all dates, including the dates being scored.
Action
Choose one cutoff date for the whole panel. Train every series on data before it and score every series after it. If new-series performance matters, hold out series as a separate, additional test with the same time cutoff.
Reason
Series in a panel are not independent: they react to the same calendar and market events. A date seen in training is a date partly known in test.
Boundary
Global or cross-learning forecasting models over routes, stores, SKUs or meters.

Symptom or risk

Training routes share the same holidays, fuel shocks and disruptions as the held-out routes on the same dates, so common shocks leak across series.

Series from unrelated domains and calendars (different countries and years) share little.

A time cutoff leaves fewer training rows for the evaluated period than a random series split.

Grouped K-fold by series is the right tool for tabular entity problems and the wrong one here without the time cutoff.