Time series and forecasting · insight-0228
Decide explicitly how to treat a regime-change period in training and evaluation
- Evidence type
- practitioner heuristic
- Difficulty
- hard
- Roles
- Data scientist, ML engineer
- Industries
- aviation, finance, retail
Summary: situation, action, reason, boundary
Situation
Two years of history include an abrupt collapse and slow recovery in travel demand.
Action
Try three treatments in a backtest: exclude the period from training, include it with regime indicators, or down-weight it. Report accuracy on normal and stressed origins separately, and record the choice in the model's decision log.
Reason
Regime changes alter level, seasonality and dynamics at once, so no single default treatment is safe.
Boundary
Pandemics, sanctions, currency crises, regulatory shocks, platform migrations.
Symptom or risk
Including the period untreated distorts trend and seasonality; silently deleting it hides how the model behaves under stress.
Exceptions
If the regime may recur, keep it visible in training so intervals reflect that risk.
Trade-offs
Excluding data improves normal-period accuracy but leaves the model unprepared for the next shock.
Pitfalls
Comparing a model trained with exclusions against one without on different evaluation windows.