Time series and forecasting · insight-0205

Put a gap between training and test windows when features or labels span time

Situation
Features include 14-day rolling sums, or labels mature only after a booking window closes.
Action
Insert a gap at least as long as the longest feature window or label maturation period between each training end and test start, and drop training rows whose labels were not final at the cutoff.
Reason
A buffer removes training samples whose features or targets overlap the test period, so evaluation sees only information that would really exist.
Boundary
Rolling features, overlapping multi-day targets, delayed or revised labels, finance return windows.

Symptom or risk

Training rows just before the cutoff share information with the first test rows, so the backtest leaks and looks better than production.

Pure univariate one-step models with no overlapping windows need no gap.

A gap discards the most recent training data, which can hurt when the series shifts quickly; measure that cost explicitly.

Setting the gap in rows when the data are irregular; define it in time.

In scikit-learn, TimeSeriesSplit(gap=14) removes 14 samples from the end of each training fold.

TimeSeriesSplit's gap parameter excludes a number of samples from the end of each training set before the test set.
sklearn.model_selection.TimeSeriesSplit (scikit-learn API reference)