Practical insights
- 1000published
- 25in review
- 8topic families
Results
- Drop or forecast any regressor that is not known at the forecast origin for the full horizon
A demand model uses yesterday's competitor fare as a feature, and the business needs forecasts 1 to 28 days ahead.
mediumdocumented fact - Report ex-ante accuracy, not ex-post, when a model uses future-unknown regressors
A team reports that adding temperature cut daily electricity-load forecast error by a third.
mediumdocumented fact - Use many rolling origins that span the seasons you care about, not one final holdout
A forecast model is chosen from a single holdout of the last eight weeks before peak season.
easydocumented fact - Put a gap between training and test windows when features or labels span time
Features include 14-day rolling sums, or labels mature only after a booking window closes.
mediumdocumented fact - Choose expanding or sliding training windows by backtest, not by habit
A logistics volume series changed behaviour after a network redesign two years ago, and the model trains on all history.
mediumpractitioner heuristic - Check exchangeability before trusting split-conformal coverage on a time series
Split conformal intervals are calibrated on last year's residuals and promised to give 90% coverage.
harddocumented fact - Track realised interval coverage online and adjust the miscoverage level when it drifts
A deployed 90% interval for hourly call volumes covered 90% in backtests but 78% over the last month.
harddocumented fact - Score a quantile forecast with pinball loss at that quantile
A team forecasts the 0.9 quantile of demand and reports its MAPE against actuals.
mediumdocumented fact - Decide explicitly how to treat a regime-change period in training and evaluation
Two years of history include an abrupt collapse and slow recovery in travel demand.
hardpractitioner heuristic - Detrend or difference the target before using tree models on trending series
A gradient-boosted forecast of a growing subscription base flattens at the highest level seen in training.
mediumdocumented fact - Test one global model across many related series before maintaining thousands of local ones
A team fits and monitors a separate ARIMA for each of 4,000 store-SKU series.
mediumempirical finding - Scale each series by its own recent level when training a global model
A global model across routes of 50 to 5,000 weekly passengers is accurate on big routes and poor on small ones.
mediumpractitioner heuristic - Fit normalisation statistics at each cutoff, never on the full time series
A notebook standardises the whole series with its mean and standard deviation, then runs a rolling-origin backtest.
easydocumented fact - Choose recursive, direct or multi-output strategies by backtest at your horizons
A one-step-ahead ML model is fed its own predictions to produce a 28-day forecast.
mediumempirical finding - Backtest recursive forecasts with their own predicted lags, as production will run
A recursive LightGBM forecast reports similar error at day 1 and day 14.
mediumpractitioner heuristic - In direct models, keep every lag at least as long as the horizon it predicts
A direct model for 14-days-ahead demand uses lag 7 sales as a feature.
easypractitioner heuristic - Shift before rolling so window features exclude the target period
A 7-day rolling mean of sales is computed and joined to the same row as the target.
easydocumented fact - Treat public-benchmark scores of forecasting foundation models as possibly contaminated
A time-series foundation model is proposed because it tops a public leaderboard.
mediumdocumented fact - Make a foundation or deep forecaster beat seasonal naive and ETS on your backtest first
A team plans to replace its forecasting stack with a transformer-based model after a promising demo.
mediumempirical finding - Store first-reported and final actuals so forecast accuracy uses the right version
Revenue figures are updated days after the flight date, and accuracy reports are rerun against the latest table.
mediumpractitioner heuristic - Freeze the event and schedule calendar as it stood at each origin before using it as a feature
A route-demand model uses an event table (strikes, airport works, schedule changes) that operations kept editing after the fact; the backtest reads the current table.
mediumpractitioner heuristic - Fit smoothers, interpolation and seasonal adjustment inside each backtest origin, because they look forward
A series is cleaned once with STL seasonal adjustment, centred moving averages and linear interpolation of gaps, and then split into train and test.
mediumdocumented fact - Split a panel of series by time for every series together, not by holding out whole series
A global model is trained on 200 routes; the backtest holds out 20 routes entirely and trains on the other 180 across all dates, including the dates being scored.
mediumpractitioner heuristic - End each backtest origin on a complete aggregation period
Daily bookings are rolled up to weeks, and the cutoff falls on a Wednesday; the training week's total contains three days and the test week begins mid-week.
easypractitioner heuristic - Build the naive benchmark at the production horizon from the origin value, not from the previous day
A 28-day-ahead model is compared with a naive baseline computed as yesterday's actual for every target day.
easydocumented fact - Calibrate intervals separately at each horizon, since one-step residual spread is too narrow far ahead
A residual-based 80% interval built from one-step errors is reused for horizons 1 to 28.
mediumpractitioner heuristic - Check and fix quantile crossing before scoring or serving multi-quantile forecasts
A gradient-boosted model fits the 10th, 50th and 90th percentiles with separate runs and the dashboard sometimes shows the 90th below the median.
mediumdocumented fact - Add non-negativity to reconciliation when the series are counts or sales
Route forecasts from a model are all non-negative, but after MinT reconciliation a thin route shows -3 passengers on some days.
mediumempirical finding - Monitor forecast error as a ratio to a frozen benchmark, not as an absolute number
A weekly dashboard alerts when MAPE exceeds 12%, and it fires every holiday season and stays quiet through real model decay in calm months.
mediumpractitioner heuristic - Log the action taken on each forecast so that monitored error is not contaminated by your own response
A capacity forecast triggers extra flights on routes forecast high, and the monitored error on those routes keeps improving.
hardpractitioner heuristic - Report how many series the new model beats the benchmark on, next to the average improvement
A candidate model improves average MASE over 2,000 series by 4% and the team proposes a full cutover.
easypractitioner heuristic - Decide the ARIMA constant deliberately, because with differencing it sets the long-run trend
An automatic ARIMA run on quarterly fuel-cost data returns d = 1 with a non-zero constant, and the five-year forecast climbs in a straight line.
harddocumented fact - Train on the driver forecasts that existed at each origin, not on the values that were later observed
A day-ahead load model is trained on observed temperature, but in production it receives a weather forecast whose error grows with lead time.
hardpractitioner heuristic - In a direct model, align future-known drivers with the target date and history features with the origin
A per-horizon boosted model for h = 1 to 14 days is built by shifting every column, including the promotion calendar, by h days.
mediumpractitioner heuristic - Pick the MinT covariance estimator by the number of series relative to the history length
A retail group reconciles forecasts for 4,000 item-store series with 104 weeks of history, using MinT with the sample covariance of one-step forecast errors.
harddocumented fact - Forecast a new route or product with a global model on its attributes instead of waiting for local history
A new route launches with three weeks of data, so a local seasonal model has nothing to estimate seasonal indices from.
easypractitioner heuristic - Weight a global model's loss by the weights of the metric that will judge it
A global boosted model for 30,000 SKU series is trained with unweighted squared error, but it will be judged on an error that is weighted by recent revenue.
hardpractitioner heuristic - Give a boosted tree the same-weekday and same-season lags explicitly
A daily boosted model uses lags 1 to 7 and a rolling 7-day mean, and its Monday forecasts keep missing the Monday level after a holiday week.
easypractitioner heuristic - Route intermittent series to a method by interval and size variability, then confirm by backtest
A spare-parts catalogue of 20,000 items is forecast with a single method, although some items sell steadily and others sell every few months in varying quantities.
mediumempirical finding - Use a probability-updating method such as TSB when items can become obsolete
A wholesaler's slow items sometimes stop selling for good, yet a Croston forecast stays at the old demand rate for months because it only updates when a sale occurs.
harddocumented fact - Check that a zero-shot foundation forecaster can use your future-known drivers before expecting promotion peaks
A team evaluates an off-the-shelf pretrained forecaster on weekly sales with a known promotion calendar and finds that it misses every promotion peak.
mediumdocumented fact - Match a foundation model's context window to your longest seasonal cycle
Hourly electricity demand has weekly and annual cycles, and a pretrained forecaster gets the last 512 hours as context.
mediumempirical finding - Handle sparse and flat series before sending them to a mean-scaled, tokenizing forecaster
A pretrained model that scales each series by the mean absolute value of its context is run on spare-part demand where one week in twenty has a sale of one unit.
harddocumented fact - Feed a statistical model's out-of-sample forecast to the ML model as a feature, rebuilt at every origin
A boosted-tree forecaster is worse than ETS at trend and seasonal shape but better at promotions, and the team wants both strengths.
hardpractitioner heuristic - Bag an ETS model with bootstrapped STL remainders when single fits are unstable
Monthly ETS forecasts for 400 series change noticeably each time the models are refitted, though the history changed by one point.
mediumempirical finding