Practical insights
- 1000published
- 25in review
- 8topic families
Results
- Drop or forecast any regressor that is not known at the forecast origin for the full horizon
A demand model uses yesterday's competitor fare as a feature, and the business needs forecasts 1 to 28 days ahead.
mediumdocumented fact - Report ex-ante accuracy, not ex-post, when a model uses future-unknown regressors
A team reports that adding temperature cut daily electricity-load forecast error by a third.
mediumdocumented fact - Widen or label scenario forecasts because they exclude uncertainty in the assumed drivers
Finance asks for next quarter's demand under three fuel-price scenarios, each with an 80% interval.
mediumdocumented fact - Use many rolling origins that span the seasons you care about, not one final holdout
A forecast model is chosen from a single holdout of the last eight weeks before peak season.
easydocumented fact - Put a gap between training and test windows when features or labels span time
Features include 14-day rolling sums, or labels mature only after a booking window closes.
mediumdocumented fact - Report error by horizon and evaluate at the horizon the decision uses
A dashboard shows one pooled error for a 1-to-90-day forecast that feeds pricing decisions made 21 days before departure.
easydocumented fact - Choose expanding or sliding training windows by backtest, not by habit
A logistics volume series changed behaviour after a network redesign two years ago, and the model trains on all history.
mediumpractitioner heuristic - Guard MASE against zero denominators and state which naive scaling you used
MASE is computed across thousands of SKUs and some values come back infinite or enormous.
mediumdocumented fact - Compute WAPE at the level where actuals cannot sum to zero
WAPE is reported for every store-SKU-week cell and many cells are blank or infinite.
easypractitioner heuristic - Do not average MAPE across series of very different volumes
A portfolio report averages per-series MAPE, and a few slow movers make the whole forecast look broken.
easydocumented fact - Treat intermittent demand forecasts as rates and use bias-corrected Croston-type methods
A spare-parts series is zero most weeks with occasional orders of 3 to 20 units, and exponential smoothing chases each spike.
mediumdocumented fact - Do not choose intermittent-demand models by MAE because it rewards forecasting zero
On a mostly-zero series, an all-zero forecast wins the MAE comparison against every model.
harddocumented fact - Unconstrain sold-out flights before training a demand forecast
A route forecast is trained on passengers flown, and many peak departures hit capacity.
harddocumented fact - Separate stockout zeros from true zero demand before modelling a retail series
A store-SKU series looks intermittent, and stock records show the item was frequently unavailable.
mediumpractitioner heuristic - Reconcile hierarchical forecasts so route, region and network numbers agree
Route-level and network-level demand forecasts are produced separately, and the routes do not add up to the network total.
mediumdocumented fact - Match the forecasting level of a hierarchy to the noise and the decision
Store-SKU daily sales are too noisy to forecast well, but replenishment decisions are made per store and SKU.
mediumpractitioner heuristic - Check exchangeability before trusting split-conformal coverage on a time series
Split conformal intervals are calibrated on last year's residuals and promised to give 90% coverage.
harddocumented fact - Track realised interval coverage online and adjust the miscoverage level when it drifts
A deployed 90% interval for hourly call volumes covered 90% in backtests but 78% over the last month.
harddocumented fact - Evaluate prediction intervals on coverage and width together
Two interval methods are compared, and the team picks the one with the highest coverage.
mediumdocumented fact - Check interval coverage by season and segment, not only overall
Overall coverage of a 90% demand interval is 89%, and the team declares it calibrated.
mediumpractitioner heuristic - Pick the forecast quantile from the ratio of under- and over-forecast costs
Operations asks for 'the forecast' to set meal loads per flight, where running out costs more than waste.
mediumdocumented fact - Score a quantile forecast with pinball loss at that quantile
A team forecasts the 0.9 quantile of demand and reports its MAPE against actuals.
mediumdocumented fact - Model each seasonal period explicitly in hourly or daily data
Hourly electricity load shows daily, weekly and annual cycles, and a model with one seasonal period leaves strong patterns in the residuals.
mediumdocumented fact - Handle weekly data's non-integer annual season with Fourier terms or STL
A weekly demand series is fitted with a 52-period seasonal model, and the holiday peaks drift by a week every few years.
mediumdocumented fact - Encode moving holidays as windows before and after, not as a single day flag
A Turkish domestic route forecast misses the travel surge around Eid holidays, which move about eleven days earlier each Gregorian year.
mediumdocumented fact - Model one-off events as interventions and keep them out of learned seasonality
A city hosted a single major sports final, and hotel and flight demand spiked for one week.
easydocumented fact - Represent a known level shift with a step regressor or a post-break training window
A competitor added about 25% more seats on three routes from a known date, and the route forecasts now run high.
mediumdocumented fact - Decide explicitly how to treat a regime-change period in training and evaluation
Two years of history include an abrupt collapse and slow recovery in travel demand.
hardpractitioner heuristic - Bias-adjust back-transformed forecasts when totals or means matter
Daily demand is modelled on a log or Box-Cox scale, and the back-transformed daily forecasts are summed into a monthly budget.
mediumdocumented fact - Enforce positivity with a transformation, not by clipping negative forecasts to zero
A linear or ML forecast of low-volume sales produces negative values, and the pipeline replaces them with zero.
mediumdocumented fact - Difference only as much as a unit-root test and backtest support
A model is fitted on second differences because the first-differenced plot 'still looked wavy'.
mediumdocumented fact - Read ACF and PACF as hints, then choose orders by information criteria and residual checks
An analyst picks ARIMA(2,1,1) because the PACF 'cuts off at lag 2'.
mediumpractitioner heuristic - Use bookings already on hand as a known-at-origin input for departure-level forecasts
A revenue-management team forecasts final bookings for a departure 21 days out using only past departures.
mediumempirical finding - Detrend or difference the target before using tree models on trending series
A gradient-boosted forecast of a growing subscription base flattens at the highest level seen in training.
mediumdocumented fact - Test one global model across many related series before maintaining thousands of local ones
A team fits and monitors a separate ARIMA for each of 4,000 store-SKU series.
mediumempirical finding - Scale each series by its own recent level when training a global model
A global model across routes of 50 to 5,000 weekly passengers is accurate on big routes and poor on small ones.
mediumpractitioner heuristic - Fit normalisation statistics at each cutoff, never on the full time series
A notebook standardises the whole series with its mean and standard deviation, then runs a rolling-origin backtest.
easydocumented fact - Choose recursive, direct or multi-output strategies by backtest at your horizons
A one-step-ahead ML model is fed its own predictions to produce a 28-day forecast.
mediumempirical finding - Backtest recursive forecasts with their own predicted lags, as production will run
A recursive LightGBM forecast reports similar error at day 1 and day 14.
mediumpractitioner heuristic - In direct models, keep every lag at least as long as the horizon it predicts
A direct model for 14-days-ahead demand uses lag 7 sales as a feature.
easypractitioner heuristic - Shift before rolling so window features exclude the target period
A 7-day rolling mean of sales is computed and joined to the same row as the target.
easydocumented fact - Treat public-benchmark scores of forecasting foundation models as possibly contaminated
A time-series foundation model is proposed because it tops a public leaderboard.
mediumdocumented fact - Make a foundation or deep forecaster beat seasonal naive and ETS on your backtest first
A team plans to replace its forecasting stack with a transformer-based model after a promising demo.
mediumempirical finding - Prefer simple models on very short series and say how little the backtest can tell you
A new route has 30 weeks of history, and someone proposes a seasonal ARIMA with many parameters.
easydocumented fact - Measure forecast value added at every step, including human overrides
Planners adjust the statistical forecast every month, and nobody knows whether the adjustments help.
easypractitioner heuristic - Track forecast bias separately from accuracy
A forecast with acceptable MAE consistently runs below actual demand on high-season weeks.
easypractitioner heuristic - Do not answer a price what-if by changing the price input of a demand forecast
A manager asks what bookings would be if fares were cut 5%, and the analyst re-scores the forecasting model with lower fares.
harddocumented fact - Model spill and unconstrained demand before estimating gains from added capacity
A planner asks for passenger gains if the aircraft on a route changes from 160 to 180 seats.
hardpractitioner heuristic - Evaluate an intervention against a counterfactual forecast with control series
A loyalty campaign launched on one market, and leadership wants to know whether bookings rose because of it.
harddocumented fact - Set low expectations and strict out-of-sample tests for financial return forecasts
A model with a strong in-sample fit is proposed to forecast monthly equity returns for allocation decisions.
mediumempirical finding - Count every strategy variant you tried before trusting the best backtest
A trading or pricing signal looks strong after trying many lookback windows, thresholds and universes on the same history.
harddocumented fact - Try a simple average of diverse forecasts before building a model-selection layer
ETS, ARIMA and a gradient-boosting model each win on different series, and the team plans a meta-model to pick a winner per series.
easyempirical finding - Build an explicit time grid so missing periods are not mistaken for zeros or skipped
A route's daily table has a nine-day gap where no rows exist, and the lag-7 feature is computed with row offsets.
easypractitioner heuristic - Use count-appropriate intervals for small integer series
Weekly claims at a small branch average three, and a normal-error model produces intervals from minus two to eight.
mediumpractitioner heuristic - Store first-reported and final actuals so forecast accuracy uses the right version
Revenue figures are updated days after the flight date, and accuracy reports are rerun against the latest table.
mediumpractitioner heuristic - Freeze the event and schedule calendar as it stood at each origin before using it as a feature
A route-demand model uses an event table (strikes, airport works, schedule changes) that operations kept editing after the fact; the backtest reads the current table.
mediumpractitioner heuristic - Fit smoothers, interpolation and seasonal adjustment inside each backtest origin, because they look forward
A series is cleaned once with STL seasonal adjustment, centred moving averages and linear interpolation of gaps, and then split into train and test.
mediumdocumented fact - Split a panel of series by time for every series together, not by holding out whole series
A global model is trained on 200 routes; the backtest holds out 20 routes entirely and trains on the other 180 across all dates, including the dates being scored.
mediumpractitioner heuristic - Estimate reconciliation weights and aggregate features only from data before each origin
A hierarchical forecast reconciles with MinT; its error covariance is estimated once from residuals over the whole history and then used in every backtest origin.
hardpractitioner heuristic - End each backtest origin on a complete aggregation period
Daily bookings are rolled up to weeks, and the cutoff falls on a Wednesday; the training week's total contains three days and the test week begins mid-week.
easypractitioner heuristic - Score lead-time demand as a cumulative quantity when the decision covers the whole lead time
A replenishment or seat-allocation decision commits stock for the next four days; the team reports daily MAPE for horizons 1 to 4.
mediumpractitioner heuristic - Build the naive benchmark at the production horizon from the origin value, not from the previous day
A 28-day-ahead model is compared with a naive baseline computed as yesterday's actual for every target day.
easydocumented fact - Align the seasonal benchmark to moving holidays, not to the same weekday last year
A seasonal naive forecast (value 364 days ago) is the benchmark for daily demand around religious holidays whose date moves about 10 to 11 days earlier each Gregorian year.
mediumpractitioner heuristic - Retire sMAPE from forecast reports because of its asymmetry and its failure at zero
A report uses symmetric MAPE because it is bounded and the team believes it fixes MAPE's bias toward low forecasts.
easydocumented fact - Aggregate ratios of errors to a benchmark with a geometric mean, not an arithmetic mean
Model A is compared with seasonal naive on 500 SKUs by averaging each SKU's error ratio (model error divided by benchmark error).
mediumempirical finding - Say whether a point forecast targets the mean or the median, and score it consistently
A skewed demand model is trained on squared error but selected on MAE, and two teams argue over which candidate is more accurate.
mediumdocumented fact - Split squared forecast error into bias and variance across origins before choosing a fix
A model's RMSE is above target and the team is deciding between a bias correction, more regularisation and an ensemble.
mediumpractitioner heuristic - Never add per-item quantile forecasts to get a quantile for the total
A planner sums the 90th-percentile forecasts of 40 flights to size a network-wide buffer.
mediumdocumented fact - Put a sampling error on measured interval coverage before calling a model miscalibrated
An 80% interval covered 74% of 50 backtest origins and the team wants to retune the model.
easypractitioner heuristic - Treat hits of overlapping multi-step intervals as dependent when testing coverage
Intervals for a 14-day horizon are issued every day, and coverage is computed over 120 consecutive origins as if each hit were independent.
harddocumented fact - Calibrate intervals separately at each horizon, since one-step residual spread is too narrow far ahead
A residual-based 80% interval built from one-step errors is reused for horizons 1 to 28.
mediumpractitioner heuristic - Check and fix quantile crossing before scoring or serving multi-quantile forecasts
A gradient-boosted model fits the 10th, 50th and 90th percentiles with separate runs and the dashboard sometimes shows the 90th below the median.
mediumdocumented fact - Divide CRPS or interval scores by a benchmark's score before averaging across series
A portfolio of 300 routes reports mean CRPS per model, and the average is driven by five large trunk routes.
mediumpractitioner heuristic - Evaluate a demand forecast against sales through the capacity cap, using quantiles
A model forecasts demand and is scored against recorded bookings on flights that sold out; bookings are capped at 180 seats.
hardpractitioner heuristic - Do not measure bias on uncensored days only, because dropping stock-outs selects low-demand days
To avoid censoring, a team computes forecast bias only on days that did not sell out and reports a persistent over-forecast of 10%.
hardpractitioner heuristic - Replay intermittent-demand forecasts through a stock policy and report service and stock, not only error
Spare-parts demand is mostly zeros with occasional orders, and two methods tie on MASE.
hardpractitioner heuristic - Compare two forecasts with a paired test on per-origin loss differences
Model B has a 3% lower mean error than model A over 52 weekly origins and is proposed as the replacement.
harddocumented fact - Add non-negativity to reconciliation when the series are counts or sales
Route forecasts from a model are all non-negative, but after MinT reconciliation a thin route shows -3 passengers on some days.
mediumempirical finding - Evaluate reconciled forecasts level by level against the base forecasts, since coherence is not accuracy
A reconciled forecast sums exactly and the team ships it on the strength of a better headline number at the network level.
mediumempirical finding - Monitor forecast error as a ratio to a frozen benchmark, not as an absolute number
A weekly dashboard alerts when MAPE exceeds 12%, and it fires every holiday season and stays quiet through real model decay in calm months.
mediumpractitioner heuristic - Log the action taken on each forecast so that monitored error is not contaminated by your own response
A capacity forecast triggers extra flights on routes forecast high, and the monitored error on those routes keeps improving.
hardpractitioner heuristic - Report how many series the new model beats the benchmark on, next to the average improvement
A candidate model improves average MASE over 2,000 series by 4% and the team proposes a full cutover.
easypractitioner heuristic - Choose between a deterministic and a stochastic trend on purpose, because the choice sets interval width
Annual passenger totals on a route have grown steadily for a decade, and two fitted models agree on the growth rate: a straight-line trend with ARMA errors, and an ARIMA model with one difference and drift.
mediumdocumented fact - Take the seasonal difference first and let a seasonal-strength measure decide whether you need one
Daily bookings show a strong weekly pattern and a trend, and an analyst applies a first difference and then wonders why the weekly spikes are still there.
easydocumented fact - Decide the ARIMA constant deliberately, because with differencing it sets the long-run trend
An automatic ARIMA run on quarterly fuel-cost data returns d = 1 with a non-zero constant, and the five-year forecast climbs in a straight line.
harddocumented fact - Compare models by AICc only within one model class and one differencing order
A team puts ETS, ARIMA and an ARIMA with an extra difference into one AICc table and ships the lowest value.
mediumdocumented fact - Let a damped trend compete with a linear trend whenever automatic forecasts run beyond a few periods
A monthly forecast for 1,000 route-level series is produced automatically, and a linear-trend Holt model projects 24 months of continued growth on routes that recently grew.
easydocumented fact - Difference the response and every predictor together in a dynamic regression
Weekly ticket sales are regressed on a price index, search volume and a trend; the response needs one difference to be stationary, but the predictors are left in levels.
mediumdocumented fact - In a regression with ARIMA errors, test the ARIMA innovations and not the regression residuals
A model with a promotion regressor and ARIMA(1,0,2) errors is checked by running a Ljung-Box test on the regression residuals, and the test fails.
harddocumented fact - Convert monthly totals to per-day or per-trading-day rates before modelling seasonality
Monthly revenue for a retailer swings partly because months contain different numbers of trading days, and the seasonal model learns a short-February dip and a long-month bump as seasonality.
easydocumented fact - Look for outliers in the robust STL remainder, because thresholds on the raw series flag seasonal peaks
A z-score rule on daily demand flags every December peak and misses a one-day system-outage dip in March.
mediumdocumented fact - Check that the method tolerates gaps, and model systematic closures with dummies rather than interpolating them
A store's daily sales are missing on public holidays when it is closed, and the pipeline either fails inside an exponential smoothing fit or silently interpolates the closures.
easydocumented fact - Train on the driver forecasts that existed at each origin, not on the values that were later observed
A day-ahead load model is trained on observed temperature, but in production it receives a weather forecast whose error grows with lead time.
hardpractitioner heuristic - In a direct model, align future-known drivers with the target date and history features with the origin
A per-horizon boosted model for h = 1 to 14 days is built by shifting every column, including the promotion calendar, by h days.
mediumpractitioner heuristic - Treat cross-classified series as grouped, not as an arbitrary single tree
Seats sold are available by route, cabin and channel, and an analyst forces them into route, then cabin, then channel to use a hierarchical method, although channel totals across routes are also reported.
mediumdocumented fact - Pick the MinT covariance estimator by the number of series relative to the history length
A retail group reconciles forecasts for 4,000 item-store series with 104 weeks of history, using MinT with the sample covariance of one-step forecast errors.
harddocumented fact - Forecast a new route or product with a global model on its attributes instead of waiting for local history
A new route launches with three weeks of data, so a local seasonal model has nothing to estimate seasonal indices from.
easypractitioner heuristic - Weight a global model's loss by the weights of the metric that will judge it
A global boosted model for 30,000 SKU series is trained with unweighted squared error, but it will be judged on an error that is weighted by recent revenue.
hardpractitioner heuristic - Give a boosted tree the same-weekday and same-season lags explicitly
A daily boosted model uses lags 1 to 7 and a rolling 7-day mean, and its Monday forecasts keep missing the Monday level after a holiday week.
easypractitioner heuristic - Route intermittent series to a method by interval and size variability, then confirm by backtest
A spare-parts catalogue of 20,000 items is forecast with a single method, although some items sell steadily and others sell every few months in varying quantities.
mediumempirical finding - Use a probability-updating method such as TSB when items can become obsolete
A wholesaler's slow items sometimes stop selling for good, yet a Croston forecast stays at the old demand rate for months because it only updates when a sale occurs.
harddocumented fact - Add a change-of-slope regressor when the growth rate shifts but the level does not jump
A competitor opens a route on a known date in 2024 and market volume does not drop at once; the growth rate slows from that date.
mediumdocumented fact - Treat a closed fare class as censoring even when the cabin is not full
A hypothetical flight with 180 seats has sold 118 when its lowest fare class closes, and that class's booking curve goes flat from that day.
hardpractitioner heuristic - Include lagged promotion or advertising effects and pick the lag count by AICc on a common sample
Weekly sales respond to a campaign over several weeks, but the model contains only this week's campaign flag.
mediumdocumented fact - Check that a zero-shot foundation forecaster can use your future-known drivers before expecting promotion peaks
A team evaluates an off-the-shelf pretrained forecaster on weekly sales with a known promotion calendar and finds that it misses every promotion peak.
mediumdocumented fact - Match a foundation model's context window to your longest seasonal cycle
Hourly electricity demand has weekly and annual cycles, and a pretrained forecaster gets the last 512 hours as context.
mediumempirical finding - Handle sparse and flat series before sending them to a mean-scaled, tokenizing forecaster
A pretrained model that scales each series by the mean absolute value of its context is run on spare-part demand where one week in twenty has a sale of one unit.
harddocumented fact - Feed a statistical model's out-of-sample forecast to the ML model as a feature, rebuilt at every origin
A boosted-tree forecaster is worse than ETS at trend and seasonal shape but better at promotions, and the team wants both strengths.
hardpractitioner heuristic - Bag an ETS model with bootstrapped STL remainders when single fits are unstable
Monthly ETS forecasts for 400 series change noticeably each time the models are refitted, though the history changed by one point.
mediumempirical finding - Forecast a monthly series at several temporal aggregation levels and reconcile them
Monthly route demand is forecast with one model at one frequency, while planners also use quarterly and annual figures that disagree with the monthly sum.
hardempirical finding