Practical insights
- 1000published
- 25in review
- 8topic families
Results
- Do not choose intermittent-demand models by MAE because it rewards forecasting zero
On a mostly-zero series, an all-zero forecast wins the MAE comparison against every model.
harddocumented fact - Unconstrain sold-out flights before training a demand forecast
A route forecast is trained on passengers flown, and many peak departures hit capacity.
harddocumented fact - Check exchangeability before trusting split-conformal coverage on a time series
Split conformal intervals are calibrated on last year's residuals and promised to give 90% coverage.
harddocumented fact - Track realised interval coverage online and adjust the miscoverage level when it drifts
A deployed 90% interval for hourly call volumes covered 90% in backtests but 78% over the last month.
harddocumented fact - Decide explicitly how to treat a regime-change period in training and evaluation
Two years of history include an abrupt collapse and slow recovery in travel demand.
hardpractitioner heuristic - Do not answer a price what-if by changing the price input of a demand forecast
A manager asks what bookings would be if fares were cut 5%, and the analyst re-scores the forecasting model with lower fares.
harddocumented fact - Model spill and unconstrained demand before estimating gains from added capacity
A planner asks for passenger gains if the aircraft on a route changes from 160 to 180 seats.
hardpractitioner heuristic - Evaluate an intervention against a counterfactual forecast with control series
A loyalty campaign launched on one market, and leadership wants to know whether bookings rose because of it.
harddocumented fact - Count every strategy variant you tried before trusting the best backtest
A trading or pricing signal looks strong after trying many lookback windows, thresholds and universes on the same history.
harddocumented fact - Estimate reconciliation weights and aggregate features only from data before each origin
A hierarchical forecast reconciles with MinT; its error covariance is estimated once from residuals over the whole history and then used in every backtest origin.
hardpractitioner heuristic - Treat hits of overlapping multi-step intervals as dependent when testing coverage
Intervals for a 14-day horizon are issued every day, and coverage is computed over 120 consecutive origins as if each hit were independent.
harddocumented fact - Evaluate a demand forecast against sales through the capacity cap, using quantiles
A model forecasts demand and is scored against recorded bookings on flights that sold out; bookings are capped at 180 seats.
hardpractitioner heuristic - Do not measure bias on uncensored days only, because dropping stock-outs selects low-demand days
To avoid censoring, a team computes forecast bias only on days that did not sell out and reports a persistent over-forecast of 10%.
hardpractitioner heuristic - Replay intermittent-demand forecasts through a stock policy and report service and stock, not only error
Spare-parts demand is mostly zeros with occasional orders, and two methods tie on MASE.
hardpractitioner heuristic - Compare two forecasts with a paired test on per-origin loss differences
Model B has a 3% lower mean error than model A over 52 weekly origins and is proposed as the replacement.
harddocumented fact - Log the action taken on each forecast so that monitored error is not contaminated by your own response
A capacity forecast triggers extra flights on routes forecast high, and the monitored error on those routes keeps improving.
hardpractitioner heuristic - Decide the ARIMA constant deliberately, because with differencing it sets the long-run trend
An automatic ARIMA run on quarterly fuel-cost data returns d = 1 with a non-zero constant, and the five-year forecast climbs in a straight line.
harddocumented fact - In a regression with ARIMA errors, test the ARIMA innovations and not the regression residuals
A model with a promotion regressor and ARIMA(1,0,2) errors is checked by running a Ljung-Box test on the regression residuals, and the test fails.
harddocumented fact - Train on the driver forecasts that existed at each origin, not on the values that were later observed
A day-ahead load model is trained on observed temperature, but in production it receives a weather forecast whose error grows with lead time.
hardpractitioner heuristic - Pick the MinT covariance estimator by the number of series relative to the history length
A retail group reconciles forecasts for 4,000 item-store series with 104 weeks of history, using MinT with the sample covariance of one-step forecast errors.
harddocumented fact - Weight a global model's loss by the weights of the metric that will judge it
A global boosted model for 30,000 SKU series is trained with unweighted squared error, but it will be judged on an error that is weighted by recent revenue.
hardpractitioner heuristic - Use a probability-updating method such as TSB when items can become obsolete
A wholesaler's slow items sometimes stop selling for good, yet a Croston forecast stays at the old demand rate for months because it only updates when a sale occurs.
harddocumented fact - Treat a closed fare class as censoring even when the cabin is not full
A hypothetical flight with 180 seats has sold 118 when its lowest fare class closes, and that class's booking curve goes flat from that day.
hardpractitioner heuristic - Handle sparse and flat series before sending them to a mean-scaled, tokenizing forecaster
A pretrained model that scales each series by the mean absolute value of its context is run on spare-part demand where one week in twenty has a sale of one unit.
harddocumented fact - Feed a statistical model's out-of-sample forecast to the ML model as a feature, rebuilt at every origin
A boosted-tree forecaster is worse than ETS at trend and seasonal shape but better at promotions, and the team wants both strengths.
hardpractitioner heuristic - Forecast a monthly series at several temporal aggregation levels and reconcile them
Monthly route demand is forecast with one model at one frequency, while planners also use quarterly and annual figures that disagree with the monthly sum.
hardempirical finding