Time series and forecasting · insight-0212
Do not choose intermittent-demand models by MAE because it rewards forecasting zero
- Evidence type
- documented fact
- Difficulty
- hard
- Roles
- Data scientist, Data analyst
- Industries
- manufacturing, retail
Summary: situation, action, reason, boundary
Situation
On a mostly-zero series, an all-zero forecast wins the MAE comparison against every model.
Action
Evaluate intermittent forecasts with metrics aligned to the decision: MSE or scaled MSE for mean rates, pinball loss at the service-level quantile, or a simulated inventory cost and fill rate.
Reason
Absolute error is minimised by the median; when more than half of periods are zero, the median is zero.
Boundary
Spare parts, long-tail retail, rare-event counts.
Symptom or risk
A model that never predicts demand is promoted, and the inventory system stops stocking the part.
Exceptions
If zero stock is genuinely acceptable for the item, the zero forecast may be the right decision; make that explicit.
Trade-offs
Squared-error metrics are sensitive to occasional large orders; simulation is more relevant but more work.
Pitfalls
Reporting a portfolio MAE that improved only because more items are forecast at zero.
Lessons and concepts
Claims and their sources
Under symmetric piecewise linear (absolute) loss the optimal point forecast is a median, and under asymmetric piecewise linear loss it is a quantile.