Time series and forecasting · insight-1220
Evaluate a demand forecast against sales through the capacity cap, using quantiles
- Evidence type
- practitioner heuristic
- Difficulty
- hard
- Roles
- Data scientist
- Industries
- aviation, logistics
Summary: situation, action, reason, boundary
Situation
A model forecasts demand and is scored against recorded bookings on flights that sold out; bookings are capped at 180 seats.
Action
Score quantile forecasts after applying the cap: compare min(q_tau, capacity) with observed sales using pinball loss. Report separately the share of origins where the forecast exceeds capacity.
Reason
The tau-quantile of a capped variable equals the cap-limited tau-quantile of the uncapped one, so quantile scores stay valid under censoring, whereas the mean does not commute with the cap.
Boundary
Sales, bookings and throughput limited by capacity, inventory or queue size.
Symptom or risk
A correct demand forecast of 200 is penalised against 180, and candidates that shrink toward the cap look better, which teaches the system to under-forecast.
Exceptions
Demand forecasts for pricing or capacity decisions need the unconstrained value; judge them on uncensored periods and with a model-based check.
Trade-offs
Capped scoring says nothing about demand above the cap.
Pitfalls
Scoring the mean forecast with min(f, cap); that is not the mean of the capped demand.
Example
Hypothetical demand normal (100, 20) with cap 110: mean demand is 100 but mean sales are 96.0, so an unbiased forecast looks 4% high.