Time series and forecasting · insight-0219
Evaluate prediction intervals on coverage and width together
- Evidence type
- documented fact
- Difficulty
- medium
- Roles
- Data scientist
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
Two interval methods are compared, and the team picks the one with the highest coverage.
Action
Check that empirical coverage is close to nominal for each horizon, then prefer the narrower intervals among calibrated ones; summarise with a Winkler or interval score that penalises both width and misses.
Reason
Probabilistic forecasts should maximise sharpness subject to calibration, and interval scores combine width with a penalty for observations outside.
Boundary
Inventory buffers, staffing, overbooking limits, any interval-driven decision.
Symptom or risk
Very wide intervals always cover, so 'highest coverage wins' promotes useless ranges that no planner can act on.
Exceptions
Where one-sided risk dominates (stockouts only), evaluate the relevant quantile rather than the interval.
Trade-offs
Proper scores are less intuitive than coverage percentages; report both.
Pitfalls
Measuring coverage on the training period.
Lessons and concepts
Claims and their sources
Probabilistic forecasting aims to maximise the sharpness of predictive distributions subject to calibration.
The Winkler score adds the interval width to a penalty when the observation falls outside the interval.