Time series and forecasting · insight-0245
Measure forecast value added at every step, including human overrides
- Evidence type
- practitioner heuristic
- Difficulty
- easy
- Roles
- Data analyst, Data scientist
- Industries
- retail, manufacturing, logistics
Summary: situation, action, reason, boundary
Situation
Planners adjust the statistical forecast every month, and nobody knows whether the adjustments help.
Action
Archive the naive forecast, the statistical forecast and each adjusted version, compute the same accuracy metric for each, and report the change each step adds relative to the step before and to naive.
Reason
Forecast value added analysis is designed to reveal steps in the forecasting process that do not improve accuracy.
Boundary
Demand planning, sales and operations planning, budgeting with judgmental input.
Symptom or risk
Manual overrides consume hours and may make accuracy worse, while the process is credited for good months and the model blamed for bad ones.
Exceptions
Overrides that incorporate information unavailable to the model (a signed contract) should be evaluated as a separate category.
Trade-offs
Showing negative value added can be politically sensitive; present it as process data, not blame.
Pitfalls
Comparing steps on different time windows or item sets.
Lessons and concepts
Claims and their sources
Forecast value added analysis is presented as a way to find waste in the forecasting process, meaning steps that do not improve accuracy.