Classical ML, validation, metrics, interpretation, debugging · insight-0144
Expect MAPE-selected models to under-forecast and check bias before adopting them
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- Data scientist, Data analyst
- Industries
- retail, logistics, aviation
Summary: situation, action, reason, boundary
Situation
You choose between models or tune a model by minimizing MAPE on positive targets.
Action
Report mean signed error alongside MAPE, choose the metric from the decision's cost asymmetry, and prefer a quantile or cost-weighted loss when under- and over-prediction cost differently.
Reason
For a positive actual, an under-forecast's percentage error is capped at 100%, while an over-forecast's is unbounded, so minimizing MAPE rewards low forecasts.
Boundary
Demand, capacity, staffing and inventory forecasts evaluated with percentage errors.
Symptom or risk
The winner systematically predicts low, causing shortages, under-capacity or spill of demand.
Exceptions
If under-forecasting genuinely costs less than over-forecasting, the bias may be acceptable, but make it a stated choice.
Trade-offs
Signed-error tracking adds a second number to explain but prevents silent bias.
Pitfalls
Comparing MAPE across models that were trained on different losses without examining their bias.
Example
Hypothetical actual of 100: forecasting 50 scores 50% error, forecasting 150 also 50%, forecasting 300 scores 200%, while forecasting 0 is capped at 100%.