Classical ML, validation, metrics, interpretation, debugging · insight-1195
Publish a score contract that says whether the number is a probability, a rank or a margin, and how it was calibrated
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- Data scientist, ML engineer
- Industries
- banking, aviation, insurance
Summary: situation, action, reason, boundary
Situation
Downstream teams consume a model's score in pricing, capacity or campaign logic that they own.
Action
Write the contract with the model: what the score means (calibrated probability, rank only, margin), the calibration method, the calibration data and date, the valid range, and the refresh trigger. Check calibration on held-out data at each release and fail the release if the contract is broken.
Reason
Predicted probabilities are not automatically calibrated, and a calibrator fitted on training data is biased; the consumer needs to know which kind of number they hold.
Boundary
Scores shared across teams or used in expected-value calculations.
Symptom or risk
A boosted or forest score, or one trained with class weights, is multiplied by money as if it were a probability; a retrain shifts its scale and the downstream arithmetic changes without any alarm.
Exceptions
Scores used only for ordering inside one team's own review queue.
Trade-offs
The contract constrains later changes to the model and needs ownership.
Pitfalls
Describing the score as a probability because the output lies between 0 and 1.
Lessons and concepts
Claims and their sources
scikit-learn states that predict_proba can be read as a confidence level only for well calibrated classifiers, that logistic regression is more likely to be calibrated by itself while random forest outputs are averaged over high-variance trees, and that calibrating on training data yields a biased calibrator.