Engineering, production, governance, cost, communication · insight-1283
Evaluate against past decision-makers with a selective-labels method, not on approved cases alone
- Evidence type
- practitioner heuristic
- Difficulty
- hard
- Roles
- Data scientist
- Industries
- banking, insurance, healthcare
Summary: situation, action, reason, boundary
Situation
A credit model is scored on repaid versus defaulted loans, but outcomes exist only for applicants that underwriters approved (hypothetical).
Action
State how labels go missing. If cases reach reviewers of differing leniency as if at random, compare model and reviewers at the same approval rate on the cases the most lenient reviewers released. Otherwise label results as approved-population only.
Reason
When a decision determines whether the outcome is observed, labelled data are a non-random sample, and the selective-labels literature uses the heterogeneity of decision-makers to compare a model with them.
Boundary
Lending, bail-like risk assessment, claims triage, clinical referrals: any process where a decision hides some outcomes.
Symptom or risk
Accuracy on approved cases says nothing about the region the underwriters rejected, so a claimed gain over human decisions may be an artefact of who was observed.
Exceptions
If a randomised exploration slice exists, evaluate on that slice instead.
Trade-offs
Needs many reviewers and many cases each; estimates are noisy.
Pitfalls
Assigning hard cases to particular reviewers, which breaks the as-if-random assumption.
Lessons and concepts
Claims and their sources
Lakkaraju and colleagues describe evaluating predictions when outcomes are observed only for cases a decision-maker selected, and propose contraction, which uses heterogeneity among decision-makers and assumes cases are as if randomly assigned to them.