Engineering, production, governance, cost, communication · insight-1293
Seed review queues with known-answer cases to test whether reviewers really check
- Evidence type
- empirical finding
- Difficulty
- medium
- Roles
- Data scientist, ML engineer, Applied AI
- Industries
- banking, insurance, healthcare
Summary: situation, action, reason, boundary
Situation
Analysts approve 99 percent of a fraud model's suggested actions (hypothetical).
Action
Inject a small share of cases with known correct answers, including some deliberately wrong suggestions, into the queue. Track catch rate and time per case by reviewer, and redesign the screen to show evidence, not only the recommendation.
Reason
Reviews of the human-automation literature find that automation bias causes omission and commission errors when an aid is imperfect, affects experts and resists simple practice and instruction.
Boundary
Human-in-the-loop fraud, content, claims, clinical and compliance review.
Symptom or risk
Review has become a rubber stamp: reviewers follow the suggestion, including wrong ones, and miss issues the tool did not flag, so the human control is not operating.
Exceptions
Plant cases only with privacy review, and tell staff that checks exist.
Trade-offs
Costs reviewer time and a maintained set of known-answer cases.
Pitfalls
Planting easy cases that overstate vigilance.
Lessons and concepts
Claims and their sources
Parasuraman and Manzey review evidence that automation bias yields omission and commission errors with imperfect decision aids, occurs in naive and expert users, and resists training and instructions.