LLMs, GenAI, RAG, agents, evaluation · insight-1537
Set the abstain threshold from a coverage-versus-error curve on labelled answerable and unanswerable questions
- Evidence type
- empirical finding
- Difficulty
- medium
- Roles
- Applied AI, Data scientist
- Industries
- insurance, finance
Summary: situation, action, reason, boundary
Situation
A RAG assistant refuses when the top retrieval score is below a threshold someone picked by eye, and product wants fewer refusals.
Action
Sweep the threshold over a labelled set containing answerable and unanswerable questions, plot the share answered against the error rate among answered, and choose the point that matches the cost of a wrong answer.
Reason
Selective classification formalises abstention as trading coverage for lower error at a user-set risk level; the curve makes that trade explicit and reviewable.
Boundary
Assistants where a wrong answer costs more than a handoff: policy, compliance, medical or financial support.
Symptom or risk
Moving the threshold shifts wrong answers and unnecessary refusals against each other without anyone seeing the trade, so changes are argued from anecdotes.
Exceptions
Exploratory search where showing weak results is the point.
Trade-offs
The curve is only as good as the unanswerable questions in the set, and it must be redrawn when the retriever changes.
Pitfalls
Using a raw similarity score as the signal across retriever versions; scores are not comparable between them.
Example
Hypothetical sweep: answering everything gives 18% errors; a threshold answering 70% gives 8%; answering 50% gives 4%. Counting a wrong answer as five refusals, 70% coverage costs 0.7 x 0.08 x 5 + 0.30 = 0.58 per question against 0.60 at 50%; at ten refusals, 50% wins (0.70 against 0.86).
Lessons and concepts
Claims and their sources
The selective classification paper describes a reject option that meets a user-set risk level by rejecting inputs, trading coverage for lower error, for example 2% top-5 ImageNet error at nearly 60% coverage.