LLMs, GenAI, RAG, agents, evaluation · insight-1579
Audit the items every variant fails before concluding the models are the problem
- Evidence type
- empirical finding
- Difficulty
- medium
- Roles
- Applied AI, Data scientist
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
A handful of eval items are failed by every prompt version and model you have tried.
Action
List items with unanimous failure, have a domain reviewer re-label each blind to the model outputs, and sort them into label error, ambiguous question, missing context or genuine capability gap.
Reason
When independent systems agree with each other and disagree with the gold label, a label problem is more likely than a shared model weakness. Public test sets also contain measurable label error rates.
Boundary
Golden sets with a few hundred items or more and at least three diverse variants to compare.
Symptom or risk
Those items are often mislabelled, ambiguous or unanswerable from the given context, which caps the score and misdirects prompt work toward a fix that does not exist.
Exceptions
Adversarially built items are expected to be failed by all systems; check that they are valid rather than relabel them.
Trade-offs
Reviewer time, and a risk of relabelling toward the models; keep the reviewer blind to outputs.
Pitfalls
Fixing only the items that fail; passing items can be wrong too, so sample those for review as well.
Lessons and concepts
Claims and their sources
Northcutt et al. estimate an average of at least 3.3% label errors in the test sets of 10 widely used datasets.