LLMs, GenAI, RAG, agents, evaluation · insight-1578
Compute headroom before trusting a saturated eval, and harden or retire it
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- Applied AI, Data scientist
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
Several prompt or model variants all score above 95% on your eval and the differences between them look tiny.
Action
Compute headroom (100% minus the best score) and the interval on the score; if the interval is wider than the gains you care about, add harder items mined from real failures and keep the old set only as a regression guard.
Reason
Benchmarks lose discriminating power once models solve most of them, which is why dynamic benchmarks collect new items that current models get wrong.
Boundary
Mature assistants whose eval was written when the task was harder.
Symptom or risk
Near the ceiling, a few items decide the ranking, so the eval can no longer separate variants and teams read noise as progress or conclude that nothing matters.
Exceptions
A guard set is meant to stay near 100%; only a drop matters there.
Trade-offs
Harder items are costlier to label and make the headline score fall; version the change and re-baseline.
Pitfalls
Adding obscure trick questions that no user sends; derive hard items from observed failures.
Example
Hypothetical: 96% accuracy on 200 items has a 95% interval of about plus or minus 2.7 points, so a one-point gain is invisible.
Lessons and concepts
Claims and their sources
The Dynabench authors argue that models reach strong benchmark scores quickly yet still fail simple challenge examples, and propose collecting new items that a target model gets wrong.