LLMs, GenAI, RAG, agents, evaluation · insight-0622
Include unanswerable questions and measure how often the assistant abstains
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- Applied AI, Data scientist
- Industries
- aviation, insurance
Summary: situation, action, reason, boundary
Situation
The golden set contains only questions that the corpus can answer.
Action
Add 10 to 20 percent of realistic out-of-corpus questions, define the expected abstention response, and report the false-answer rate on that slice next to accuracy on answerable ones.
Reason
Language models produce fluent text whether or not evidence exists; hallucination research separates output contradicted by the source from content the source cannot verify, and the second kind needs its own test.
Boundary
Customer-facing assistants, policy and compliance Q&A.
Symptom or risk
The system is never tested on the case where it should say "the documents do not cover this", so it invents plausible fare rules or coverage terms in production.
Exceptions
Assistants explicitly allowed to answer from general knowledge; then label which answers must be grounded.
Trade-offs
Pushing abstention too hard produces refusals on answerable questions; track both rates.
Pitfalls
Writing unanswerable questions that are obviously off-topic; use near-misses such as a route or product the corpus does not cover.
Lessons and concepts
Claims and their sources
Hallucination surveys distinguish intrinsic hallucination (contradicting the source) from extrinsic hallucination (content that cannot be verified from the source).