LLMs, GenAI, RAG, agents, evaluation · insight-1550
Check that each cited passage actually supports its claim, using an entailment check, not only that the id exists
- Evidence type
- empirical finding
- Difficulty
- medium
- Roles
- Applied AI, ML engineer
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
A RAG answer shows inline citations, and the pipeline already verifies that every cited id is in the retrieved set.
Action
Split the answer into claims, run an entailment or judge check of each claim against its cited passage, flag or remove unsupported ones, and track citation recall and precision on a labelled sample.
Reason
Audits of citation-producing systems find substantial unsupported text: the ALCE benchmark reports best models lacking full citation support about half the time on ELI5, and a human audit of generative search engines found about 51.5% of sentences fully supported.
Boundary
Assistants that show citations to users or auditors.
Symptom or risk
The id is valid, but the cited passage does not say what the sentence claims, so the interface displays unsupported claims with a credible-looking source.
Exceptions
Free-form brainstorming where there is no evidence to cite.
Trade-offs
The check adds a model call per claim and can itself be wrong.
Pitfalls
Using the generating model as its own checker, which tends to approve its own citations.
Lessons and concepts
Claims and their sources
The ALCE benchmark found that even the best models lack complete citation support about half the time on ELI5.
A human audit of four generative search engines found about 51.5% of generated sentences fully supported by their citations and about 74.5% of citations supporting their sentence.