LLMs, GenAI, RAG, agents, evaluation · insight-0513

Triage hallucinations as contradictions or unsupported additions because the fixes differ

Situation
Reviewers label 40 generated summaries of delay-compensation cases simply as hallucinated or fine.
Action
Label each bad output as intrinsic (contradicts the provided source) or extrinsic (adds content the source cannot verify), and count both types per pipeline stage.
Reason
The hallucination literature separates these types; contradictions point to failures in reading the given context, while unverifiable additions point to the model filling gaps from parametric knowledge.
Boundary
Summarisation, grounded Q&A, report generation.

Symptom or risk

The team cannot tell whether to fix retrieval, the prompt or decoding, so fixes are tried at random.

An extrinsic addition can be true; the label means unverified against the source, not false.

Two-type labelling takes more reviewer time than a binary flag.

Accepting a true but unsupported statement where the product promises provenance.

Ji et al. distinguish intrinsic hallucination (output contradicting the source) from extrinsic hallucination (output that cannot be verified from the source).
Survey of Hallucination in Natural Language Generation