LLMs, GenAI, RAG, agents, evaluation · insight-0504

Slice LLM accuracy by entity popularity because long-tail facts fail first

Situation
A travel assistant answers well about major hubs in demos, and the team plans to launch it for all airports in the network.
Action
Tag eval questions by how common the entity is (passenger volume or document count as a proxy), report accuracy per popularity bucket, and route low-popularity entities to retrieval or lookups.
Reason
Studies tie closed-book factual accuracy to how often the relevant information appears in pretraining data, and find retrieval helps most for less popular entities.
Boundary
Closed-book factual Q&A over any catalogue with a long tail.

Symptom or risk

Questions about small regional airports, rare aircraft types or niche fare products get confident wrong answers that the demo never exercised.

When every answer comes from retrieved documents, retrieval coverage matters more than entity popularity.

Popularity tags need a proxy and some labelling time.

Building the eval set from the same frequent cases that were used in the demo.

Kandpal et al. found QA accuracy strongly related to the number of relevant pretraining documents, and that retrieval augmentation reduced this dependence.
Large Language Models Struggle to Learn Long-Tail Knowledge
Mallen et al. found LMs struggle with less popular factual knowledge, retrieval helps most on long-tail questions, and retrieval can hurt large models on popular entities.
When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories