LLMs, GenAI, RAG, agents, evaluation · insight-0602
Give each chunk the context it lost when it was cut out of the document
- Evidence type
- empirical finding
- Difficulty
- medium
- Roles
- Applied AI, ML engineer
- Industries
- aviation, finance
Summary: situation, action, reason, boundary
Situation
Many chunks say things like "this fee is waived" or "revenue grew 3%" without naming the product, route or period they refer to.
Action
Prepend a short context header to each chunk before embedding and lexical indexing: document title, section path, effective date, and, if affordable, a generated one-line situating summary.
Reason
Both embedding similarity and BM25 can only match words and meaning present in the indexed text. Anthropic reported fewer retrieval failures on its own test sets when chunks were contextualised this way.
Boundary
Corpora with many similar documents that differ by entity, date or product.
Symptom or risk
Queries that name the entity ("change fee on IST-FRA economy") miss the right chunk because the chunk never mentions it.
Exceptions
Self-contained chunks such as FAQ question-answer pairs gain little.
Trade-offs
Generated summaries add ingestion cost per chunk and must be regenerated when documents change.
Pitfalls
Treating the vendor-reported gain as a guarantee; measure recall@k on your own golden set before and after.
Lessons and concepts
Claims and their sources
Anthropic reported that prepending model-generated chunk context before embedding and BM25 indexing cut the top-20 retrieval failure rate on its test sets from 5.7% to 2.9%, and to 1.9% with reranking (vendor-reported).