LLMs, GenAI, RAG, agents, evaluation · insight-1533

Retrieve small passages for matching, then expand to the parent section for the generator

Situation
Long sections embed poorly because they mix several topics, but short chunks hand the generator half a rule without its conditions.
Action
Index small units (a clause or a few sentences) with a parent id, retrieve and rerank on those, then replace hits by their parent section, deduplicate parents and cap the context at a token budget.
Reason
Matching and reading are different tasks: small units sharpen the match, and the parent restores conditions and exceptions that the small unit cannot carry.
Boundary
Policies, contracts, manuals and API docs where meaning depends on neighbouring clauses.

Symptom or risk

The index must pick one chunk size for two jobs, matching a query precisely and giving enough surrounding text, and it does neither well.

Corpora made of short self-contained items such as FAQ entries.

Parent expansion inflates context tokens and can pull in text unrelated to the question.

Several hits sharing one parent consume the budget repeatedly if parents are not deduplicated first.