LLMs, GenAI, RAG, agents, evaluation · insight-0611

Put the strongest evidence at the start of the context, not buried in the middle

Situation
Context assembly concatenates passages in arbitrary order, for example by document id or ingestion time.
Action
Order passages by reranker score with the best first, keep the total short, and test position sensitivity by moving the gold passage on a few golden questions.
Reason
Liu and colleagues found a position effect: accuracy was often highest when relevant information was at the beginning or end of the input and dropped when it sat in the middle, even for long-context models.
Boundary
Multi-passage RAG prompts, long tool outputs, conversation histories.

Symptom or risk

The model answers from a weaker passage at the top while the decisive one sits in the middle of a long context.

Newer models may show weaker position effects; measure on your model rather than assuming the published curve.

Strict score ordering can separate passages that only make sense together; keep adjacent chunks of one document together.

Fixing ordering while the context still holds 40 passages; shortening the context usually matters more.

Model accuracy was often highest when relevant information appeared at the beginning or end of the context and degraded when it was in the middle.
Lost in the Middle: How Language Models Use Long Contexts