LLMs, GenAI, RAG, agents, evaluation · insight-1536

Rewrite follow-up chat turns into standalone queries, and keep the original for fusion

Situation
A chat assistant embeds the raw user message, and a follow-up such as "what about the child fare?" arrives with no route or product named.
Action
Rewrite the latest turn with recent history into a self-contained query, retrieve with both rewritten and original text and fuse by rank, log both, and evaluate on a labelled multi-turn set.
Reason
The text a user types often differs from the text that retrieves the right evidence; research on rewriting queries for retrieval-augmented LLMs reports gains from adapting the query to the retriever, on single-turn QA benchmarks; the benefit for multi-turn chat is yours to measure.
Boundary
Multi-turn assistants with pronouns, ellipsis and topic carry-over.

Symptom or risk

The retriever searches on a fragment that carries no entity, so the top results come from an unrelated topic and the answer drifts.

Single-turn search boxes, where the query is already self-contained.

Each rewrite is an extra model call and can inject a wrong assumption from earlier turns.

Evaluating only on single-turn questions, which never exercises the rewrite step.

The Rewrite-Retrieve-Read paper reports that adapting the search query, because input text and the knowledge needed for retrieval often differ, gave consistent gains on open-domain and multiple-choice QA.
Query Rewriting for Retrieval-Augmented Large Language Models