LLMs, GenAI, RAG, agents, evaluation · insight-1507
Restate the full requirement set in one message instead of revealing it across turns
- Evidence type
- empirical finding
- Difficulty
- medium
- Roles
- Applied AI, ML engineer
- Industries
- aviation, cross-industry
Summary: situation, action, reason, boundary
Situation
An itinerary assistant receives a trip request in pieces: dates in turn one, budget in turn three, a baggage constraint in turn five.
Action
Evaluate the same tasks sharded across turns and as one consolidated message; in production, keep requirements as structured state, send a consolidated restatement, and restart the thread from it after corrections.
Reason
Laban et al. simulated over 200,000 conversations and found an average 39% performance drop from single-turn to multi-turn across six tasks, mainly from unreliability, with models rarely recovering from early wrong turns.
Boundary
Underspecified multi-turn assistance such as planning, coding and data analysis.
Symptom or risk
The model commits to an early assumption, builds on it, and later turns patch rather than replace it, so final plans satisfy fewer constraints than a single message would.
Exceptions
Tasks fully specified in the first message, and casual chat.
Trade-offs
Restating costs tokens and needs state tracking, and a restart loses conversational nuance.
Pitfalls
Adding a correction as one more turn on top of a wrong earlier attempt.
Lessons and concepts
Claims and their sources
Laban et al. report that all tested top LLMs did significantly worse in multi-turn than in single-turn settings, with an average drop of 39% across six generation tasks over more than 200,000 simulated conversations, attributed mainly to unreliability rather than lost aptitude.