LLMs, GenAI, RAG, agents, evaluation · insight-1709
Trim agent history by whole tool exchanges so a call and its result stay together
- Evidence type
- practitioner heuristic
- Difficulty
- easy
- Roles
- ML engineer, Applied AI
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
An agent loop drops the oldest messages to fit the context window, and the cut falls between a tool call and its result.
Action
Cut only at boundaries that keep each tool call and its result in the same kept or dropped group. Add a unit test that trims at every offset of a recorded run and checks that every result has its call, then validate the history before sending.
Reason
Anthropic's tool-use documentation (as read on 2026-10-12) says each tool result must immediately follow its tool use in the history and match its id, and shows the error returned when this is broken.
Boundary
Hand-written agent loops, memory managers and any code that truncates or compacts a message list.
Symptom or risk
The API rejects the history for an unmatched tool id, or the model sees a result with no call and reasons from an action it cannot see.
Exceptions
SDK tool runners that manage history themselves, and providers with different message formats, whose rules should be checked.
Trade-offs
Whole-exchange cuts drop more or less than the target, so budgets are approximate.
Pitfalls
Trimming by token count only, or dropping the call and keeping a summary that omits what the tool returned.
Example
Hypothetical nine-message history: user, assistant (call), user (result), assistant, user, assistant (call), user (result), assistant, user. Dropping two messages leaves the first result without its call; dropping four removes the whole first exchange and starts at a user message.
Lessons and concepts
Claims and their sources
Anthropic's tool-use documentation (as read on 2026-10-12) states that tool_result blocks must immediately follow the corresponding tool_use blocks in the message history, matched by tool_use_id, and that a violation causes an error naming tool_use ids found without tool_result blocks.