LLMs, GenAI, RAG, agents, evaluation · insight-0645
Cap and shape tool results before they enter the context window
- Evidence type
- practitioner heuristic
- Difficulty
- easy
- Roles
- Applied AI, ML engineer
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
A tool returns a full API payload, a 2,000-row query result or an entire web page to the agent.
Action
Return only the fields the task needs, paginate or summarise large results in code, include counts ("2,014 rows, first 20 shown"), and let the agent request more explicitly.
Reason
Tool results are billed and processed as input on every later turn of the loop, and long, noisy context reduces how well models use the relevant parts.
Boundary
Database, search, file and browsing tools.
Symptom or risk
Each step adds thousands of tokens, cost and latency climb, key fields get buried, and untrusted text in the payload rides along.
Exceptions
Tasks that need the full document, such as contract review; then budget for it.
Trade-offs
Over-trimming hides the field the agent needed; log what was cut.
Pitfalls
Truncating mid-structure (half a JSON object), which the model then misreads.
Lessons and concepts
Claims and their sources
Tool-use requests are billed on all input tokens, including tool_result content blocks.
Language models use information in long contexts unevenly, with degraded accuracy for relevant content placed mid-context.