LLMs, GenAI, RAG, agents, evaluation · insight-1604
Treat third-party tool descriptions as untrusted input and pin approved versions
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- Applied AI, ML engineer
- Industries
- saas, cross-industry
Summary: situation, action, reason, boundary
Situation
An agent connects to tool servers run by other teams or vendors, and their tool names, descriptions and schemas are loaded into the prompt.
Action
Snapshot approved tool manifests (name, description, schema, annotations), hash them, refuse to load on mismatch and review diffs like a dependency upgrade. Show the user the full arguments before sensitive calls.
Reason
The Model Context Protocol specification says clients must treat tool annotations as untrusted unless they come from trusted servers, lets servers announce changes to the tool list at runtime, and advises showing tool inputs to the user to avoid exfiltration. The spec names annotations; extending the same distrust to descriptions and schemas is this record's own judgement.
Boundary
Any agent that loads tools it does not own, whether through MCP or another plugin mechanism.
Symptom or risk
A description edited after approval adds an instruction or a new parameter, and the model follows it because tool text reads like documentation.
Exceptions
Tools compiled into your own service and deployed through your normal review.
Trade-offs
Pinning delays picking up legitimate improvements and adds a review step per update.
Pitfalls
Approving the server once and never re-checking after it announces a tool-list change.
Lessons and concepts
Claims and their sources
The MCP 2025-06-18 tools specification says clients MUST treat tool annotations as untrusted unless from trusted servers, defines a list-changed notification, and says clients SHOULD show tool inputs to the user before calling the server to avoid data exfiltration.