LLMs, GenAI, RAG, agents, evaluation · insight-0636
Return structured, classified tool errors so the agent can recover or stop
- Evidence type
- practitioner heuristic
- Difficulty
- easy
- Roles
- ML engineer, Applied AI
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
Tool failures reach the model as raw stack traces or a generic "error occurred".
Action
Return errors in a fixed shape (type: transient, invalid_input, not_permitted or not_found; a short message; whether retrying can help) and let code, not the model, enforce the retry limit.
Reason
Retrying is only useful for transient failures; Google's SRE guidance separates retriable from permanent errors and warns against unbounded retries.
Boundary
Agents calling internal APIs, databases and third-party services.
Symptom or risk
The agent retries permanent errors forever, gives up on transient ones, or invents a result to keep going.
Exceptions
None worth making; even prototypes benefit from a consistent error shape.
Trade-offs
Classifying errors well requires knowing each downstream service's failure modes.
Pitfalls
Leaking internal details (hostnames, SQL, credentials) in error text that the model may repeat to users.
Lessons and concepts
Claims and their sources
Google's SRE guidance advises not retrying permanent errors or malformed requests and limiting retries per request.