LLMs, GenAI, RAG, agents, evaluation · insight-1570
Put a circuit breaker on each tool so a failing dependency stops being called
- Evidence type
- documented fact
- Difficulty
- medium
- Roles
- ML engineer
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
A pricing tool starts returning errors, and the agent keeps retrying it across many steps and many concurrent runs.
Action
Wrap each tool with a breaker that opens after repeated failures, returns an immediate unavailable result with a retry time, lets a few trial calls through when half-open, and tells the agent to use an alternative or stop.
Reason
The circuit breaker pattern prevents repeating operations likely to fail, fails fast, and probes for recovery, which differs from retrying in hope of eventual success.
Boundary
Agents calling shared internal or third-party services.
Symptom or risk
Runs waste their budgets, the failing service is hammered further, and users see long delays before an error.
Exceptions
One-off batch jobs with a single caller may only need retries with a limit.
Trade-offs
Thresholds need tuning; an open breaker can block calls that would have worked.
Pitfalls
One breaker covering several independent providers behind the same tool.
Lessons and concepts
Claims and their sources
Microsoft's circuit breaker pattern blocks calls after failures reach a threshold, with closed, open and half-open states, and recommends separate breakers when independent providers sit behind one resource.