LLMs, GenAI, RAG, agents, evaluation · insight-1551
Multiply per-step reliability before promising an end-to-end agent success rate
- Evidence type
- practitioner heuristic
- Difficulty
- easy
- Roles
- Applied AI, Data scientist, ML engineer
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
A team demos an agent whose individual steps look reliable and plans to promise a task success rate to the business.
Action
Estimate per-step success from traces, multiply along the planned path, then shorten the path, add verification gates or cut optional steps until the product meets the target you will commit to.
Reason
Probabilities of independent steps multiply, and vendor guidance on agents names compounding errors as a risk of autonomy.
Boundary
Multi-step agents with sequential tool calls where one wrong step spoils the result.
Symptom or risk
A long chain of independently fallible steps succeeds far less often than any single step, so the promised rate is missed in production.
Exceptions
Steps that recover from earlier mistakes, or are strongly correlated, break the independence assumption; measure the real end-to-end rate.
Trade-offs
Gates and shorter paths lower flexibility and add latency.
Pitfalls
Quoting the per-step accuracy as if it were the task accuracy.
Example
Hypothetical figures. Ten steps at 95 percent each give 0.95^10, about 60 percent, and at 99 percent each about 90 percent.
Lessons and concepts
Claims and their sources
Anthropic's guidance on agents notes that their autonomy brings higher cost and the potential for compounding errors, and recommends sandboxed testing and guardrails.