LLMs, GenAI, RAG, agents, evaluation · insight-0661
Branch error handling on the error type, because two 429s can need opposite responses
- Evidence type
- documented fact
- Difficulty
- easy
- Roles
- ML engineer, Applied AI
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
All non-200 responses from the model API go through one generic "wait and retry" handler.
Action
Map error types to actions: rate limit with retry-after, back off and retry; spend cap without retry-after, alert and degrade; overloaded, back off or fail over; invalid request, fix and do not retry; log the request id each time.
Reason
Anthropic documents a rate-limit 429 that carries retry-after, a spend-cap 429 that does not and keeps failing until access resumes, a separate 529 overloaded error, and a request id on every response for debugging.
Boundary
Every integration with a hosted model API; other providers have their own taxonomies.
Symptom or risk
A spend-cap rejection is retried for hours, malformed requests are retried forever, and overload errors trigger a stampede.
Exceptions
None; even prototypes save debugging time with typed handling.
Trade-offs
Provider-specific handling must be updated as error types evolve.
Pitfalls
Parsing human-readable error messages instead of typed error fields or SDK exception classes.
Lessons and concepts
Claims and their sources
Anthropic's rate-limit 429 includes a retry-after header, while a monthly spend-cap 429 has none and keeps failing until access resumes.
Anthropic's API distinguishes error types such as invalid_request_error (400), rate_limit_error (429), api_error (500) and overloaded_error (529), and returns a request id with every response.