Engineering 8 min read· 3 Aug 2026· By Engineering
Error handling patterns for LLM APIs
The five error classes you'll see in production and what to do about each.
LLM APIs fail in more interesting ways than a typical REST service. Classify the error, then respond.
The five classes#
- 4xx auth — bad key. Do not retry.
- 402 wallet — top up.
- 429 rate limit — exponential backoff with jitter.
- 5xx upstream — retry once, then fallback model.
- 200 but bad JSON — retry with
response_format: json_schema.
Never blanket-retry every exception — you'll accidentally double-bill successful requests and slow down real failures.