7
Backend
Retry or fail fast for user-facing API calls with a 1-second budget
Resilience ↔ Resource protection
Context
Call the inventory service from the API gateway within a 1 s request budget.
Our Go API gateway calls an inventory service and has a 1-second budget for the whole user request.
Option A: retry failed calls with exponential backoff and jitter.
Option B: fail fast: one attempt with a 300 ms timeout, return a degraded response, let the client decide.
At 5K req/s, retries can double load at exactly the wrong moment. But failing fast drops requests that would have succeeded a moment later.
Constraints
- Budget
- 1 s for the whole user request
- Calls
- Mostly idempotent reads; some writes
- Traffic
- 5K requests/sec at peak
Retry with backoff and jittervsFail fast (one attempt, tight timeout)
Community verdict
Retry with backoff and jitter 44%Fail fast (one attempt, tight timeout) 56%
9 engineers · 5 opinions
With these constraints, what would you choose?
One choice per engineer. You can change it any time.
Retry with backoff and jitter
Fail fast (one attempt, tight timeout)
Resource protection →
Trade-offs
Dimensions
Retry with backoff and jitterFail fast (one attempt, tight timeout)out of 5
- Success on transient faults
- 5Retry with backoff and jitter scores 5 of 52Fail fast (one attempt, tight timeout) scores 2 of 5
- Resource protection
- 2Retry with backoff and jitter scores 2 of 55Fail fast (one attempt, tight timeout) scores 5 of 5
- Tail latency
Community discussion
5 comments
Have you made this decision in production? Share your reasoning.
Sign in to commentA fast, honest error with a "try again" button beats a spinner that times out after 3 seconds.
In a fan-out, retries at every layer multiply: three layers retrying 3 times each is 27 attempts. Retry at one layer only, ideally the edge, and fail fast everywhere else.
We banned retries in internal libraries for this reason. Only the gateway retries.
Retry, but budgeted: pass the request deadline down (Go's context does this well) and only retry if there is time left. Add a retry budget, e.g. retries may be at most 10% of requests, so a real outage can't multiply load.
Idempotent reads are safe to retry. Writes need idempotency keys first.
Retry budgets are the piece I was missing. That caps the amplification.