Exponential backoff

A retry is the most reasonable-looking line of code in any client. The call failed, the failure was probably transient, so try again — and every part of that reasoning is sound in isolation. What makes retries dangerous is that they are correlated: they fire when something is wrong, which means they arrive together, at the worst possible moment, from everywhere at once.

Retries multiply, they do not add#

Google’s SRE book walks the loop directly. A backend reaches capacity and begins rejecting requests; the rejections prompt the frontend to retry; the retries are additional load. The volume grows — its own numbers are 100 queries per second of retries in the first second, then 200, then 300. The service is being asked to do more work as a direct consequence of being unable to do the original work, and nothing in that loop is self-limiting.

The part people underestimate is what happens when retries exist at more than one level, which they usually do because different teams added them at different times. The SRE book gives the arithmetic: with a client, a frontend and a backend each retrying three times, a single user action can produce 64 attempts at the lowest layer. Three independently sensible policies compose into one that nobody would have written down.

That multiplication is the argument for deciding where retries live rather than letting each layer decide for itself. A layer that retries needs to know whether the layer beneath it already did.

Backoff is half of it#

Try it: spread the retry crowd

These clients all fail at time zero. Compare fixed waits of 1, 2 and 4 seconds with one illustrative set of randomised waits. Each row is one client; each dot is a retry.

Clients

4

Wait policy Backoff onlyFull jitter example

0 s2 s4 s6 s8 s

All 4 clients retry together at 1, 3 and 7 seconds. Longer waits alone leave the crowd synchronised. Enable JavaScript to change the policy.

A fixed illustrative schedule, with three retries per client. This does not simulate server recovery, retry budgets, network delays or fresh random samples.

Exponential backoff addresses the first problem: rather than retrying immediately and repeatedly, each attempt waits longer than the last. One client’s attempts spread out, the pressure it applies falls away quickly, and a service that needs a few seconds to recover gets them.

What backoff does not fix is that the clients are synchronised. A thousand clients failing at the same instant, all doubling their waits, all come back at the same instant — a second later, then two, then four. The load is spread over time and remains concentrated in spikes, and each spike is capable of knocking over a service that had just started to recover. Which is why the SRE book’s guidance is to always use randomized exponential backoff. The randomisation is not a refinement on the backoff; it is the half that breaks the correlation between clients, and backoff without it produces a slower thundering herd rather than no herd.

Which raises the question the advice usually leaves hanging: randomise how? AWS’s published comparison names three variants and simulates them against each other, and they differ in what they keep of the backoff. Full jitter keeps the exponential upper bound, but none of it as a mandatory minimum wait: the sleep is a random value drawn between zero and the computed backoff, written in the article as sleep = random_between(0, min(cap, base * 2 ** attempt)). Equal jitter keeps half — half the computed backoff, held fixed, plus a random amount up to the other half — on the reasoning that a client should not be able to retry almost immediately after a long wait. Decorrelated jitter abandons the attempt counter and derives each wait from the previous one instead, allowing its range to grow or shrink with the previous random wait rather than following a fixed attempt curve.

The simulation is what makes the choice tractable, and it does not crown a single winner. Full and equal jitter came out close on total work, with decorrelated doing more of it; equal jitter was the slowest to complete, and full jitter did less work than decorrelated at a small cost in time. All three cut the work sharply against backoff with no jitter at all, which is the finding that matters: the gap between some randomisation and none is far larger than the gap between the three ways of doing it. Pick full jitter unless you have a reason not to, and treat the choice between them as a tuning question rather than the decision.

What a budget is for#

Backoff and jitter shape retries; they do not cap them. The SRE book adds two limits that do. A per-request cap stops one call retrying indefinitely, and a process-wide retry budget — its example is allowing only 60 retries per minute in a process — bounds the total regardless of how many individual calls are failing. The budget is the one that saves you during a broad outage, because that is exactly when every request is failing and every per-request cap is being honoured.

Do not repeat a request known to be permanently invalid. Correct malformed input or obtain an authorised credential rather than sending the same failed request again. A missing resource can have different meanings in different services; check the error contract instead of treating every missing-resource response as transient or permanent. Google’s retry guidance distinguishes permanent errors from retryable ones and makes the retry budget a reason to stop.

Backoff and jitter pace attempts; a retry cap or exhausted budget stops further retries. A circuit breaker can also reject calls before they reach a failing dependency. Resilience4j’s documented model has CLOSED, OPEN and HALF_OPEN states. Closed records calls normally. After the configured minimum number of calls, a failure or slow-call rate at or above its threshold opens the breaker; open rejects calls immediately. A breaker does not replace retry limits, and the outcome depends on which failures its configuration records.

After its open-state wait, Resilience4j permits a configured number of half-open trial calls. By default, a subsequent call triggers that transition; automatic timer-driven transition is a separate option. At or above the configured failure or slow-call threshold, the trial returns it to open; below both thresholds it can close. This cap belongs to one breaker instance. Many independent clients can still probe together, so it does not provide a fleet-wide recovery guarantee.

Two things elsewhere in the system finish the design. The server needs a way to say not now and be understood, which is what a rate limiter’s response is for — and a client that honours the wait it is given beats any backoff it invents locally. And because a retry that succeeds may be a duplicate of one that also succeeded, the receiving end has to be built so a repeated request is a routine case rather than a second one. Retry policy is a property of the pair, not of the caller.

Questions people also ask

4 QUESTIONS

▸ Why do retries make an overloaded service worse?

Because the failure is what triggers the extra load. Google's SRE book traces the loop: a backend at capacity starts rejecting, the rejections prompt retries, and the volume grows — 100 queries per second of retries in the first second leads to 200, then 300. The system is now being asked to do more work precisely because it could not do the original work.

▸ How bad does retry amplification get across layers?

Multiplicative, not additive. The SRE book works the example: if a client, a frontend and a backend each retry three times independently, one user action can become 64 attempts at the lowest layer. Every layer that retries multiplies the layer below it, which is why retry logic added independently by three teams is so much worse than any of them intended.

▸ Is exponential backoff enough on its own?

No, because it spaces out one client's attempts without spreading out the crowd. A thousand clients that all failed at the same moment and all double their wait will all return at the same moment, repeatedly. The SRE book's guidance is to always use randomized exponential backoff — the randomisation is what breaks the synchronisation.

▸ Which errors should not be retried?

Do not repeat a request that is known to be permanently invalid. A malformed request needs correction; a revoked credential needs an authorised replacement. A missing resource may be permanent or temporarily unavailable, so follow the service's documented error contract. Retry only when another attempt can help and repeating the operation is safe.