Request deadlines: calculate the retry budget

Two attempts fit this request, not three

A request with a 1,000 ms overall deadline, 230 ms already spent and an 80 ms return reserve has 690 ms available for a downstream operation. Under the sample policy, two 150 ms attempt allowances plus a 100 ms backoff consume 400 ms. A third attempt would add a 200 ms wait and another 150 ms allowance, bringing the plan to 750 ms. It does not fit.

These are calculated scenario values. They are not measured service latency, a recommended universal timeout or a prediction of request success. The purpose of the model is to make a retry decision inspectable: elapsed work, backoff and the time needed to return a response all count against the same user-visible deadline.

A configured maximum of three attempts is therefore only a ceiling. It is not permission to perform three attempts after the request has already consumed most of its time elsewhere.

Define the accounting before choosing a timeout

The planner subtracts elapsed time and a return reserve from the original request allowance. It admits an attempt only when its full allowance and preceding backoff fit the remaining budget. Each attempt is assumed to consume its complete allowance, and all attempts are assumed to fail until the next one is considered. This is conservative scheduling arithmetic, not a simulation of a latency distribution.

available = max(0, overall_deadline - elapsed - return_reserve)
next_cost = preceding_backoff + full_attempt_allowance
admit only if used + next_cost <= available

The return reserve represents work that still must happen after the dependency call, such as assembling the response and sending it back. Its correct value depends on the application. The model uses 80 ms to expose that category of cost. It does not establish that 80 ms is sufficient on a real network.

Include connection establishment, queueing and response reading in the attempt boundary you measure. A "150 ms timeout" that starts only after connection acquisition does not bound a call that spends another unbounded interval waiting for a connection. Name the boundary in telemetry and in the client configuration review.

The gRPC deadline guide distinguishes a deadline from a duration timeout and describes propagating remaining time to downstream calls. It also explains that application code must stop work it has spawned when cancellation occurs. Verify the behavior of the actual language implementation instead of assuming every client propagates deadlines automatically.

Compare requests that arrive at the dependency later

The following sweep keeps the overall deadline, return reserve, attempt allowance and backoffs fixed. Only the time already spent before the dependency changes. "Attempts admitted" counts the initial attempt as well as retries.

Unused time is not evidence of inefficiency. Under this full-attempt policy, 120 ms cannot fund a 150 ms attempt. A different policy could allow a shortened final attempt, but it would need evidence that such an attempt is useful and that the receiver respects the tighter timeout. The example deliberately keeps that alternative outside the planner so its behavior remains easy to verify.

The discontinuities matter when reading latency dashboards. A modest increase in upstream queueing can remove an entire retry opportunity even if the dependency's own latency distribution has not changed. Looking only at the dependency timeout can miss why the same request type now fails more often near its end-to-end deadline.

Check sensitivity to the attempt allowance

At the same 230 ms elapsed point, the available dependency budget remains 690 ms. Changing the attempt allowance changes how much work fits:

Three short attempts are not necessarily better than one longer attempt. If useful responses usually need more than the short allowance, aggressive timeouts can repeatedly abandon work that was close to finishing. Conversely, a long attempt can consume the time needed for a useful retry elsewhere. This model has no response-time samples or success probabilities, so it cannot choose the optimal allowance.

Download the deadline planner, JSON results and elapsed-time sweep CSV. Save the program as budget.py and run python3 budget.py in an empty directory. It uses standard-library arithmetic and writes the result files without making network calls. The JSON includes the full sensitivity sweep and each admitted step.

Put a traffic budget beside the time budget

A deadline limits how long one caller is willing to wait. It does not independently limit how much retry traffic a service receives. Many callers can each have time for another attempt while collectively overloading the same dependency.

Google's SRE guidance on handling overload describes per-request and per-client retry controls and warns about retries at multiple layers. Treat those as separate decisions: which layer owns retries, which failures qualify and how much extra traffic is permitted during widespread failure. Its published settings describe Google's system, not defaults for this sample.

For an illustrative stack with three retrying layers, each allowing three attempts for every failed downstream call, the leaf can receive up to 27 attempts for one original request. That is the product 3 × 3 × 3 under a worst-case retry expansion, not an observed rate and not a claim that every timeout produces that load. A shared remaining deadline may constrain the expansion, but it should not be the only protection against it.

Use an explicit retry owner and propagate an outcome that tells upstream callers when another retry would be inappropriate. A traffic budget, admission control and jitter address aggregate load. They do not replace a time budget. The rate-limit model provides a separate way to reason about accepted traffic.

Carry the decision into the running service

At runtime, recompute remaining time before each wait and each attempt. Earlier attempts may finish before their allowance or spend time outside the measured call boundary. Backoff must remain cancellable. Reject a retry when the operation is unsafe to repeat, the failure is nonretryable, the traffic budget is exhausted or the remaining deadline cannot fund useful work.

Stopping the client wait also does not prove the server stopped executing. Propagate cancellation where supported, and make the worker observe it at meaningful boundaries. For a mutation with an uncertain outcome, use the application's idempotency and reconciliation contract rather than assuming a timeout means nothing happened.

Before changing production settings, collect end-to-end traces that separate connection acquisition, processing and return work. Compare initial attempts with retries and inspect how much deadline remained when each began. The sample's next useful input is that measured distribution. Until it exists, the planner can reject an impossible schedule, but it cannot promise a better success rate.

Sources

Documentation checked September 5, 2026.

1. gRPC: deadlines and propagation
2. Google SRE: handling overload and retry controls