Skip to main content
One application request can require more than one provider attempt. Gateway keeps those attempts inside one original logical call, so recovery cannot hide extra sends, change the required result or reset the limits. The first distinction is simple: a candidate considered during planning is not an attempt. A permitted physical provider dispatch is an attempt and needs its own identity and record.

​
An error name is not enough to choose a retry

A transport failure says something went wrong with communication. It does not, by itself, establish whether the provider performed the work. Gateway therefore keeps the failure kind separate from outcome certainty. It also considers the affected scope, the original plan, whether output may already be visible, who still owns execution and which limits remain. A request-local error must not automatically become evidence that an entire provider is unhealthy. A compliance signal must not be treated as a reason to rotate accounts and bypass the restriction.

​
Four situations that look similar but are not

For a same-target retry after a transport failure, the accepted rule requires known certainty. “Unknown” is not a blanket permission to try again, but neither does it forbid every policy-permitted live alternative before output.

​
One budget, not one budget per layer

The accepted baseline has a ceiling of six physical dispatches in total, including the first. It also limits same-target retries, combined provider/model hops and the one-use timeout alternative. These limits are ceilings, not a target to exhaust; exact pins and purpose policies can make the permitted set smaller. The baseline’s two same-target retries are call-wide. Changing provider does not renew them. The logical generation lifetime starts at the first provider dispatch and is shared across attempts and waits. An SDK, HTTP wrapper or application retry loop must not create additional invisible sends. Every permitted physical send belongs to the same accounting and control envelope.

​
Waiting must fit the original limits

A provider may supply a retry-after or reset time. Gateway cannot shorten that delay merely to squeeze another attempt into its remaining lifetime. Waiting must satisfy both the applicable pushback rule and the time remaining. The selected recovery policy also keeps known, estimated and unresolved prior consumption in view. An uncertain attempt does not become free capacity because a later attempt succeeds.

​
Stop conditions remain stop conditions

Output commitment, cancellation, lifetime exhaustion, downstream stall, response-size limits, internal faults, refusal and compliance have source-defined stop behavior. Recovery is not a competition to obtain some answer at any cost. The pure recovery decision has an accepted reason order, while current execution ownership and evidence are checked before that decision. Health observations remain separate; the order is not a new public error-priority API.

​
Deliberately asking again is a different decision

An authorized application may initiate another inference as new work. It must keep a distinct durable identity, respect its aggregate mandate and retain the earlier call’s unresolved obligations. It must not disguise that choice as replay of the same completed request. Continue with The output boundary and Restarting without repeating the work.