Skip to main content
Connecting an application to an AI provider solves one problem: sending a request and receiving a response. It does not settle who may use a model, what should happen after a timeout or how to record work that may have happened even when no answer arrives. Gateway exists to handle those execution responsibilities consistently, instead of leaving every application to solve them independently.

​
The problem with separate execution rules

Imagine two applications using the same provider. One retries a failed request immediately. The other waits and changes provider. A client library may also retry underneath them. To the application, each interaction looks like one question. To the providers, it may be several billable requests. Now add a request that needs both reasoning and a schema-valid answer. A model may support each feature separately but not the combination. A provider may accept the request yet use an output limit differently from what the application expected. These are illustrative situations, not reports of new incidents. They show why sharing a provider adapter is not enough: the systems also need to share the rules that decide what may be sent and how the result is interpreted.

​
What Gateway brings together

Gateway’s selected design gives one execution core responsibility for planning a compatible call, counting permitted provider attempts, controlling recovery and recording the resulting evidence. The core first uses approved information about model offerings. It does not guess a missing capability because a provider name looks familiar. It then applies the caller’s requirements and restrictions. An unsuitable request is refused before provider work begins rather than silently weakened to make it run. When execution is permitted, every physical send belongs to the original request and its remaining budget. A change of provider does not create a fresh allowance for retries. The execution record is established before effects are allowed to get ahead of it.

​
Why failures need more than an error message

A lost connection can mean several things. The request may never have reached the provider. It may still be running. It may have completed, with only the response lost. Gateway must preserve those differences. It cannot turn “we do not know” into “nothing happened,” nor assume that absent usage data means zero cost. There is another boundary once answer text may have reached the reader. If a stream breaks midway, Gateway must not quietly ask a different model to finish that answer. It reports the incomplete result instead of presenting two separate generations as one coherent response.

​
What stays with the application

The application decides why it needs the answer and whether the answer is useful. It stores the answer when it promises that the exact result can be retrieved later. A business component decides whether work is accepted; financial components decide whether an authorized payment actually committed. Gateway therefore supports many kinds of callers without becoming their workflow manager. A normal authenticated application does not need to become a professional-work system just to request model output.

​
The benefit—and the cost

The benefit is a more explainable execution path: one original request, an approved plan, visible attempt history and outcomes that retain uncertainty where necessary. The cost is real. Durable records and current permissions become execution dependencies. Some requests must stop rather than use an unverified model or an unsafe retry. That is a deliberate trade-off, not an accidental limitation to hide. Continue with How Gateway works for the components, or Your first request for a concrete walkthrough.