Skip to main content
Gateway sits between an application that needs an AI result and a provider that performs the model work. The application states what it needs. Gateway checks whether that request can be executed under the applicable rules, then manages the permitted attempts and their evidence. The components below describe the selected architecture. They are responsibilities, not necessarily separate network services.

​
1. The entry point establishes the request

An HTTP endpoint or an in-process host adapter translates the caller’s input into a common request. The trusted entry boundary establishes the caller’s identity and permitted scope; a tenant or purpose string inside an untrusted request cannot grant itself permission. Different entry formats can remain different. What should not differ is the underlying rule for planning, retrying or recording one model call.

​
2. The planner chooses a compatible execution

The planner combines the request’s requirements with approved information about available model offerings. That information comes from Registry, the component responsible for publishing the approved catalogue and policy. The planner checks the exact combination: for example, the required output format, reasoning behavior, streaming support and allowed destination. It produces a resolved execution plan, called an EffectiveCall, or explains why no candidate is eligible. It does not contact a provider while making this decision. An adapter later translates the already-decided parameters into the provider’s protocol; it must not change a required feature to make serialization easier.

​
3. The execution core owns the attempts

One logical call represents the application’s original inference request. A physical attempt represents one permitted provider dispatch. Several attempts may belong to one call, but they share its plan, limits and history. Before a provider request is sent, the required attempt record and capacity reservation must be committed together and confirmed. Provider network work then happens outside that database transaction. The arrows show the ordinary direction of work. Failure and confirmation checks can stop the path; this is not a diagram of unconditional success.

​
4. Delivery follows the chosen mode

For a buffered answer, successful content is withheld until the required final record is confirmed. For a validated document, the complete result must also pass its output contract. For a live stream, Gateway records that meaningful output may be exposed before releasing the first text, tool argument or refusal. After that boundary, it cannot replace the response with another model’s answer under the same call.

​
5. Records and feedback have different jobs

O1 is the design’s protected, content-free execution record. It links calls, attempts, outcomes and settlement notifications. Native G4 records the provider-capacity effects that must be coupled to those transitions. Neither is an answer archive or a customer wallet. R9 is the provider-observation reporting path. It reports events such as rate limiting or a provider-declared retirement notice. It does not independently approve a model capability, and a slow reporting sink must not be confused with failure of mandatory execution persistence. The application stores answer content when needed. Financial owners process their own effects after the relevant notification; an execution final is not a payment receipt. Continue with Who owns what or follow a complete text request.