Skip to main content
Some model offerings use generation capacity for both internal reasoning and the visible answer. A limit that looks large enough for an answer can still be too small once reasoning is included. Gateway must plan with the offering’s actual accounting rules. A generic max_tokens value is not an explanation of those rules. Tokens are the units used to count model input or generation; the relevant offering defines which kinds of tokens each limit includes.

​
A worked example

The following numbers are illustrative planning inputs, not production defaults or a provider benchmark. The application requires room for 2,000 visible answer tokens. The selected offering has a verified reasoning bound of 6,000 tokens. Its native output ceiling counts reasoning and visible output together. The required combined allowance is therefore:
An offering with a compatible 8,000-token ceiling can satisfy this capacity calculation, subject to the other request checks. A 7,000-token ceiling cannot. Gateway must not quietly reduce the answer allowance or switch reasoning off to make the request fit. This calculation allows capacity; it does not promise that the model will produce exactly 2,000 useful visible tokens.

​
Different controls need different evidence

Reasoning may be controlled by an on/off setting, a token budget, an effort level or an adaptive behavior. Those are not automatically interchangeable. An effort label needs an accepted mapping or bound for the relevant offering. An always-on or adaptive mechanism still needs the evidence required by the planner. A missing mapping must not be converted into a guessed reasoning allocation. Optional reasoning can be omitted only where the caller’s policy permits that choice. Missing evidence for mandatory reasoning is an incompatibility, not an invitation to use zero.

​
Check the context window too

Input and output limits also depend on what the offering counts. If its context window includes input plus generation, the input size and the combined generation allowance must fit together. If it counts only input in that window, the calculation is different. The planner must use the verified rule, including checked arithmetic. It cannot repair an oversized governed request by trimming sealed content after the caller approved it.

​
Progress is not answer text

During execution, the protocol may establish that reasoning is progressing without exposing the reasoning content. The accepted event model can report reasoning size or progress; it does not publish private reasoning text as user output. A heartbeat from Gateway’s edge is weaker still: it says the connection is alive, not that the provider is thinking or producing an answer.

​
Planned capacity is not observed usage

An allowance of 6,000 reasoning tokens is not proof that 6,000 were consumed. The provider may report actual usage, omit that dimension or report a value whose accounting requires interpretation. Gateway retains those evidence states separately. A reported zero is different from a missing count. If the model exhausts its output limit before producing a useful answer, that outcome must remain visible; it is not a normal successful completion. Read Usage, provider cost and customer payment for the accounting consequences, and Understanding timeouts for progress and lifetime controls.