Skip to main content
A live stream lets the reader see an answer while the provider is still generating it. It also creates a difficult boundary: once part of the answer may be visible, Gateway cannot undo that exposure. Consider an illustrative stream that produces three text fragments and then loses its provider connection. The correct ending is an explicitly incomplete result—not a second model quietly completing the sentence.

​
First, establish genuine streaming support

The exact approved offering must support the required streaming behavior. General adapter tests are useful, but they do not establish the capability of every model deployment, account or protocol combination. The original call and its permitted attempt are then recorded under the same evidence and capacity rules as a buffered request. No provider task starts merely because the application asked for a stream.

​
Read native events as they arrive

The adapter incrementally parses the provider’s events across network chunks. Network chunk boundaries do not necessarily match text characters or complete protocol events. Gateway distinguishes meaningful answer text, tool arguments and refusal from liveness, usage and reasoning-size observations. A heartbeat generated by Gateway is not provider progress. An end-of-file without the required successful terminal evidence is not a normal successful stop. Collecting a complete answer first and replaying it as chunks later does not satisfy a promise of genuine live streaming.

​
Record the exposure boundary before showing text

The output marker records possible exposure, not proof that the user read the fragment. This conservative rule matters if delivery acknowledgement is lost immediately afterward. Once that boundary is crossed, Gateway must not replace the answer with a different provider’s output under the same call. It can finish the same attempt or stop truthfully.

​
A slow reader needs its own control

The stream relay has both event-count and byte limits. When the caller stops reading, the relay applies backpressure instead of allowing memory to grow indefinitely. During that induced pause, provider-idle clocks pause where specified; a separate downstream-stall clock measures the caller’s inability to drain. The overall logical-call lifetime does not pause or restart. Provider completion does not end delivery responsibility. A full caller queue can still stall after the provider finishes, so delivery timing and owner cancellation remain active through the final drain.

​
Cancellation follows the real owner

An HTTP stream may own its call. A browser observing a durable task may not. The applicable ownership contract decides whether disconnection cancels the underlying computation. Closing local transport also does not prove the provider stopped computing or charging. Gateway retains provider certainty, delivery state and usage evidence separately.

​
What the available tests establish

The recorded local streaming experiment demonstrated incremental functionality with a fake provider and real protected database behavior. Its bounded-lifecycle qualification remained partial because protected database waits were not fully bounded. It was also separate from the caller-integration branch. Those limitations are detailed in Implementation status; this journey does not claim an already-qualified live production path. Continue with The output boundary and Cancellation and slow readers.