Skip to main content
A provider can produce text promptly while the application stops reading it. If Gateway keeps buffering everything, memory grows. If it blames the resulting silence on the provider, it misdiagnoses the problem. The selected design uses bounded buffers, backpressure and explicit ownership to handle this situation.

​
Backpressure means slowing the producer-facing path

When the caller-facing relay cannot drain, Gateway stops pulling more events through the blocked path. That pressure keeps an unlimited backlog from accumulating. An event-count limit is not enough. A queue with one item can still hold an enormous string or tool argument. The parser, individual events, assembled response, formatter and caller queue each need the applicable byte bounds. Text can be split at valid character boundaries where the contract permits. Tool identity or other indivisible metadata cannot be truncated into a different value simply to fit a buffer.

​
Measure the right source of delay

During a relay-induced block, the applicable provider-idle clocks pause. A separate downstream-stall clock measures the caller’s failure to drain. The logical-call lifetime does not pause or restart. The system therefore cannot keep a call alive forever by alternating between buffering and waiting. After the provider terminates, its upstream clocks end, but delivery still has an owner. Downstream-stall timing and cancellation remain active until the final drain finishes or stops.

​
Cancellation belongs to a computation owner

There is no universal rule that “a disconnected browser cancels everything.” A browser may merely observe a durable task whose owner still holds a mandate, budget and result destination. In other profiles, the response stream owns the computation. For an HTTP event stream, that ownership must live with the response body rather than disappear when the handler returns response headers. For an in-process incremental call, events and completion can have separate receivers. Dropping the events receiver alone may leave a valid bounded completion path. Dropping both before termination ends that ownership and cancels.

​
A local stop does not prove a remote stop

Gateway must propagate a legitimate cancellation to its owned tasks and transport. It must not leave an untracked provider task running after its call owner is gone. However, closing local transport does not prove that the remote provider stopped computing or charging. Provider outcome, delivery failure and usage certainty remain separate facts. If a caller stalls while the provider is unfinished, the result may contain a known delivery failure and an unknown provider outcome. It must not invent a completed provider result merely to close the local response.

​
One remaining implementation concern

The dated local streaming work did not fully bound protected database lock and commit-observation waits. That gap does not justify unbounded queues, but it does limit a claim that the complete runtime lifecycle is already bounded. The required fix must preserve commit uncertainty while giving unresolved work a bounded observation process and an explicit owner. A cancellation timer alone is not proof that a database rolled back. Continue with Understanding timeouts and When a confirmation is lost.