Skip to the content.

Token streaming TTFT with whole-turn tool gating (2026-06-25)

Issue #47 changes the streaming contract from “finish the turn, adjudicate it, then synthesize SSE” to “stream safe prose while the turn is still decoding, but hold every tool-call byte until the kernel has seen the complete proposed call set.” The split is intentional: prose deltas are ordinary assistant text, while tool_calls / tool_use.input bytes are executable intent and stay behind k.Decide.

Wire behavior

Both live paths use the lift guard before writing prose. If a model emits a known text-form tool-call dialect inside content, the guard withholds that span, the normalizer lifts it into a structured call, and the call is adjudicated before any surviving structured tool block is written.

Perceived TTFT

For prose-first turns, perceived TTFT now tracks the upstream model’s first content fragment plus gateway framing overhead. It no longer waits for the full turn and the tool-call gate.

For tool-heavy turns, perceived TTFT depends on whether the model emits leading prose:

This preserves the trust floor while making TTFT/TPOT/ITL measurable for the parts of a turn that are safe to expose incrementally.