Streaming

Streaming is capability-driven. Providers that implement the streaming methods and return supports_streaming() == true can emit token deltas; other providers use the non-streaming response path. The runtime forwards available streams to channel adapters that support partial updates.

What gets streamed

The provider trait emits StreamEvent values as the model generates output: text deltas, structured tool calls, provider-side pre-executed tool calls and their results, token-usage reports, and a final completion marker. The authoritative, per-variant definitions live with the type in crates/zeroclaw-api/src/model_provider.rs (enum StreamEvent); reasoning tokens arrive as text deltas, not a separate variant.

The runtime consumes these events. The channel orchestrator uses the Channel trait’s draft-delivery methods and capability flags to surface progressive output where supported.

Capability flags

A provider exposes two flags so the runtime knows what it can expect:

#![allow(unused)]
fn main() {
fn supports_streaming(&self) -> bool { false }
fn supports_streaming_tool_events(&self) -> bool { false }
}

• supports_streaming: true only when the concrete provider opts into streaming; the trait default is false
• supports_streaming_tool_events: true when the provider emits ToolCall events during the stream rather than at the end

OpenAI-compatible providers differ: some stream tool-call arg deltas chunk-by-chunk, others only emit the call once complete. The compatible.rs SSE parser handles both.

Channel-side streaming

Channels advertise their own streaming capabilities through the Channel trait:

#![allow(unused)]
fn main() {
fn supports_draft_updates(&self) -> bool;           // edit a message in place
fn supports_multi_message_streaming(&self) -> bool; // split one reply into many messages
}

A channel’s capability follows from its config: a channel with the stream_mode enum (off / partial / multi_message) supports both draft updates and multi-message; a channel with the stream_drafts boolean supports draft updates only. This table is generated from the channel config schema, so it stays correct as channels gain or lose streaming support:

When both the provider and the channel support streaming, the flow is: provider emits TextDelta → runtime passes to channel → channel edits the sent message. The edit cadence is bounded by that channel’s draft_update_interval_ms setting to avoid rate-limiting; defaults vary by channel.

Reasoning blocks

StreamEvent has no separate ReasoningDelta variant. When a provider exposes reasoning during streaming, it uses the reasoning field on the StreamChunk carried by TextDelta; provider and runtime configuration determine whether that content is requested or surfaced. Consumers should follow the StreamChunk contract rather than matching a nonexistent event variant.

Tool calls mid-stream

When a streaming provider decides to call a tool, it emits a structured ToolCall stream event. The runtime:

1. Reads the stream to completion, collecting structured ToolCall events and forwarding visible text until Final
2. Recovers the tool calls after the stream ends
3. Runs the tools (subject to security validation, see Security → Overview)
4. Opens a new streaming call to the provider for the next assistant turn, with the tool results appended to the conversation

The current provider stream is never paused and resumed mid-read; tool execution happens after that stream reaches Final, and the next turn is a fresh streaming call.

From the user’s perspective: text, then a visible indicator that the agent ran a tool (via channel-specific hints), then more text. For channels without typing indicators, the gap between the tool call and the next text chunk is the only signal.

Transport completion and timeouts

Streaming transports do not rely on connection close as the success signal. OpenAI-compatible streams finish on [DONE], OpenAI Responses streams finish on their terminal response event, and Anthropic streams finish on message_stop. Servers may keep the HTTP connection open after those events.

Streaming clients use byte-idle timeouts: 300 seconds for OpenAI Responses and OpenAI-compatible providers, and 90 seconds for Anthropic. Each received body read resets the relevant timeout, so active generations are not constrained by the whole-request timeout used for non-streaming calls. Connection setup, response headers, and buffered error bodies remain bounded.

Non-streaming providers

When supports_streaming() is false, callers use the provider’s non-streaming chat path. Channel adapters can still send the completed reply, but they do not receive incremental provider stream events.

Code references

• crates/zeroclaw-api/src/model_provider.rs: ModelProvider trait, StreamEvent enum
• crates/zeroclaw-providers/src/compatible.rs: OpenAI-compat SSE parser
• crates/zeroclaw-providers/src/anthropic.rs: Anthropic streaming
• crates/zeroclaw-providers/src/ollama.rs: Ollama streaming
• crates/zeroclaw-channels/src/orchestrator/mod.rs: channel-side stream consumption