Description
Streaming Agent Responses lets a user watch an assistant answer arrive incrementally instead of waiting for the entire provider run. The dispatch runtime forwards append-only text deltas and live tool-activity events to a caller-supplied sink. It can also signal a reset when a provider's intermediate tool-calling step should be discarded before the final answer begins, avoiding a misleading transcript assembled from temporary reasoning.
Streaming is a presentation path over the standard turn lifecycle, not a second persistence model. When a provider supports streaming, the runtime emits live updates with backpressure-aware callbacks; when it does not, the same method falls back to buffered execution. In both cases the returned DispatchTurnResult and the committed conversation messages are authoritative, so a dropped client connection cannot redefine what the agent actually completed.
Selling Points
- Users see useful progress immediately during long coding-agent runs rather than staring at an inert waiting state.
- Live tool-start and tool-finish events explain what the agent is doing without treating provisional output as durable truth.
- Provider capability detection gives every provider one callable surface, with safe buffered fallback when native streaming is unavailable.
- Backpressure-aware callbacks let SSE, desktop IPC, or another transport pace delivery without corrupting the underlying run.
- Terminal reconciliation replaces any preview with the exact persisted answer, preventing drift between live UI and conversation history.
User Story
A developer asks a local agent to analyze a large repository. Text begins appearing as the agent summarizes its progress, and the interface shows tool activity as files are read and searches run. If the agent enters a tool step, the preview resets appropriately. When the turn finishes, the developer has the final answer in the transcript and can reopen it later with no difference between what was committed and a normal buffered turn.
Extension Surface
This feature is extensible. packages/chat/src/dispatch/buffered-dispatch.ts exposes the structural DispatchStreamSink with required onDelta, onReset, and onToolEvent callbacks, and packages/chat/src/contracts/provider-registry.ts admits providers that declare streaming support. Composition can connect new transports or providers in-repo, but no external streaming plugin manifest is shipped.
Capabilities & Limits
- It delivers incremental assistant fragments and tool lifecycle observations.
- It supports asynchronous sink callbacks for transport backpressure.
- It falls back to buffered completion while preserving the same public return type.
- Limit: streamed updates are best-effort presentation; only the terminal dispatch result and persisted messages are authoritative.
Implementation Map
packages/chat/src/dispatch/buffered-dispatch.tspackages/chat/src/dispatch/provider-turn.tspackages/chat/src/dispatch/execute-turn.tspackages/chat/src/contracts/provider-registry.ts