Lesson 12 — Streaming and stop reasons

Text that arrives in pieces

A long answer can take many seconds. With streaming, the API sends the reply in small chunks as the model writes it, so your app can show words straight away (that is the "typing" effect in chat apps). Your code joins the pieces into the full text.

At the end the API tells you why the model stopped — the stop reason. Your agent must check it, because each reason needs a different next step. In Anthropic's Messages API the main values are:

• end_turn — the model finished its answer. Show it.
• tool_use — the model wants a tool. Run it and call the model again (Lesson 16).
• max_tokens — the reply hit your max_tokens limit and is cut off. Raise the limit or ask the model to continue.

Watch out: each provider names these differently — other APIs use other field names and other values for the same ideas. Always check the provider's docs instead of copying names from one API to another.

Real streams are a series of events with more types than ours (in Anthropic's API, for example, the stop reason arrives in a message_delta event near the end). Our chunks are simplified: {"type": "text", ...} pieces and one {"type": "stop", ...} chunk at the end.

Why it matters: an agent that ignores the stop reason will happily show a half-finished answer, or miss that the model asked for a tool.