# Lab 4 — Streaming and SSE

**Time:** 30 minutes · **Prerequisites:** Lab 1

## Why this matters

Everything in this lab is in service of Marco not sending a bad reply.

The draft route writes to a customer, and the customer might be the person whose
daughter went to urgent care. Tone rules that look fussy on paper — lead with the
resolution, one apology, never promise a refund "today" — are each there because
a version of that message went out and made things worse. The 5–7 day language
exists because agents kept promising same-day refunds and customers kept
escalating on day two.

Streaming matters for a narrower reason: an agent will not wait fifteen seconds
staring at a spinner during a December shift with 11,000 tickets in the queue.
They will alt-tab, and the draft will sit unread. Perceived latency is the
difference between a tool that gets used and one that gets abandoned.

The two production details in Steps 4 and 6 — proxy buffering and client
disconnects — are how this feature works in your dev environment and fails in
theirs.

---

```try
{
  "tool": "stream",
  "title": "Watch tokens arrive — then break the stream",
  "lead": "Offline replay of text / thinking / done. Flip Fail mid-stream to see an in-band error after HTTP 200 — the status code stays green; the UI must not.",
  "href": null
}
```

## Objectives

- Stream a response and handle the event types you actually care about
- Get `usage` out of a stream (it isn't there until the end)
- Handle errors that occur *after* the HTTP status is already 200
- Abort an upstream stream when the client disconnects

```mermaid
sequenceDiagram
    participant Client
    participant Route as /v1/draft
    participant Claude

    Client->>Route: POST (SSE)
    Route-->>Client: HTTP 200 + stream open
    loop token generation
        Claude-->>Route: delta
        Route-->>Client: event: text
        Route-->>Client: event: thinking
    end
    Route-->>Client: event: done (usage, stop_reason)
    Note over Route,Client: Mid-stream failure → event: error (not a new status code)
```

---

## Step 1 — watch tokens arrive

```bash
curl -N -s localhost:8787/v1/draft -H 'content-type: application/json' -d '{
  "message":"My Trail Club membership renewed but I cancelled in June. I want the $89 back."
}'
```

`-N` disables curl's buffering. Without it you see the whole reply at once and
conclude streaming is broken — a preview of Step 4.

## Step 2 — the event taxonomy

Read [`src/routes/draft.ts`](https://github.com/mrlynn/claude-triage-api/blob/main/src/routes/draft.ts). The route emits four
event types:

| Event | Carries | A UI should |
|---|---|---|
| `text` | reply delta | append to the message body |
| `thinking` | summarized reasoning delta | render in a collapsed panel |
| `done` | `stop_reason`, `usage` | stop the spinner, log cost |
| `error` | error body | show a failure state |

Now comment out `display: "summarized"` in the `thinking` config and re-run.

**Q1.** The `thinking` events still arrive but carry empty text. Given that
thinking happens and is billed identically either way, what exactly does
`display` control — and why does the default (`"omitted"`) produce a bad
first-run UX?

## Step 3 — get the cost

`usage` is not available mid-stream. It arrives at the end:

```ts
// src/routes/draft.ts
const final = await stream.finalMessage();
send("done", {
  stop_reason: final.stop_reason,
  model: final.model,
  usage: summarizeUsage(final.usage, final.model),
});
```

**Q2.** A teammate proposes wrapping `stream.on("message", ...)` in a
`new Promise()` to capture the final message. Why is `finalMessage()` the
correct tool, and what states does it handle that the hand-rolled promise
would not?

## Step 4 — the production failure you will hit

Everything above works locally. Deployed behind nginx with default settings,
the client receives the entire reply in one chunk after full generation.

Look at `SSE_HEADERS` in [`src/lib/sse.ts`](https://github.com/mrlynn/claude-triage-api/blob/main/src/lib/sse.ts).

**Q3.** Which header prevents this, and why is `Cache-Control: no-transform`
also there? Name one other layer between your service and the browser that can
buffer a stream.

```quiz
[
  {
    "question": "Your streaming endpoint fails upstream halfway through generating. What does the client see?",
    "options": [
      "A 502, because the request failed",
      "HTTP 200 with a half-written response, unless you emit an error event in-band",
      "The connection drops and fetch throws"
    ],
    "answer": 1,
    "explain": "By the time generation starts, the status line is already sent. A mid-stream failure cannot be expressed as a non-2xx response, so it has to be delivered as an in-band event the client explicitly handles.",
    "note": "A client that only checks `res.ok` will show a truncated customer reply as if it were complete."
  }
]
```

## Step 5 — errors after 200

Simulate an upstream failure mid-stream. Easiest reproduction: start the server
with a valid key, begin a request, then note that `src/routes/draft.ts` catches
inside the `ReadableStream` and emits an in-band `error` event.

**Q4.** By the time generation starts, the HTTP status is already 200. Write
the one-sentence rule a client integrator must follow, and explain what breaks
if they only check `res.ok`.

## Step 6 — hang up

```ts
c.req.raw.signal.addEventListener("abort", () => stream.abort());
```

Start a request and kill curl mid-generation (Ctrl-C).

**Q5.** Without this line, what continues to happen, and who pays for it?

## Step 7 — re-run the scoreboard

```bash
npm run eval:quick
```

Streaming is a delivery change, not a reasoning change. If the score moved,
something other than delivery changed and you want to know what.

---

## Checkpoint

- [ ] Which `content_block_delta` delta types matter, and what do they mean?
- [ ] How do you get `usage` from a stream?
- [ ] Why can't a mid-stream error be an HTTP error?
- [ ] What must you do on client disconnect?
- [ ] Scoreboard re-run; you can say why it did or did not move

---

## Extension

Build a 40-line HTML page that consumes `/v1/draft` with `EventSource` (note:
`EventSource` is GET-only, so you will need `fetch` + a `ReadableStream` reader
for a POST body — discovering that is part of the exercise). Render `text` into
the body and `thinking` into a `<details>` element.

```mistake
[
  {
    "id": "final-message-hand-rolled",
    "wrong": "const final = await new Promise((resolve) => stream.on(\"message\", resolve));",
    "right": "const final = await stream.finalMessage();",
    "symptom": "Works on every happy-path request. When the stream errors or the client aborts, the promise never settles, and the request hangs with no `done` event.",
    "why": "`message` only fires on success. `finalMessage()` resolves with the complete message and rejects on errors and aborts, which is the part a hand-rolled promise leaves out."
  },
  {
    "id": "stream-error-not-sent",
    "wrong": "} catch (err) { console.error(\"draft stream failed\", err); }",
    "right": "} catch (err) {\n  console.error(\"draft stream failed\", err);\n  // The status is already 200, so the failure has to travel in-band.\n  send(\"error\", toHttpError(err).body);\n}",
    "symptom": "The client sees HTTP 200, half a reply, and the stream closing. It shows the half reply as if it were finished. To show any Lab 4 mistake without the API, a mock route must enqueue bytes, not strings: encode each event with a `TextEncoder` (`controller.enqueue(encoder.encode(sseEvent(event, data)))`, as `src/routes/draft.ts` does). A client reading `res.body` and calling `TextDecoder.decode()` throws on a string chunk before anything is shown. And do not open a fixed port: call the handler in-process with a `Request`, or `listen(0)` and read the port the OS assigns. Port 8787 is the course API's own port, so a learner following the labs usually has it taken.",
    "why": "By the time generation starts the status line has been sent. A mid-stream failure can only reach the client as an event it explicitly handles."
  },
  {
    "id": "disconnect-not-aborted",
    "wrong": "request.signal.addEventListener(\"abort\", () => console.log(\"client disconnected\"));",
    "right": "request.signal.addEventListener(\"abort\", () => stream.abort());",
    "symptom": "Kill the client mid-reply and the server log looks fine, but generation runs to the end and the output tokens are billed for a reply nobody receives.",
    "why": "The client leaving does not stop the upstream request. Only aborting the stream does."
  },
  {
    "id": "sse-headers-buffered",
    "wrong": "const SSE_HEADERS = { \"Content-Type\": \"text/event-stream\", \"Cache-Control\": \"no-cache\" };",
    "right": "const SSE_HEADERS = {\n  \"Content-Type\": \"text/event-stream; charset=utf-8\",\n  \"Cache-Control\": \"no-cache, no-transform\",\n  Connection: \"keep-alive\",\n  \"X-Accel-Buffering\": \"no\",\n};",
    "symptom": "Streams token by token locally. Behind nginx in production, the whole reply arrives in one chunk after generation finishes.",
    "why": "A proxy buffers responses by default. `X-Accel-Buffering: no` turns that off for nginx, and `no-transform` stops intermediaries that would compress or rewrite the stream."
  },
  {
    "id": "thinking-display-omitted",
    "wrong": "  thinking: { type: \"adaptive\" },",
    "right": "  thinking: { type: \"adaptive\", display: \"summarized\" },",
    "symptom": "`thinking` events arrive with empty text, so the reasoning panel stays blank while the user waits, even though the thinking is happening and being billed.",
    "why": "`display` controls whether thinking text is returned, not whether thinking happens. The default omits it."
  }
]
```

**Answers:** [../solutions/lab-4.md](https://triage.mlynn.dev/docs/solutions/lab-4)

Source: https://triage.mlynn.dev/docs/labs/lab-4-streaming
