# Streaming

Read the answer as it is written, on chat completions, responses and messages, and what a stream looks like when it fails.

Send `"stream": true` and the answer arrives as server-sent events while the model writes it, instead of in one piece at the end. Each route streams in its own SDK's format, so the SDK's streaming loop reads it unchanged.

```bash
# -N prints each event as it arrives; the stream ends with data: [DONE]
curl -N https://api.vultrinference.com/v1/chat/completions \
  -H "Authorization: Bearer $VULTR_INFERENCE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4.1-flash",
    "stream": true,
    "messages": [{"role": "user", "content": "Count from one to five."}]
  }'
```

```python
import os

from openai import OpenAI

client = OpenAI(base_url="https://api.vultrinference.com/v1", api_key=os.environ["VULTR_INFERENCE_API_KEY"])

stream = client.chat.completions.create(
    model="deepseek-v4.1-flash",
    stream=True,
    messages=[{"role": "user", "content": "Count from one to five."}],
)
for chunk in stream:
    if chunk.usage:
        # The last chunk carries the usage, always
        print(f"\n{chunk.usage.prompt_tokens} in, {chunk.usage.completion_tokens} out")
    elif chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
```

```typescript
import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://api.vultrinference.com/v1", apiKey: process.env.VULTR_INFERENCE_API_KEY });

const stream = await client.chat.completions.create({
  model: "deepseek-v4.1-flash",
  stream: true,
  messages: [{ role: "user", content: "Count from one to five." }],
});
for await (const chunk of stream) {
  if (chunk.usage) {
    // The last chunk carries the usage, always
    console.log(`\n${chunk.usage.prompt_tokens} in, ${chunk.usage.completion_tokens} out`);
  } else {
    process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
  }
}
```

## Chat completions

Each event is one chunk on a `data:` line, and the stream ends with `data: [DONE]`. This is the whole stream of a short answer, "hi there", with each chunk's `id`, `model` and other fixed fields left out, and the usage shortened:

```sse
data: {"choices":[{"delta":{"role":"assistant","content":""},"finish_reason":null}]}
data: {"choices":[{"delta":{"content":"hi"},"finish_reason":null}]}
data: {"choices":[{"delta":{"content":" there"},"finish_reason":"stop"}]}
data: {"choices":[{"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":8,...}}
data: [DONE]
```

- A delta holds only what it adds, and never a field set to null. Thinking arrives the same way, as `reasoning`.
- A tool call arrives in pieces of `tool_calls`: its id and name first, then its arguments, matched by `index`.
- The last chunk always carries `usage`, whether you asked for it or not. Its one choice has an empty delta, so a loop that reads `chunk.choices[0]` keeps working.

## Responses

Each event is an `event:` line and a `data:` line, numbered by `sequence_number`, in this order:

<div class="flow" role="img" aria-label="response.created, then response.output_item.added, then response.output_text.delta repeated, then response.output_item.done, then response.completed">
<span>response.created</span><span>output_item.added</span><span class="repeat">output_text.delta</span><span>output_item.done</span><span>response.completed</span>
</div>

The pieces of the answer come in three kinds of delta:

| Event | Carries |
| --- | --- |
| `response.output_text.delta` | The answer's text |
| `response.reasoning_text.delta` | The model's thinking |
| `response.function_call_arguments.delta` | A tool call's arguments |

There is no `[DONE]`. The last event says how the response ended: `response.completed`, `response.incomplete` when `max_output_tokens` cut it short, or `response.failed`.

## Messages

`/v1/messages` streams Anthropic's events, in this order, with one start, run of deltas and stop for each block of the answer:

<div class="flow" role="img" aria-label="message_start, then content_block_start, content_block_delta repeated and content_block_stop for each block, then message_delta, then message_stop">
<span>message_start</span><span>content_block_start</span><span class="repeat">content_block_delta</span><span>content_block_stop</span><span>message_delta</span><span>message_stop</span>
</div>

| Delta | Carries |
| --- | --- |
| `text_delta` | The answer's text |
| `thinking_delta` | The model's thinking |
| `signature_delta` | The thinking block's signature, at its end |
| `input_json_delta` | A tool call's input, as pieces of JSON |

`message_delta` carries the `stop_reason`.

> [!NOTE]
> The token counts in a Messages stream read 0. Read them from the non-streaming answer when you need them.

## When a stream fails

A request that fails before the first token is answered with the real status and an ordinary error body, never a stream: check the status before you read events.

A failure after the stream has started cannot change the status, which was 200. Each route ends the stream with an error of its own kind instead:

| Route | The last thing sent |
| --- | --- |
| Chat completions | A chunk with `error` (the usual envelope) and `finish_reason: "error"`, then `data: [DONE]` |
| Responses | An `error` event, then `response.failed` |
| Messages | An `error` event (`api_error`), then `message_stop` |

Treat the text received before it as incomplete. [Errors](/docs/concepts/errors#when-a-stream-fails) shows the chunk.

## Other streams

- **Speech**: with `stream_format: "sse"` the audio arrives in `speech.audio.delta` events, and `speech.audio.done` ends with the usage. Only `pcm` is streamed as it is made; other formats arrive in one piece.
- **Transcriptions**: with `stream: true` the text arrives in `transcript.text.delta` events, and `transcript.text.done` holds all of it.
- **Music**: a song can be heard while it renders, from its job's `stream_url` ([Async jobs](/docs/concepts/async-jobs#hear-a-song-as-it-renders)).