Streaming

Stream responses token-by-token using server-sent events (SSE). This gives your users a real-time typing experience instead of waiting for the full response.

Enable streaming

Set stream: true in your request:

stream = client.chat.completions.create(
    model="gpt-oss-20b",
    messages=[{"role": "user", "content": "Write a poem about the Nile"}],
    stream=True
)
 
for chunk in stream:
    if not chunk.choices:  # usage-only chunk at the end of the stream
        continue
    content = chunk.choices[0].delta.content
    if content:
        print(content, end="", flush=True)

Raw HTTP streaming

curl -N https://backend.sovereigneg.com/v1/chat/completions \
  -H "Authorization: Bearer sk-..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-oss-20b",
    "messages": [{"role": "user", "content": "Hello"}],
    "stream": true
  }'

The -N flag disables curl's output buffering, so you see tokens as they arrive.

SSE format

Each event is a JSON object prefixed with data: :

data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":" world"},"finish_reason":null}]}

data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[],"usage":{"prompt_tokens":12,"completion_tokens":3,"total_tokens":15}}

data: [DONE]

Streams always end with a token-usage chunk, even if you did not set stream_options.include_usage. Its choices list is empty, so check choices before reading choices[0].

The stream ends with data: [DONE].

When credit runs out mid-stream

When your organization's spendable credit runs out while a stream is running, the stream is not dropped: it ends cleanly. The last chunk carries finish_reason: "length" for every choice, a usage block with the token counts you are billed for, and an extra top-level key, stop_reason: "balance_exhausted"; data: [DONE] follows as usual. There is no separate usage-only chunk after it.

data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","model":"<id reported by the serving model>","choices":[{"index":0,"delta":{},"finish_reason":"length"}],"usage":{"prompt_tokens":1234,"completion_tokens":8192,"total_tokens":9426},"stop_reason":"balance_exhausted"}

data: [DONE]

Keep the text you received: it is billed as usage shows, and the OpenAI SDKs pass stop_reason through as an extra field on the chunk. A new request succeeds after a top-up, or once your organization's other requests still running release the credit they hold: while they run, a new streaming request naming its model first waits for them (up to about 25 seconds) and is refused with 429 inflight_busy (retry after Retry-After) only if they are still running then, or with 402 insufficient_balance when the credit is really gone; a new request with stream: false naming its model gets that 429 inflight_busy at once or after waiting up to about 10 seconds. Only a stream ends part-way like this: a request with stream: false is served with max_tokens lowered up front to what your credit can pay for (X-Output-Clamped, see Chat Completions). A streaming request is lowered the same way, at what your credit buys, with one exception: on a model that can reason before it answers where you cannot see it (Claude, OpenAI GPT-5 and o-series, Gemini thinking models, Grok), max_tokens is never lowered below 1,024 output tokens, because the reasoning is billed as output and could use up a smaller cap and leave an empty answer. Below that such a request gets 402 insufficient_balance, or 429 inflight_busy after waiting up to about 25 seconds for them while your organization's other requests still running hold the credit. A streaming request whose own max_tokens already fits your credit is served as is, however small.

When a model ignores max_tokens

A few models were measured generating far past the max_tokens they were sent. When one of them runs past the cap a streaming request covers, the stream ends the same clean way, with finish_reason: "length" and stop_reason: "cap_exceeded" instead of balance_exhausted. It has nothing to do with your credit: a top-up does not change it, and a new request with a larger max_tokens runs normally. Output past the max_tokens the stream ran with is not billed.

A monthly spend cap on your API key or project is a hard limit as well. When the cap has no room left for the rest of a running stream, the stream ends the same clean way, with stop_reason: "spend_cap_reached", and nothing is billed past the cap. A top-up does not change it: a new request runs once the cap is raised (a project's cap, under Projects → Limits; a key's cap can't be changed, so use a key with a higher cap), once the requests still running under the same cap finish, or when the month resets (Cairo time).

Errors

Where an error shows up depends on whether the stream has started.

• Before the stream starts, a streaming request fails exactly like a non-streaming one: an HTTP error status with a JSON error body, and no stream. This covers a request the model's provider rejects, no available backend, and an empty response. See Errors for the codes. The OpenAI SDKs raise this error from the create call itself.
• Once the stream has started, the response is already 200 OK, so the error arrives inside the stream as an event carrying an error object, and the stream then ends:

data: {"error":{"message":"Engine unavailable","code":"engine_unavailable","type":"service_unavailable"}}

The OpenAI SDKs raise this one while you iterate over the stream.

Next steps

The Streaming Responses guide has examples for async Python, Node.js, and React, along with error-handling code and timeout advice.

Never put your sk- key in browser code: anyone who opens the page can read it and spend your balance (see Authentication). Call SovereignEG from your server and stream the text on to the browser.