Streaming: the answer a piece at a time
Streaming is a call with stream=True that returns the answer in chunks as the model writes it, in the same chunk format for every provider, so one loop shows a reply growing on screen.
Last updated: 12 Oct, 2026 · LiteLLM 1.105
Model aliases: names of your own chose the model by name. Which model answers does not change how the answer arrives: so far each call waited for the whole reply. A chat window cannot wait, so this lesson reads the reply piece by piece.
Syntax: add stream=True and loop over the result. Each chunk's new text is at choices[0].delta.content.
for chunk in completion(model=model, messages=messages, max_tokens=400, stream=True):
piece = chunk.choices[0].delta.content # new text only, or None
print(piece)Looking at the first chunks
Each chunk carries a delta, the newly written part and not the whole answer so far. This example prints the first five chunks and counts the rest. It uses the 20b model, which is quick.
from litellm import completion
messages = [{"role": "user", "content": "Count from 1 to 5, one number per line."}]
chunks = completion(model="groq/openai/gpt-oss-20b", messages=messages, max_tokens=400, stream=True)
count = 0
for chunk in chunks:
count += 1
if count <= 5:
delta = chunk.choices[0].delta
print(repr(delta.content), repr(getattr(delta, "reasoning_content", None)))
print(count, "chunks in all")'' None None 'User' None ' wants' None ' to' None ' count' 53 chunks in all
Reading chunks that carry no text
- The first chunk carries no text, and the chunks after it have
contentofNone. A reasoning model writes its thinking first, and that arrives inreasoning_content, not incontent. - The text of the answer starts later, in chunks where
contentis a string. - The count is the number of chunks, which includes the thinking chunks, so it is larger than the number of visible pieces. It differs from run to run.
Putting the answer together
delta.content or "" turns a missing piece into an empty string, so the pieces can be joined. A chat window instead prints each piece as it arrives, with end="" to stay on one line and flush=True so it appears at once.
from litellm import completion
messages = [{"role": "user", "content": "Count from 1 to 5, one number per line."}]
chunks = completion(model="groq/openai/gpt-oss-20b", messages=messages, max_tokens=400, stream=True)
parts = []
for chunk in chunks:
piece = chunk.choices[0].delta.content or ""
print(piece, end="", flush=True)
parts.append(piece)
print()
print(repr("".join(parts)))1 2 3 4 5 '1\n2\n3\n4\n5'
Rebuilding a full response from the chunks
The chunks of a stream carry text and no token counts. litellm.stream_chunk_builder joins a list of chunks back into one ModelResponse, and works the usage and the cost out of them.
import litellm
messages = [{"role": "user", "content": "Count from 1 to 5, one number per line."}]
chunks = list(litellm.completion(model="groq/openai/gpt-oss-20b", messages=messages, max_tokens=400, stream=True))
response = litellm.stream_chunk_builder(chunks, messages=messages)
print(type(response).__name__)
print(response.usage.total_tokens)
print(response._hidden_params["response_cost"])ModelResponse 148 2.55e-05
The result has the same fields as a response from a call without streaming, including the cost, which Tokens and cost reads in detail.
Streaming without blocking using async for
acompletion(stream=True) returns an async iterator, and the loop becomes async for. A web server uses this form so one process can stream to many users at once.
import asyncio
from litellm import acompletion
async def main():
messages = [{"role": "user", "content": "Name one colour."}]
stream = await acompletion(model="groq/openai/gpt-oss-20b", messages=messages, max_tokens=400, stream=True)
parts = []
async for chunk in stream:
parts.append(chunk.choices[0].delta.content or "")
print("".join(parts))
asyncio.run(main())Blue.
Streaming vs waiting for the whole answer
| Without stream | With stream=True | |
|---|---|---|
| What the call returns | One ModelResponse | An iterator of chunks |
| Where the text is | message.content | delta.content of each chunk |
| When the first text appears | After the whole answer is written | As soon as the model writes it |
| Token usage | On the response | Not on the chunks. Rebuild the response to read it |
| Best for | Batch work and short answers | Chat windows and long answers |
Where you use streaming
- A chat interface that prints words as they arrive instead of showing a blank box.
- A server-sent events endpoint that forwards each chunk to a browser.
- A long answer where the first words are useful long before the last ones exist.
delta.content is None in some chunks. Joining chunks without or "" raises TypeError the first time one arrives. It can arrive in the middle of an answer, not only at the end.Related
- Previous: Model aliases: names of your own
- Next: Tokens and cost
- Reference: Streaming and async in the LiteLLM docs
- Change the prompt to
Write one sentence about parcels.and compare the chunk count. - Remove
or ""from the joining example and read theTypeError. - Print
chunks[-1].choices[0].finish_reasonafter collecting the list, and read what the last chunk reports.
Slow is fine. Stopping is the only problem.