LiteLLMLiteLLM 1.105 · Python 3.10+
Dashboard
0%

Streaming: the answer a piece at a time

Streaming is a call with stream=True that returns the answer in chunks as the model writes it, in the same chunk format for every provider, so one loop shows a reply growing on screen.

Last updated: 12 Oct, 2026 · LiteLLM 1.105

Model aliases: names of your own chose the model by name. Which model answers does not change how the answer arrives: so far each call waited for the whole reply. A chat window cannot wait, so this lesson reads the reply piece by piece.

Syntax: add stream=True and loop over the result. Each chunk's new text is at choices[0].delta.content.

python
for chunk in completion(model=model, messages=messages, max_tokens=400, stream=True):
    piece = chunk.choices[0].delta.content   # new text only, or None
    print(piece)

Looking at the first chunks

Each chunk carries a delta, the newly written part and not the whole answer so far. This example prints the first five chunks and counts the rest. It uses the 20b model, which is quick.

ExampleAPI keyThe first five chunks of a reasoning model
from litellm import completion

messages = [{"role": "user", "content": "Count from 1 to 5, one number per line."}]
chunks = completion(model="groq/openai/gpt-oss-20b", messages=messages, max_tokens=400, stream=True)

count = 0
for chunk in chunks:
    count += 1
    if count <= 5:
        delta = chunk.choices[0].delta
        print(repr(delta.content), repr(getattr(delta, "reasoning_content", None)))
print(count, "chunks in all")

Reading chunks that carry no text

  • The first chunk carries no text, and the chunks after it have content of None. A reasoning model writes its thinking first, and that arrives in reasoning_content, not in content.
  • The text of the answer starts later, in chunks where content is a string.
  • The count is the number of chunks, which includes the thinking chunks, so it is larger than the number of visible pieces. It differs from run to run.

Putting the answer together

delta.content or "" turns a missing piece into an empty string, so the pieces can be joined. A chat window instead prints each piece as it arrives, with end="" to stay on one line and flush=True so it appears at once.

ExampleAPI keyPrinting each piece and joining them
from litellm import completion

messages = [{"role": "user", "content": "Count from 1 to 5, one number per line."}]
chunks = completion(model="groq/openai/gpt-oss-20b", messages=messages, max_tokens=400, stream=True)

parts = []
for chunk in chunks:
    piece = chunk.choices[0].delta.content or ""
    print(piece, end="", flush=True)
    parts.append(piece)
print()
print(repr("".join(parts)))

Rebuilding a full response from the chunks

The chunks of a stream carry text and no token counts. litellm.stream_chunk_builder joins a list of chunks back into one ModelResponse, and works the usage and the cost out of them.

ExampleAPI keyOne response from many chunks
import litellm

messages = [{"role": "user", "content": "Count from 1 to 5, one number per line."}]
chunks = list(litellm.completion(model="groq/openai/gpt-oss-20b", messages=messages, max_tokens=400, stream=True))

response = litellm.stream_chunk_builder(chunks, messages=messages)
print(type(response).__name__)
print(response.usage.total_tokens)
print(response._hidden_params["response_cost"])

The result has the same fields as a response from a call without streaming, including the cost, which Tokens and cost reads in detail.

Streaming without blocking using async for

acompletion(stream=True) returns an async iterator, and the loop becomes async for. A web server uses this form so one process can stream to many users at once.

ExampleAPI keyThe same loop with async for
import asyncio
from litellm import acompletion

async def main():
    messages = [{"role": "user", "content": "Name one colour."}]
    stream = await acompletion(model="groq/openai/gpt-oss-20b", messages=messages, max_tokens=400, stream=True)
    parts = []
    async for chunk in stream:
        parts.append(chunk.choices[0].delta.content or "")
    print("".join(parts))

asyncio.run(main())

Streaming vs waiting for the whole answer

Without streamWith stream=True
What the call returnsOne ModelResponseAn iterator of chunks
Where the text ismessage.contentdelta.content of each chunk
When the first text appearsAfter the whole answer is writtenAs soon as the model writes it
Token usageOn the responseNot on the chunks. Rebuild the response to read it
Best forBatch work and short answersChat windows and long answers

Where you use streaming

  • A chat interface that prints words as they arrive instead of showing a blank box.
  • A server-sent events endpoint that forwards each chunk to a browser.
  • A long answer where the first words are useful long before the last ones exist.
Watch out. delta.content is None in some chunks. Joining chunks without or "" raises TypeError the first time one arrives. It can arrive in the middle of an answer, not only at the end.
Try it yourself
  • Change the prompt to Write one sentence about parcels. and compare the chunk count.
  • Remove or "" from the joining example and read the TypeError.
  • Print chunks[-1].choices[0].finish_reason after collecting the list, and read what the last chunk reports.

Slow is fine. Stopping is the only problem.