Streaming: the answer a piece at a time

stream=True returns the answer in chunks as it is produced, in the same chunk format for every provider, so one loop shows a reply appearing word by word.

from litellm import completion

chunks = completion(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "Apologise for a late parcel"}],
    mock_response="Sorry your parcel is late, we are on it.",
    stream=True,
)
for chunk in chunks:
    print(repr(chunk.choices[0].delta.content))

Each chunk carries a delta: the new piece of text, not the whole answer so far. The last chunk has None content, and its finish_reason says the answer is complete. The SDK docs note mock_response streams too, which is why no key is needed here; the pieces are cut by LiteLLM, where a real model would send tokens.

Putting the answer together

parts = []
for chunk in completion(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "Apologise for a late parcel"}],
    mock_response="Sorry your parcel is late, we are on it.",
    stream=True,
):
    parts.append(chunk.choices[0].delta.content or "")

print("".join(parts))

or "" turns the final None into an empty string, so joining works. A chat interface prints each piece as it arrives instead of collecting them.

A provider class streams only if it implements the streaming method; the ShopLLM from lesson 4 does not, which is why this lesson uses mock_response.