LiteLLMLiteLLM 1.105 · Python 3.10+
Dashboard
0%

Endpoints beyond chat

An endpoint is one job a model API can do, and LiteLLM has one function for each: completion() for chat, embedding() for vectors, transcription() for speech to text and more, all taking the same provider/model string.

Last updated: 12 Oct, 2026 · LiteLLM 1.105

Callbacks: tracking every call watched calls as they finished. Every call so far was a chat call. A support desk needs more than answers: it finds similar tickets and transcribes voice messages. Each of those is its own endpoint, with the same model strings, the same exceptions and the same cost fields.

Syntax: the function changes with the job, and the model string works as before.

python
litellm.embedding(model="gemini/gemini-embedding-001", input=["text"])
litellm.transcription(model="groq/whisper-large-v3-turbo", file=audio_file)
litellm.responses(model="groq/openai/gpt-oss-20b", input="text")

Turning tickets into vectors with embedding()

An embedding is a list of numbers that stands for the meaning of a text. Two texts with similar meanings get vectors that point the same way. embedding() takes a list of texts and returns one vector for each, under data.

ExampleAPI keyThree tickets embedded in one call
import litellm

tickets = ["Where is my parcel?", "My package has not arrived", "Please refund my payment"]
result = litellm.embedding(model="gemini/gemini-embedding-001", input=tickets)

print(len(result.data), "vectors")
print(len(result.data[0]["embedding"]), "numbers in each")
print(result.usage.prompt_tokens, "prompt tokens")

Three texts gave three vectors with the same length, and the usage reports only prompt tokens, because an embedding has no completion. Comparing vectors is plain arithmetic. The cosine of the angle between two vectors is close to 1 when they point the same way.

ExampleAPI keyHow close the tickets are in meaning
import math
import litellm

tickets = ["Where is my parcel?", "My package has not arrived", "Please refund my payment"]
vectors = [item["embedding"] for item in litellm.embedding(model="gemini/gemini-embedding-001", input=tickets).data]

def cosine(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    return dot / (math.sqrt(sum(x * x for x in a)) * math.sqrt(sum(y * y for y in b)))

print("parcel vs package:", round(cosine(vectors[0], vectors[1]), 3))
print("parcel vs refund: ", round(cosine(vectors[0], vectors[2]), 3))

The two tickets about a missing delivery scored higher with each other than either did with the refund ticket. This is how a desk finds a similar earlier ticket, and the same vectors power the semantic cache in Semantic caching.

Turning speech into text with transcription()

transcription() takes an audio file and returns its text. The example first makes a short spoken file. On macOS the built-in say command writes one. On another system, use any short recording of your own in place of the first call, and keep the file name.

ExampleAPI keyA spoken ticket turned into text
import subprocess
import litellm

# macOS only: speak a sentence into a file. Elsewhere, use your own ticket.wav.
subprocess.run(["say", "-o", "ticket.wav", "--data-format=LEI16@16000", "My order A17 has not arrived."], check=True)

with open("ticket.wav", "rb") as audio:
    result = litellm.transcription(model="groq/whisper-large-v3-turbo", file=audio)
print(result.text)
print(result._hidden_params.get("response_cost"))

Groq serves a Whisper model that answers through the same litellm call shape. The model is an audio model, so it is called with transcription() and not completion(). The result has a text field. The last line printed None. This call came back with no cost figure, so code that adds up costs should expect a missing value from some endpoints.

Calling the Responses API with responses()

responses() uses OpenAI's newer Responses API shape: input in place of messages, and max_output_tokens in place of max_tokens. LiteLLM offers it for providers that can be mapped onto it, and Groq is one.

ExampleAPI keyThe same question through the Responses shape
import litellm

response = litellm.responses(model="groq/openai/gpt-oss-20b", input="Name one colour.", max_output_tokens=400)
print(type(response).__name__)
print(response.output_text)

The other endpoints

The rest of the family follows the same pattern, and the course keys do not run them. Each of these functions exists in 1.105.0, and each needs a provider that serves that kind of model, which is often a paid one.

  • litellm.image_generation() returns pictures for a prompt.
  • litellm.speech() turns text into audio.
  • litellm.rerank() orders a list of documents by how well each answers a query.
  • litellm.moderation() judges whether text breaks a provider's rules.
  • litellm.create_batch() sends work in bulk, to be collected later at a lower price.

The image generation, rerank and moderation pages in the documentation list their providers. None of them is run in this course.

Everything else still applies

These endpoints are not a side road. The same model strings, mapped exceptions and cost fields apply to them. A Router model group can hold embedding deployments. The gateway serves them at /v1/embeddings and the other OpenAI paths behind the same key, as The LiteLLM proxy shows.

Where you use these endpoints

  • Finding similar tickets with embeddings and a cosine score.
  • Turning a voice message into a ticket with transcription, then sorting it with a chat call.
  • Moving a program between providers for embeddings the same way it moves for chat, by changing the model string.
Watch out. A model works only with the endpoint it was built for. Calling completion() with an embedding model, or embedding() with a chat model, fails with an error from the provider. Check the model's kind on the provider's page before calling.
Try it yourself
  • Add a fourth ticket, Charged twice for one order, and score it against the refund ticket.
  • Embed one ticket on its own and print the first three numbers of its vector.
  • Change the sentence given to say and read how the transcription follows it.

You understood something today that you didn't yesterday.