LiteLLMLiteLLM 1.101 · Python 3.10+
0%
1
Curious builder0 XP earned · 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
20 small wins to finish your pathNext lesson →

What you are going to build

You will finish this course with a model gateway for a support team: one address every app calls, with cheap and careful models, fallbacks when a provider fails, caching and cost tracking.

LiteLLM is an open-source Python library and server that calls more than a hundred model providers through one interface, in the OpenAI format. As a library it is a function you call. As a gateway, called the proxy in its docs, it is a server your whole company's apps send requests to.

One request through the gateway
An app sends a request. In the OpenAI format, with a gateway key and a model group name such as careful, not a provider's model id.Step 1 of 5
The five parts of the course
One callwhy one interfacecompletion()providers and keysa stand-in providerWhat comes backstreamingtokens and costerrorscallbacksStaying upretriesthe RouterfallbackscooldownsSpending lesscachingthe cheap model firstThe gatewayconfig and startOpenAI clientskeysfallbacks and cacheLiteLLM

No API key

LiteLLM can answer without calling anyone, and it lets you plug in your own provider class. In lesson 4 you write a stand-in provider that sorts support tickets, and in lesson 7 two that fail on purpose, so retries and fallbacks can be seen working. Every output is from a real run.

What you need

  • Python 3.10 or later.
  • Functions, classes, async and Pydantic from Python for AI, and the idea of an API from APIs for AI.
Example
pip install "litellm[proxy]==1.101.0" tenacity prisma openai

[proxy] adds the gateway server. tenacity is needed for retries in lesson 9 and prisma for the gateway's key checks in lesson 17; lessons 9 and 17 show what happens without them. openai is used to call the gateway in lesson 16.

Back toAll frameworks

Every expert started right here.