Hugging Face's production server, now in maintenance mode but still widely deployed. Built by Hugging Face.
Others in inference and serving
Common questions
Is Text Generation Inference free to use?
Text Generation Inference is open source and free to run yourself. You still pay whichever model provider you point it at, and this tutorial is free with no signup.
Do I need to know Python to use Text Generation Inference?
Basic Python is enough. Functions, dictionaries and imports cover most of what Text Generation Inference asks of you.
When should I not use Text Generation Inference?
When something else in inference and serving fits the job better. The others in that group are listed below.
Lessons for Text Generation Inference are being written. Meanwhile the APIs for AI tutorial covers the same ground: state, tools, loops, memory and human approval. Most of it carries straight over.
Start the APIs for AI tutorial →