Python, Git and how a model behaves
Three things have to be true before any of the operations work makes sense. You can read and write the Python a service is made of, not just scripts. You can commit, branch and review code in Git, because every pipeline in this roadmap triggers on a push and every versioning problem later is a Git problem. And you know what a language model actually costs you per request, in tokens, money and milliseconds, because those three numbers are what the rest of the job is spent controlling.
Python for AI
Git and GitHub
LLM FundamentalsWhat LLMOps is, and the lifecycle
Before touching a tool, get the map. LLMOps has a lifecycle with named stages, and every stage after this one is one of them, so knowing the whole shape tells you why a gateway comes before a cost dashboard and why evaluation is not the last step. You also need a clear answer to what changed coming from MLOps, because the pipeline looks familiar and the hard parts have moved. Then you look at the thing being operated, end to end, so the system you are about to put behind a gateway and a monitor is not a black box.
LLMOps
Generative AI and Agentic AIPlan the build: pipeline, data and model
Most LLM projects fail at the plan, not the code. You need requirements concrete enough to evaluate against, a decision about whether this is retrieval, fine-tuning or both, a data pipeline that matches that decision, and a model chosen for reasons you can defend when the bill arrives. This stage makes those four choices properly, then goes and looks at the two builds you just chose between, so the choice is informed rather than theoretical.
LLMOpsGenerative AI and Agentic AIStructure the app for production
This is the stage that most separates an LLMOps engineer from someone with a working notebook. The same code gets a layout a stranger can navigate, configuration instead of constants, logging you can operate on, exceptions you can trace, prompts in one place with versions, a single entry point with an API in front of it, and packaging so it installs. Everything after this stage, the gateway, the traces, the container, the pipeline, assumes the app is shaped this way.
LLMOpsOne gateway in front of every model
Once more than one service calls a model, calling providers directly stops scaling. Keys spread, nobody knows what is spent or by whom, a provider outage takes the product down, and switching models means editing every service. A gateway fixes that by being the one thing in the middle: one interface for every provider, routing and retries, fallbacks when one fails, caching for the repeats, a key per application, and a single place where cost and usage are visible. This is the most load-bearing piece of infrastructure in the roadmap.
LiteLLMTrack experiments, prove quality
Opinions about prompts are worthless and cheap, so this stage replaces them with recorded runs and scores. Experiment tracking gives every change a parameter set, a metric and an artifact, so you can say which version was better and why. Evaluation gives you the metric in the first place: a golden dataset of cases that must keep working, metrics suited to retrieval rather than generic accuracy, and a test run that can fail in CI. From here on, a change ships because it scored better.
LLMOps
RAGAS
DeepEvalObservability, signals and cost
Once real users arrive, the question stops being is it good and becomes what happened on that request. Evaluation measures a system you control; observability explains a system you do not. This stage covers why monitoring an LLM app differs from monitoring an ordinary service, tracing a run so a bad answer can be reconstructed, prompts and scores kept where they can be versioned and compared, and the gateway-level metrics that tell you about latency, errors and spend before a user does.
LLMOps
LangfuseLiteLLMContainers, cloud and CI/CD
Now it leaves your machine. The app gets containerised so the environment that passed tests is the environment that runs, then deployed somewhere real with an API in front of it and a plan for scale and cost. CI/CD is what makes that repeatable: a push runs the tests and the evaluation gate from stage 6, builds the image, and deploys it. The deeper infrastructure, Kubernetes, load testing and autoscaling, comes from the AI Security course, which deploys an agentic API on EKS and then puts load through it.
LLMOps
AI SecurityGuardrails, governance and data
A deployed LLM app is reachable by anyone who can type, and the model will try to be helpful about things it should refuse. This stage builds the control layer: knowing the real attack surface, rails that check what comes in and what goes out, a policy gate so an agent's tool calls are allowed rather than assumed, an audit trail that can prove what was decided, and clear handling of the customer data flowing through all of it.
AI Security
NeMo Guardrails
Agent Governance ToolkitKeep it healthy, and hand it over
Shipping is the middle of the lifecycle, not the end. Models get deprecated, prompts rot as the product changes, retrieval quality drifts as documents are added, and costs creep. This last stage covers the maintenance loop that catches all of that, builds the keyless assistant end to end and swaps its stand-ins for real services, writes the runbook that lets someone else take the pager, and ends on the course's full production project, BrandGuardian, the whole roadmap running as one real system you can read and operate.
LLMOpsPlatform engineer
10 stages, 12 frameworks and 2 projects. Finish them and you will have shipped the kind of systems these teams hire for, with the lessons to back every one.
Browse all frameworksQuestions about this roadmap
What does an AI platform or LLMOps engineer do?
They make an LLM application survive contact with production. That means a repository a team can work in, a gateway in front of every provider, experiment tracking so a change can be justified, evaluation that runs in CI, traces and cost dashboards, a container and a deployment pipeline, guardrails and an audit trail, and a maintenance loop for drift. It is MLOps discipline applied to language models, where cost, latency and safety matter as much as accuracy.
How is LLMOps different from MLOps?
The pipeline shape is similar and the hard parts move. There is usually no training run to manage, so the work shifts to prompts, retrieval and the model you chose from a provider. Quality is no longer a single accuracy number, so evaluation becomes judges and golden datasets. And cost is per token at request time rather than per training job, so it never stops being your problem. Stage 2 covers this directly.
Can I start this roadmap as a beginner?
Yes, but start at stage 1 and take it seriously. It teaches the Python a service is written in, Git and GitHub from the first commit, and how a language model behaves in terms of tokens, cost and latency. Every later stage assumes only the stages before it.
Do I need an API key, a GPU or a cloud account?
No. The courses run on small stand-in models you write yourself and a local MLflow server, so the hands-on work needs no key and no GPU. Fine-tuning, SageMaker and Azure deployment are shown as real projects and named honestly as needing hardware or an account, rather than faked.
Why does version control come before the model work?
Because every stage after it depends on commits. CI/CD triggers on a push, versioning a prompt or a dataset is a Git problem, and the single most common beginner accident in this field is committing a model file or a key. Stage 1 covers both the commands and the hygiene.
Some stops say coming soon. Why?
Three tools on this path have no full course yet: vLLM for self-hosted serving, Terraform for infrastructure as code, and Airflow-style orchestration. Those stops link to the official docs and are marked done by hand. Everything else is live lessons you can take today, and the upcoming courses drop into the same stops as they are written.
How is this different from the AI Forward Deployed Engineer roadmap?
That roadmap builds an AI system and ships it to a customer, so it spends its time on discovery, integration and delivery. This one is about the platform underneath: it goes much deeper on the gateway, tracking, observability, cost, containers and CI/CD, which the forward deployed roadmap only touches at the end.