The operated model service

The operated model service is this course in one program: the pipeline trains a model, the API serves it, a bad request is refused before the model is reached, and a drift check on the live window says whether the whole thing needs doing again. It runs end to end, and it runs on nothing but what the course taught.

Last updated: 19 Oct, 2026 · MLOps

What comes after this course drew the border around the course. Inside that border is a promise made at the start: take a trained model out of the notebook and operate it, with tracked runs and a registry behind it, a served API in a container, and the drift you watch after it ships. This is that promise, executed.

What the capstone does, in order

• Trains, and records the run. The stud.csv data is split 800 / 200, the ColumnTransformer is fitted on the training rows only, and the forest is fitted on top of it inside an MLflow run that keeps its params, its score and the model itself.
• Registers. The logged model becomes a version under one registered name, and the champion alias is pointed at it, so the serving side can ask for a model by name instead of by run id.
• Serves. A FastAPI app wraps the fitted objects behind /health and /predict, with the request schema the service validates against.
• Is called. One record goes in, the first row of the dataset, and a predicted maths score comes back.
• Refuses. The same record with a reading score of 150 is rejected before the forest is reached.
• Checks for drift. The window the model learned from is compared against the window arriving now, on the input columns only, and the share decides what happens next.

Training the model the service will answer with

This is the training pipeline's shape, condensed into one place: an imputer and a one-hot encoder for the text columns, an imputer and a scaler for the scores, and a 100-tree forest at a depth of 12 with seed 42. The preprocessor is fitted on the training rows and only transforms the test rows.

# the preprocessor is fitted on the training rows only, so the test rows stay unseen
pre = ColumnTransformer([
    ("cat", Pipeline([("impute", SimpleImputer(strategy="most_frequent")),
                      ("encode", OneHotEncoder(handle_unknown="ignore", sparse_output=False))]),
     Xtr.select_dtypes(include="object").columns.tolist()),
    ("num", Pipeline([("impute", SimpleImputer(strategy="median")),
                      ("scale", StandardScaler())]),
     Xtr.select_dtypes(exclude="object").columns.tolist()),
])

model = RandomForestRegressor(n_estimators=100, max_depth=12, random_state=42)

Recording the run and pointing the champion alias at it

The fit belongs inside a tracked run, or the number the service answers with has no record behind it. The three lines of setup are the ones every tracked run in this course starts with, and the store is emptied first so the capstone always registers version 1.

import os, shutil

os.environ["MLFLOW_ALLOW_FILE_STORE"] = "true"   # 3.17 refuses a local file store without this
shutil.rmtree("mlruns", ignore_errors=True)      # the capstone starts from an empty store

import mlflow
from mlflow.tracking import MlflowClient

mlflow.set_tracking_uri("file:./mlruns")
mlflow.set_experiment("student-performance")
NAME = "student-performance-rf"

# the fit happens inside the run, so the params and the score are recorded with it
with mlflow.start_run(run_name="capstone"):
    mlflow.log_params({"n_estimators": 100, "max_depth": 12, "random_state": 42})
    model.fit(pre.fit_transform(Xtr), train.math_score)
    r2 = round(r2_score(test.math_score, model.predict(pre.transform(Xte))), 4)

    # the model goes into the run too; skops refuses a tree model without the trusted type
    mlflow.log_metrics({"r2": r2})
    info = mlflow.sklearn.log_model(model, name="model",
                                    skops_trusted_types=["sklearn.tree._tree.Tree"])

# a version under one name, and an alias the serving side can ask for instead of a run id
version = mlflow.register_model(info.model_uri, NAME).version
MlflowClient().set_registered_model_alias(NAME, "champion", version)

# loading through the alias proves the stored model is the one the run measured
champion = mlflow.sklearn.load_model(f"models:/{NAME}@champion")

Putting the API in front of it

Two endpoints: one that says the service is up, one that takes a record and answers with a score. The record is a Pydantic model, so a malformed request never reaches the line that calls the forest.

@api.get("/health")
def health():
    return {"status": "ok"}

@api.post("/predict")
def predict(record: StudentRecord):
    # one validated record becomes a one-row frame the preprocessor understands
    row = pd.DataFrame([record.model_dump()])
    return {"math_score": round(float(model.predict(pre.transform(row))[0]), 2)}

Checking the live window for drift

The same two windows as the drift lesson, wrapped as Datasets and run through the preset. The number that matters at the end is the share of columns that moved.

# the monitor reads only what a request carries, so math_score, the target, is not in it
COLS = ["reading_score", "writing_score", "gender", "lunch"]
definition = DataDefinition(numerical_columns=COLS[:2], categorical_columns=COLS[2:])

reference = Dataset.from_pandas(df[df.test_preparation_course == "none"][COLS],
                                data_definition=definition)
current = Dataset.from_pandas(df[df.test_preparation_course == "completed"][COLS],
                              data_definition=definition)

The whole service in one run

One file, start to finish. Save it as capstone.py and run it with the course's packages installed; it downloads its own data and needs no key, no account and no artifacts on disk. That is why it fits the model in the same process instead of reading the artifacts a dvc repro left behind: a reader who has not built the project can still run every line of it, and the versioned inputs and the built image are the pieces their own lessons already proved.

import os, shutil

os.environ["MLFLOW_ALLOW_FILE_STORE"] = "true"   # 3.17 refuses a local file store without this
shutil.rmtree("mlruns", ignore_errors=True)      # the capstone starts from an empty store

import mlflow
import pandas as pd
from evidently import DataDefinition, Dataset, Report
from evidently.presets import DataDriftPreset
from fastapi import FastAPI
from fastapi.testclient import TestClient
from mlflow.tracking import MlflowClient
from pydantic import BaseModel, Field
from sklearn.compose import ColumnTransformer
from sklearn.ensemble import RandomForestRegressor
from sklearn.impute import SimpleImputer
from sklearn.metrics import r2_score
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

mlflow.set_tracking_uri("file:./mlruns")
mlflow.set_experiment("student-performance")
NAME = "student-performance-rf"

URL = "https://raw.githubusercontent.com/krishnaik06/mlproject/main/notebook/data/stud.csv"
df = pd.read_csv(URL)
train, test = train_test_split(df, test_size=0.2, random_state=42)
Xtr, Xte = train.drop(columns=["math_score"]), test.drop(columns=["math_score"])

# the preprocessor is fitted on the training rows only, so the test rows stay unseen
pre = ColumnTransformer([
    ("cat", Pipeline([("impute", SimpleImputer(strategy="most_frequent")),
                      ("encode", OneHotEncoder(handle_unknown="ignore", sparse_output=False))]),
     Xtr.select_dtypes(include="object").columns.tolist()),
    ("num", Pipeline([("impute", SimpleImputer(strategy="median")),
                      ("scale", StandardScaler())]),
     Xtr.select_dtypes(exclude="object").columns.tolist()),
])

model = RandomForestRegressor(n_estimators=100, max_depth=12, random_state=42)

# the fit happens inside a tracked run, so the params, the score and the model are recorded
with mlflow.start_run(run_name="capstone"):
    mlflow.log_params({"n_estimators": 100, "max_depth": 12, "random_state": 42})
    model.fit(pre.fit_transform(Xtr), train.math_score)
    r2 = round(r2_score(test.math_score, model.predict(pre.transform(Xte))), 4)
    mlflow.log_metrics({"r2": r2})
    info = mlflow.sklearn.log_model(model, name="model",
                                    skops_trusted_types=["sklearn.tree._tree.Tree"])
print("trained      r2", r2)

# the registry and the alias: the version other code asks for by name, not by run id
version = mlflow.register_model(info.model_uri, NAME).version
MlflowClient().set_registered_model_alias(NAME, "champion", version)
champion = mlflow.sklearn.load_model(f"models:/{NAME}@champion")
print("registered   version", version)
print("champion     r2", round(r2_score(test.math_score, champion.predict(pre.transform(Xte))), 4))


class StudentRecord(BaseModel):
    gender: str
    race_ethnicity: str
    parental_level_of_education: str
    lunch: str
    test_preparation_course: str
    reading_score: int = Field(ge=0, le=100)
    writing_score: int = Field(ge=0, le=100)


api = FastAPI(title="Student Performance API")


@api.get("/health")
def health():
    return {"status": "ok"}


@api.post("/predict")
def predict(record: StudentRecord):
    row = pd.DataFrame([record.model_dump()])
    return {"math_score": round(float(model.predict(pre.transform(row))[0]), 2)}


client = TestClient(api)
RECORD = {"gender": "female", "race_ethnicity": "group B",
          "parental_level_of_education": "bachelor's degree", "lunch": "standard",
          "test_preparation_course": "none", "reading_score": 72, "writing_score": 74}
print("health      ", client.get("/health").json())
print("predict     ", client.post("/predict", json=RECORD).json())

# the failure mode: a score outside the range the schema allows never reaches the model
bad = client.post("/predict", json={**RECORD, "reading_score": 150})
print("bad request ", bad.status_code, bad.json()["detail"][0]["msg"])

# the drift check reads only what a request carries, so math_score, the target, is not in it
COLS = ["reading_score", "writing_score", "gender", "lunch"]
definition = DataDefinition(numerical_columns=COLS[:2], categorical_columns=COLS[2:])
reference = Dataset.from_pandas(df[df.test_preparation_course == "none"][COLS],
                                data_definition=definition)
current = Dataset.from_pandas(df[df.test_preparation_course == "completed"][COLS],
                              data_definition=definition)
report = Report(metrics=[DataDriftPreset()]).run(current_data=current, reference_data=reference)
print("drift       ", report.dict()["metrics"][0]["value"])

trained      r2 0.8485
registered   version 1
champion     r2 0.8485
health       {'status': 'ok'}
predict      {'math_score': 69.34}
bad request  422 Input should be less than or equal to 100
drift        {'count': 2.0, 'share': 0.5}

Reading the seven lines it printed

• r2 0.8485 on the 200 held-out rows is the pipeline's number, the same one the training pipeline and the metrics file carry. The capstone did not get a different model by being written in one file.
• Version 1 of student-performance-rf is the registered model the run produced. It is version 1 on every run because the store is emptied first; in a real project the number climbs and the alias is what moves.
• The champion re-scores 0.8485. That model was serialised, written to the store, and loaded back through the alias rather than used from memory, so the registry is holding the model the run measured and not a near miss.
• /health answered next. It is the endpoint a load balancer calls, and it deliberately touches nothing: if it needed the model it could not tell you the process was alive while the model was broken.
• The prediction is 69.34 for the first row of the dataset, whose true maths score is 72. One record, one answer, through validation and the preprocessor.
• The bad request came back 422 with the message the schema generated. No prediction was made, which is the point.
• The drift check found a count of 2 and a share of 0.5. Two of the four input columns moved between the two windows, which lands exactly on the 0.5 the preset compares against. One column either way decides it, so this is a report an operator reads rather than a switch that fires a retrain.

The failure mode, and why it is the part that matters

A demo that only shows the happy path is not evidence that a service works. The interesting line in that output is the 422: a reading score of 150 is not a possible mark, and the bound on the field stopped it. FastAPI checks the request against the schema before the endpoint body runs, so the forest never saw the record and no number was invented for it.

Compare the two outcomes. Without the bound, that record would have been one-hot encoded, scaled against a distribution it does not belong to, and scored; the service would have returned a confident number with nothing behind it, and the only trace would have been a strange value in a log. With the bound, the caller is told what is wrong, in the field that is wrong, and the rejected request becomes a signal on the monitoring board rather than a silent bad answer.

The same model, answering from a container

The run above calls the app in the same process. The deployed version is the image the container lessons built: the artifacts and the API inside python:3.12-slim, started with uvicorn and reached over a port. The record is the one the capstone sent, so the two answers are comparable.

$ docker run -d --name sp-test -p 8077:8000 student-performance:v1
$ curl -s http://localhost:8077/health
{"status":"ok"}

# the same record the capstone sent, this time over HTTP into the container
$ curl -s -X POST http://localhost:8077/predict -H "Content-Type: application/json" \
    -d '{"gender":"female","race_ethnicity":"group B","parental_level_of_education":"bachelor'\''s degree","lunch":"standard","test_preparation_course":"none","reading_score":72,"writing_score":74}'
{"math_score":69.34}
$ docker rm -f sp-test

69.34 from the container and 69.34 from the local run, for the same record. That identity is the proof the whole course has been working towards: the thing you tested and the thing that is deployed are the same model, with the same preprocessing, answering the same way. A container that gave a different number would mean an artifact mismatch, and it is worth checking on every deploy.

What an operator does when the share crosses the line

• Look at which columns moved. Both score columns moving together is a population change; one column moving alone is more often a broken feed, and the fix is upstream, not a retrain.
• Check the error rate if any true values have arrived. Drifted inputs with a steady error rate means the model is coping, and a retrain can wait for the schedule.
• Let the weekly DAG fit a candidate. It runs ingest and train on the window that now includes the new records, exactly as the retraining lesson set up.
• Compare before promoting. Candidate against champion on the same held-out rows; the alias moves only if the candidate wins.
• Write down what the window was. The data version and the thresholds that triggered the run belong with the metrics, or the next person cannot tell why this model exists.

Where this course ends and LLMOps begins

Everything here rests on one assumption: you trained the model, so you have its data, its metrics and its artifact. An application built on a hosted language model breaks that assumption. There is no training run of your own, the output is text rather than a number, correctness is a judgement rather than an error metric, and the cost is per call. The lifecycle survives; most of the tooling changes. From MLOps to LLMOps: what changes picks the thread up from exactly here.

Related

• See also: Dockerising the service
• See also: From MLOps to LLMOps: what changes