Comparing runs in the MLflow UI

Comparing runs is reading two or more records of the same experiment side by side, so the effect of the one setting you changed is visible instead of guessed.

Last updated: 19 Oct, 2026 · MLOps

The Logging a model with MLflow lesson completed one run. One run answers nothing. The video changes a single parameter, runs the pipeline again, and then opens the MLflow UI to put the runs next to each other, selecting several and reading the parallel coordinates plot to find the better model. This lesson does the same comparison twice: with the UI, and with search_runs, which prints the comparison where you are working.

The compare view was relaid out for MLflow 3, so the screen differs from the one you will see; selecting runs and reading the metrics off the table works the same way.

Starting the MLflow UI

The UI is a small web server reading the same store your code writes. Point it at the folder and open the address it prints, which is http://127.0.0.1:5000 unless you pass another port. It serves the experiment list, one page per run, and a compare view when you select several runs and choose Compare.

# serve the UI from the same store the code logged to, then open the address it prints
MLFLOW_ALLOW_FILE_STORE=true mlflow ui --backend-store-uri ./mlruns

The variable has to be set for the server too. Setting it with os.environ inside the training script only reaches that process, so the terminal running the UI needs its own. On Windows use set MLFLOW_ALLOW_FILE_STORE=true in Command Prompt, or $env:MLFLOW_ALLOW_FILE_STORE="true" in PowerShell, then the mlflow ui line on its own.

In the compare view, picking parameters and metrics draws the parallel coordinates plot the video reads: one line per run across the settings and the metrics, which is how a sweep of many runs is scanned at a glance. The UI is a server rather than something a script prints, so the comparison below is the one you can also read without leaving your code.

Pointing MLflow at a local store

The same two required lines open every tracked run: the opt-in MLflow 3.17 wants before it will use a file store, and the tracking URI that names the folder.

import os, shutil

os.environ["MLFLOW_ALLOW_FILE_STORE"] = "true"   # 3.17 refuses a local file store without this
shutil.rmtree("mlruns", ignore_errors=True)      # each lesson starts from an empty store
os.makedirs("mlruns/.trash", exist_ok=True)      # MLflow keeps deleted runs in .trash

import mlflow
mlflow.set_tracking_uri("file:./mlruns")
mlflow.set_experiment("student-performance")

Logging two runs one setting apart

The loop trains the forest twice, at depth 12 and at depth 4, and logs each as its own run under the same experiment. Everything else, including the seed, is held still, so the difference between the two rows has one cause.

# two runs, one setting apart, so the comparison carries meaning
for depth in (12, 4):
    model, scores = fit(max_depth=depth)
    with mlflow.start_run(run_name=f"depth-{depth}"):
        mlflow.log_params({"n_estimators": 100, "max_depth": depth, "random_state": 42})
        mlflow.log_metrics(scores)

runs = mlflow.search_runs(experiment_names=["student-performance"])
cols = ["params.max_depth", "metrics.r2", "metrics.mae", "metrics.rmse"]
print(runs[cols].to_string(index=False))
print("best r2 at max_depth:", runs.loc[runs["metrics.r2"].idxmax(), "params.max_depth"])

params.max_depth  metrics.r2  metrics.mae  metrics.rmse
               4      0.8374       4.8467        6.2904
              12      0.8485       4.6996        6.0721
best r2 at max_depth: 12

Reading the comparison

• Depth 12 scored r2 0.8485 and depth 4 scored 0.8374. The shallower forest is worse on every metric, which is the answer the comparison was for.
• MAE and RMSE move with r2, 4.6996 against 4.8467 and 6.0721 against 6.2904, so the ranking does not depend on which of the three you read.
• Newest first. search_runs returns the most recent run at the top, so the depth-4 row comes first even though depth 12 ran first.
• The best row is found by a query, not by eye: idxmax over the r2 column names the winning configuration, which is the same thing the parallel coordinates plot shows visually.

The UI against search_runs

Where a comparison decides something

• Choosing a configuration, where the sweep's rows are sorted by one metric and the top row is what you register.
• Reviewing a change, where a teammate's run is read beside yours with the parameters visible.
• Guarding a retrain, where a script compares the new run against the current one and stops if the metric dropped.

Related

• Previous: Logging a model with MLflow
• Next: The model registry
• Reference: The MLflow UI