Config files, not constants

A config file is a YAML file a stage reads its settings from at start-up, so a folder, a source URL or a hyperparameter is a line of data in the project rather than a literal typed into the code.

Last updated: 19 Oct, 2026 · MLOps

The logger and the exception type from Custom exceptions are the project's two shared services. The values a stage works with are the next thing to move out of the code. The project keeps three YAML files for them at the top of the repository, and the video's workflow list starts with the same instruction: update config.yaml first, then params.yaml, and only then write a line of stage code.

Why a folder name does not belong in the code

A training script that says pd.read_csv("artifacts/data_ingestion/stud.csv") has one source of truth for that path, and it is a Python file. Moving the data means editing code, re-reading the diff and redeploying. Worse, the same string is usually typed in three places, so one of them is always stale. A value kept in a YAML file is edited in one place, shows up as a one-line diff in review, and can be changed by someone who never opens the source.

The clip's config.yaml describes the deep-learning project, where a stage downloads and unzips an archive; this one describes the four student-performance stages, and the block-per-stage shape is the same.

Describing a stage in config.yaml

config/config.yaml holds the wiring: where a stage writes, and the names of the files it reads and produces. Each stage gets its own block of keys, named after the stage, so nothing is shared by accident.

data_ingestion:
  root_dir: artifacts/data_ingestion
  source_url: https://raw.githubusercontent.com/krishnaik06/mlproject/main/notebook/data/stud.csv
  local_data_file: artifacts/data_ingestion/stud.csv
  train_file: artifacts/data_ingestion/train.csv
  test_file: artifacts/data_ingestion/test.csv

root_dir is the folder the stage owns, and every other path in the block sits inside it. The three remaining stages are described the same way, each with its own root_dir and the file it leaves behind for the next one.

data_validation:
  root_dir: artifacts/data_validation
  status_file: artifacts/data_validation/status.json

data_transformation:
  root_dir: artifacts/data_transformation
  preprocessor_file: artifacts/data_transformation/preprocessor.joblib

model_trainer:
  root_dir: artifacts/model_trainer
  model_file: artifacts/model_trainer/model.joblib
  metrics_file: artifacts/model_trainer/metrics.json

Keeping the numbers in params.yaml

params.yaml holds the numbers of the run: the target column, the split size, the seed and the model's hyperparameters. They live apart from the wiring because they are what you change when you are tuning rather than re-plumbing, and because a pipeline tool can watch this one file and rerun training when a value in it moves.

TARGET_COLUMN: math_score
TEST_SIZE: 0.2
RANDOM_STATE: 42
N_ESTIMATORS: 100
MAX_DEPTH: 12

The keys are upper case by convention, which makes a params value easy to spot in stage code: self.params.MAX_DEPTH reads as a knob, c.root_dir reads as wiring.

Writing the data's shape in schema.yaml

schema.yaml holds the shape the data is expected to have: one entry per column, with the pandas dtype the rest of the project assumes. It is read by the stage that checks incoming data before any of it reaches the model, which is the subject of Data validation.

COLUMNS:
  gender: object
  race_ethnicity: object
  parental_level_of_education: object
  lunch: object
  test_preparation_course: object
  math_score: int64
  reading_score: int64
  writing_score: int64

Reading the three files

All three are ordinary YAML, so yaml.safe_load turns each one into a nested dictionary. Run this from the project root, where the three files are.

import yaml

cfg = yaml.safe_load(open("config/config.yaml"))
params = yaml.safe_load(open("params.yaml"))
schema = yaml.safe_load(open("schema.yaml"))

# one block of keys per stage, so a stage reads only its own settings
print("stages  :", list(cfg))
print("root_dir:", cfg["data_ingestion"]["root_dir"])
print("depth   :", params["MAX_DEPTH"])
print("columns :", len(schema["COLUMNS"]))

stages  : ['data_ingestion', 'data_validation', 'data_transformation', 'model_trainer']
root_dir: artifacts/data_ingestion
depth   : 12
columns : 8

What came out of the three files

• The four stage names are the top-level keys of config.yaml, so the stage that wants its paths asks for one of those four blocks and gets nothing belonging to another stage.
• The ingestion folder came back as a string, artifacts/data_ingestion, read from the file rather than typed in the code that is about to create it.
• MAX_DEPTH came from params.yaml as the integer 12, because YAML keeps number types, so a hyperparameter needs no casting on the way in.
• schema.yaml described eight columns, which is every column of the raw CSV, so a file that arrives with seven or nine can be caught before training.

A constant in code against a value in config

Where the three files earn their keep

• Moving the artifacts folder, where changing root_dir in one block moves a whole stage's output without touching a line of Python.
• Tuning a model, where a run with MAX_DEPTH: 4 and a run with MAX_DEPTH: 12 differ by a single value in params.yaml, which is also what makes the two runs comparable later.
• Catching a changed CSV, where a column renamed upstream fails against schema.yaml instead of failing deep inside the trainer.

Related

• Previous: Custom exceptions
• Next: Reading config with ConfigBox
• See also: dvc.yaml stages, deps and outs
• Reference: The YAML 1.2 specification