Project: the whole thing as one pipeline

The finished project is one fitted Pipeline: six steps that take a raw house row, with its gaps and its text, and return a price, with every number in them learned from the training houses.

Last updated: 12 Oct, 2026 · pandas 3.0 · scikit-learn 1.9

Project: the feature selection ended with a score and a list of 21 columns, spread across a dozen variables in a notebook. Serving that means repeating every line in the right order on every new house. This lesson packs the same steps into one object, and checks that it keeps the same 21 columns and the same score.

Reusing the two steps that learn nothing

flags and ages_and_logs read one row at a time and hold no state, so they go into the pipeline unchanged, wrapped in FunctionTransformer, which turns a plain function into a step.

NANCOLS = ['LotFrontage', 'MasVnrArea', 'GarageYrBlt']
YEARS = ['YearBuilt', 'YearRemodAdd', 'GarageYrBlt']
LOGS = ['LotFrontage', 'LotArea', '1stFlrSF', 'GrLivArea']

def flags(frame):
    frame = frame.copy()
    for c in NANCOLS:
        frame[c + 'nan'] = frame[c].isnull().astype(int)
    return frame

def ages_and_logs(frame):
    frame = frame.copy()
    for c in YEARS:
        frame[c] = frame['YrSold'] - frame[c]
    for c in LOGS:
        frame[c] = np.log(frame[c])
    return frame

Writing the target guided encoder as a step

The text side is different. Grouping rare labels and coding them by the training price learns something, so it has to be an object with a fit and a transform. scikit-learn has no transformer for this exact recipe, and writing one is about twenty lines: BaseEstimator gives it the parameter handling every scikit-learn object has, and TransformerMixin gives it fit_transform for free.

class TargetGuidedOrdinal(BaseEstimator, TransformerMixin):
    """Rare labels into one group, then a code per label, ordered by the training mean."""

    def __init__(self, rare=0.01):
        self.rare = rare

What fit stores

For each column: the labels that cleared the one percent share, and a dictionary from label to code built from the training target. The trailing underscore in keep_ and codes_ is the scikit-learn convention for an attribute that only exists after a fit.

    def fit(self, X, y):
        self.keep_, self.codes_ = {}, {}
        target = pd.Series(np.asarray(y), index=X.index)
        for c in X.columns:
            share = X[c].value_counts(normalize=True)
            self.keep_[c] = share[share > self.rare].index
            grouped = np.where(X[c].isin(self.keep_[c]), X[c], 'Rare_var')
            order = target.groupby(grouped).mean().sort_values().index
            self.codes_[c] = {label: i for i, label in enumerate(order)}
        return self

What transform applies

The same two operations, with nothing learned: group against the stored list, map against the stored dictionary, and code anything the map does not know as -1. get_feature_names_out is what lets the column names survive the ColumnTransformer, so the selected 21 can be named at the end.

    def transform(self, X):
        out = X.copy()
        for c in X.columns:
            grouped = np.where(X[c].isin(self.keep_[c]), X[c], 'Rare_var')
            out[c] = (pd.Series(grouped, index=X.index)
                      .map(self.codes_[c]).fillna(-1).astype(float))
        return out

    def get_feature_names_out(self, names=None):
        return np.asarray(list(self.keep_))

Routing the two kinds of column

The numeric columns need the median fill; the text columns need the Missing label and then the encoder. verbose_feature_names_out=False drops the num__ and cat__ prefixes, so the frame that comes out still has YrSold and GrLivArea under their own names, which the next step needs.

text_steps = Pipeline([
    ('fill', SimpleImputer(strategy='constant', fill_value='Missing')),
    ('code', TargetGuidedOrdinal()),
])
by_column = ColumnTransformer([
    ('num', SimpleImputer(strategy='median'), numbers),
    ('cat', text_steps, words),
], verbose_feature_names_out=False)

The six steps in order

Flags before the fill, the fill before the ages, the ages before the scaler, the scaler before the selection, and the model last. The same order the feature pass ran by hand, now written once in a list.

house_prices = Pipeline([
    ('flags', FunctionTransformer(flags)),
    ('columns', by_column),
    ('derive', FunctionTransformer(ages_and_logs)),
    ('scale', MinMaxScaler()),
    ('select', SelectFromModel(Lasso(alpha=0.005, random_state=0))),
    ('lasso', Lasso(alpha=0.005, random_state=0)),
])

Fitting the object and predicting a raw house

import numpy as np
import pandas as pd
from sklearn import set_config
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.compose import ColumnTransformer
from sklearn.feature_selection import SelectFromModel
from sklearn.impute import SimpleImputer
from sklearn.linear_model import Lasso
from sklearn.metrics import r2_score
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import FunctionTransformer, MinMaxScaler

set_config(transform_output="pandas")

AMES = ("https://raw.githubusercontent.com/krishnaik06/"
        "Advanced-House-Price-Prediction-/master/train.csv")
houses = pd.read_csv(AMES)
y = np.log(houses['SalePrice'])
X = houses.drop(columns=['Id', 'SalePrice'])
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.1, random_state=0)

NANCOLS = ['LotFrontage', 'MasVnrArea', 'GarageYrBlt']
YEARS = ['YearBuilt', 'YearRemodAdd', 'GarageYrBlt']
LOGS = ['LotFrontage', 'LotArea', '1stFlrSF', 'GrLivArea']

def flags(frame):
    frame = frame.copy()
    for c in NANCOLS:
        frame[c + 'nan'] = frame[c].isnull().astype(int)
    return frame

def ages_and_logs(frame):
    frame = frame.copy()
    for c in YEARS:
        frame[c] = frame['YrSold'] - frame[c]
    for c in LOGS:
        frame[c] = np.log(frame[c])
    return frame

class TargetGuidedOrdinal(BaseEstimator, TransformerMixin):
    """Rare labels into one group, then a code per label, ordered by the training mean."""

    def __init__(self, rare=0.01):
        self.rare = rare

    def fit(self, X, y):
        self.keep_, self.codes_ = {}, {}
        target = pd.Series(np.asarray(y), index=X.index)
        for c in X.columns:
            share = X[c].value_counts(normalize=True)
            self.keep_[c] = share[share > self.rare].index
            grouped = np.where(X[c].isin(self.keep_[c]), X[c], 'Rare_var')
            order = target.groupby(grouped).mean().sort_values().index
            self.codes_[c] = {label: i for i, label in enumerate(order)}
        return self

    def transform(self, X):
        out = X.copy()
        for c in X.columns:
            grouped = np.where(X[c].isin(self.keep_[c]), X[c], 'Rare_var')
            out[c] = (pd.Series(grouped, index=X.index)
                      .map(self.codes_[c]).fillna(-1).astype(float))
        return out

    def get_feature_names_out(self, names=None):
        return np.asarray(list(self.keep_))

numbers = [c for c in flags(X_train).columns
           if pd.api.types.is_numeric_dtype(flags(X_train)[c])]
words = [c for c in X_train.columns if pd.api.types.is_string_dtype(X_train[c])]

text_steps = Pipeline([
    ('fill', SimpleImputer(strategy='constant', fill_value='Missing')),
    ('code', TargetGuidedOrdinal()),
])
by_column = ColumnTransformer([
    ('num', SimpleImputer(strategy='median'), numbers),
    ('cat', text_steps, words),
], verbose_feature_names_out=False)

house_prices = Pipeline([
    ('flags', FunctionTransformer(flags)),
    ('columns', by_column),
    ('derive', FunctionTransformer(ages_and_logs)),
    ('scale', MinMaxScaler()),
    ('select', SelectFromModel(Lasso(alpha=0.005, random_state=0))),
    ('lasso', Lasso(alpha=0.005, random_state=0)),
])

house_prices.fit(X_train, y_train)

names = np.asarray(house_prices.named_steps['columns'].get_feature_names_out())
kept = sorted(names[house_prices.named_steps['select'].get_support()].tolist())
print("features kept:", len(kept), "of", len(names))
print(kept)
print("test R2:", round(r2_score(y_test, house_prices.predict(X_test)), 4))

one_house = X_test.iloc[[0]]
print("columns in the raw row:", one_house.shape[1],
      "of which empty:", int(one_house.isnull().sum().sum()))
log_price = float(house_prices.predict(one_house)[0])
print("predicted log price:", round(log_price, 4))
print("predicted:", round(float(np.exp(log_price))),
      "actual:", int(houses.loc[one_house.index[0], 'SalePrice']))

features kept: 21 of 82
['1stFlrSF', 'BsmtExposure', 'BsmtFullBath', 'BsmtQual', 'CentralAir', 'FireplaceQu', 'Fireplaces', 'GarageCars', 'GarageFinish', 'GarageType', 'GrLivArea', 'HeatingQC', 'KitchenQual', 'MSSubClass', 'MSZoning', 'Neighborhood', 'OverallCond', 'OverallQual', 'PavedDrive', 'RoofStyle', 'YearRemodAdd']
test R2: 0.8592
columns in the raw row: 79 of which empty: 7
predicted log price: 12.3196
predicted: 224041 actual: 200624

• The same 21 columns, and the same 0.8592. The pipeline is not a different model. It is the feature pass, the scaler and the selection from the two previous lessons, in the same order, with the same seed, which is why the numbers match to the digit.
• The raw row has 79 columns and seven of them are empty. No filling, no coding and no scaling happened before predict. That is what a single object buys: the caller hands over a row of the original file.
• 224,041 dollars against 200,624. One house, about twelve percent out. The R squared is the honest summary across all 146 held back houses; a single row is a reminder that a model is a distribution, not an oracle.

What happens when a column is missing from the row

The object expects the columns it was fitted on. A row that lost one, which is what happens the first time a form or an upstream table changes, fails at the ColumnTransformer with the name of the column, before any arithmetic happens.

one_house = X_test.iloc[[0]]
house_prices.predict(one_house.drop(columns=['YrSold']))

Traceback (most recent call last):
  File "main.py", line 2, in <module>
    house_prices.predict(one_house.drop(columns=['YrSold']))
ValueError: columns are missing: {'YrSold'}

The error names YrSold, which is the column the ages step subtracts from. Failing here is the right behaviour: a silent fill would have produced a price, and nobody would know the row was incomplete.

The hand written pass against the one object

What a finished object is used for

• Serving. A web request carries a row of the raw shape, and predict answers it. Nothing in the serving code knows what a median or a rare label is.
• Honest comparison. Two candidate pipelines differing in one step can be put through cross_val_score and compared without either of them seeing the other's folds.
• Retraining. When next year's sales arrive, the same object is fitted again and every number inside it moves together, instead of some of them being updated and some forgotten.

What this course did not cover

Related

• Previous: Project: the feature selection
• See also: Pipelines and pandas output and Data leakage
• Reference: Developing scikit-learn estimators, which defines the fit and transform contract a custom step follows