Which algorithms need scaling

A model needs feature scaling when it combines columns into a single number, through a distance or a gradient; a model that splits one column at a time does not.

Last updated: 12 Oct, 2026 · pandas 3.0 · scikit-learn 1.9 · scipy 1.18

Why features are scaled showed one column taking over a distance. The next question is which models that reaches. The answer is a short list, and the video treats it as an interview question worth memorising.

The two reasons a model cares about scale

• Gradient descent. Linear regression, logistic regression and neural networks search for the lowest point of a cost surface by stepping downhill. Columns of unequal width turn that surface into a long narrow valley, so the walk zig zags and needs more rounds.
• Euclidean distance. k nearest neighbours, k means and hierarchical clustering rank rows by distance, and the widest column supplies most of it.
• Neither applies to a tree. A decision tree asks one question at a time, ram <= 2100, and a question like that gives the same answer whatever units ram is written in. Random forest, AdaBoost and gradient boosting are built from such trees and inherit the same indifference.

Measuring the change on four models

The mobile price dataset makes the point at full size. It holds 2000 phones with 20 features whose ranges run from 0 and 1 up to ram at 256 to 3998, and a target with four price bands. Each model is scored twice on the same five folds: once on the raw columns, once inside a pipeline that standardises first.

import pandas as pd

URL = "https://raw.githubusercontent.com/krishnaik06/Feature-Engineering-Live-sessions/master/mobile_dataset.csv"
mobile = pd.read_csv(URL)                 # 2000 phones, 20 features
X = mobile.drop(columns='price_range')    # battery_power 501-1998, ram 256-3998
y = mobile['price_range']                 # four price bands, 500 phones each

from sklearn.model_selection import cross_val_score, StratifiedKFold
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

# the same five folds for every model, so the comparison is fair
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=0)

# make_pipeline fits the scaler on each training fold only, never on the test fold
scaled_model = make_pipeline(StandardScaler(), model)
score = cross_val_score(scaled_model, X, y, cv=cv).mean()

Scoring four models with and without a scaler

from sklearn.model_selection import cross_val_score, StratifiedKFold
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.neighbors import KNeighborsClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=0)
models = [("Logistic regression", LogisticRegression(max_iter=200)),
          ("KNN", KNeighborsClassifier()),
          ("Decision tree", DecisionTreeClassifier(random_state=0)),
          ("Random forest", RandomForestClassifier(n_estimators=50, random_state=0))]

for name, model in models:
    plain = cross_val_score(model, X, y, cv=cv).mean()
    scaled = cross_val_score(make_pipeline(StandardScaler(), model), X, y, cv=cv).mean()
    print(f"{name:20s} unscaled {plain:.3f}   scaled {scaled:.3f}   change {scaled - plain:+.3f}")

Logistic regression  unscaled 0.652   scaled 0.963   change +0.311
KNN                  unscaled 0.921   scaled 0.515   change -0.406
Decision tree        unscaled 0.827   scaled 0.827   change +0.001
Random forest        unscaled 0.873   scaled 0.872   change -0.002

Reading the four changes

• Logistic regression gains 0.311. This is the gradient descent case. On raw columns the solver never reaches the minimum inside its iteration budget.
• KNN loses 0.406. This is the distance case, and it moves in the other direction. On this dataset ram is both the widest column and the one that predicts the price band, so letting it dominate happens to help. On a table where the widest column is noise, the same arithmetic hurts instead.
• The decision tree moves by 0.001 and the forest by 0.002. Scaling a column is a monotonic rewrite, so the ordering of values, and therefore every split a tree can make, is unchanged. The tiny difference is floating point rounding changing which of two equally good splits is picked.
• The lesson is not that scaling raises accuracy. It is that the first two models give a different answer with and without it, so for them scaling is a decision; for the last two it is wasted work.

Counting the solver's rounds

The logistic regression result has a visible cause. n_iter_ reports how many rounds the solver used, and max_iter is the cap it is allowed.

from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

raw_fit = LogisticRegression(max_iter=200).fit(X, y)
scaled_fit = make_pipeline(StandardScaler(), LogisticRegression(max_iter=200)).fit(X, y)

print("rounds on raw columns   :", raw_fit.n_iter_[0])
print("rounds on scaled columns:", scaled_fit[-1].n_iter_[0])

rounds on raw columns   : 200
rounds on scaled columns: 23

• 200 rounds on the raw columns is the cap, not a stopping point. The solver was still moving when it ran out and prints a convergence warning.
• 23 rounds on the scaled columns is a real stop: the cost stopped improving. The same model, the same data, nine times fewer rounds.

Models that need scaling and models that do not

Where this decision shows up

• Choosing what goes in a pipeline. A pipeline ending in a forest does not need a scaler step. Adding one costs fitting time on every fold and buys nothing.
• Comparing models fairly. Scoring KNN on raw columns against a forest on raw columns is not a model comparison, because only one of them was given a fair chance.
• Reading a convergence warning. A solver that hits max_iter is usually reporting unscaled inputs rather than a hard problem.

Related

• Previous: Why features are scaled
• Next: Standardisation
• See also: Pipelines and pandas output
• Reference: Preprocessing data in the scikit-learn docs