Oversampling and SMOTE

Oversampling is balancing a training set by adding minority rows rather than removing majority ones: RandomOverSampler copies rows that exist, and SMOTE invents new ones between them.

Last updated: 12 Oct, 2026 · pandas 3.0 · scikit-learn 1.9

Undersampling balanced the training set by cutting 3430 majority rows. Oversampling reaches the same balance from the other end and keeps every row it was given.

Syntax: the same two-line shape as the under-samplers, from a different module.

from imblearn.over_sampling import RandomOverSampler, SMOTE

over = RandomOverSampler(sampling_strategy="auto", random_state=42)
smote = SMOTE(sampling_strategy="auto", random_state=42, k_neighbors=5)
X_res, y_res = smote.fit_resample(X_train, y_train)

Copying rows with RandomOverSampler

RandomOverSampler draws minority rows with replacement until the counts match. Nothing new is created: the 35 minority rows appear again and again until there are 3465 of them.

from imblearn.over_sampling import RandomOverSampler

over = RandomOverSampler(random_state=42)
X_over, y_over = over.fit_resample(X_train, y_train)

import numpy as np
from collections import Counter
from imblearn.over_sampling import RandomOverSampler

over = RandomOverSampler(random_state=42)
X_over, y_over = over.fit_resample(X_train, y_train)

print("after oversampling:", Counter(y_over.tolist()))
minority = X_over[y_over == 1]
print("minority rows now:", len(minority))
print("distinct minority rows:", len(np.unique(minority, axis=0)))

after oversampling: Counter({0: 3465, 1: 3465})
minority rows now: 3465
distinct minority rows: 35

• The counts are equal at 3465 each, so the model now sees a balanced world.
• Only 35 of those 3465 rows are distinct. Each real minority row appears about a hundred times.
• No information was added. A linear model trained on a row repeated a hundred times behaves as if that row carried a hundred times the weight, which is the connection to the class_weight argument.

Inventing rows with SMOTE

SMOTE stands for Synthetic Minority Over-sampling Technique. Instead of copying a minority row, it picks one, looks at its k_neighbors nearest minority rows, chooses one of them at random and places a new row at a random point on the straight line between the two.

from imblearn.over_sampling import SMOTE

smote = SMOTE(random_state=42, k_neighbors=5)   # not n_neighbors
X_smote, y_smote = smote.fit_resample(X_train, y_train)

The argument is k_neighbors. Passing n_neighbors raises TypeError: SMOTE.__init__() got an unexpected keyword argument 'n_neighbors'. Did you mean 'k_neighbors'?. SMOTE also needs at least k_neighbors minority rows to interpolate between, so a class with three rows and the default of five fails.

import numpy as np
import matplotlib.pyplot as plt
from imblearn.over_sampling import SMOTE

smote = SMOTE(random_state=42, k_neighbors=5)
X_smote, y_smote = smote.fit_resample(X_train, y_train)
invented = X_smote[len(X_train):]
majority = X_train[y_train == 0]

rng = np.random.default_rng(0)        # 700 of each, so the dots stay readable
shown_majority = majority[rng.choice(len(majority), 700, replace=False)]
shown_invented = invented[rng.choice(len(invented), 700, replace=False)]

plt.figure(figsize=(6, 4.5))
plt.scatter(shown_majority[:, 0], shown_majority[:, 1], s=6, color="#dfe3ec",
            label=f"class 0 (700 of {len(majority)} drawn)")
plt.scatter(shown_invented[:, 0], shown_invented[:, 1], s=10, color="#e08a1e",
            label=f"invented class 1 (700 of {len(invented)} drawn)")
plt.scatter(X_train[y_train == 1, 0], X_train[y_train == 1, 1], s=40,
            color="#d64541", marker="x", label="real class 1 (35)")
plt.xlabel("feature 1")
plt.ylabel("feature 2")
plt.title("SMOTE fills in the minority region")
plt.legend(loc="upper left", fontsize=8)
plt.show()
print("rows invented:", len(invented))

rows invented: 3430

• The orange dots form straight segments. Each one is a line between two real minority rows, which is exactly what the interpolation does.
• Nothing was invented outside the real rows' territory. SMOTE cannot imagine a minority row in a region where none was seen.
• The band is narrow. With 35 real rows spread thinly, the invented rows cluster along a few long segments rather than filling an area.

Scoring the two over-samplers

from sklearn.linear_model import LogisticRegression
from sklearn.metrics import precision_score, recall_score, accuracy_score, f1_score

half = SMOTE(sampling_strategy=0.5, random_state=42)

for name, data in [("none", (X_train, y_train)),
                   ("RandomOverSampler", (X_over, y_over)),
                   ("SMOTE", (X_smote, y_smote)),
                   ("SMOTE 0.5", half.fit_resample(X_train, y_train))]:
    Xa, ya = data
    p = LogisticRegression(max_iter=1000).fit(Xa, ya).predict(X_test)
    print(f"{name:18s} rows {len(ya):5d}  accuracy {accuracy_score(y_test, p):.4f}"
          f"  precision {precision_score(y_test, p, zero_division=0):.3f}"
          f"  recall {recall_score(y_test, p):.3f}  f1 {f1_score(y_test, p, zero_division=0):.3f}")

none               rows  3500  accuracy 0.9900  precision 0.000  recall 0.000  f1 0.000
RandomOverSampler  rows  6930  accuracy 0.9340  precision 0.132  recall 1.000  f1 0.233
SMOTE              rows  6930  accuracy 0.9360  precision 0.135  recall 1.000  f1 0.238
SMOTE 0.5          rows  5197  accuracy 0.9460  precision 0.156  recall 1.000  f1 0.270

• Both over-samplers find all 15 minority rows, as the balanced undersample did.
• They raise fewer false alarms than undersampling did, because the majority class kept all 3465 of its rows and the model still knows where it lives.
• SMOTE and plain copying score almost the same here. With two features and a minority that lies in a thin band, interpolation adds little that repetition did not.
• Stopping at sampling_strategy=0.5 scores the best F1 of the four. Full balance is a default, not a target.

Why the resampler goes inside the pipeline

Resampling before the split puts invented minority rows, and copies of real ones, on both sides of the split. The test set then contains rows the model effectively trained on, and the score it reports is a measure of memory. imblearn.pipeline.Pipeline exists to stop that: it applies the resampler when fitting a fold and skips it when scoring one.

from imblearn.pipeline import Pipeline
from sklearn.model_selection import cross_val_score, StratifiedKFold

pipe = Pipeline([("smote", SMOTE(random_state=42)),
                 ("model", LogisticRegression(max_iter=1000))])
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=0)

from imblearn.pipeline import Pipeline
from sklearn.model_selection import cross_val_score, StratifiedKFold
from sklearn.linear_model import LogisticRegression
from imblearn.over_sampling import SMOTE

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=0)

pipe = Pipeline([("smote", SMOTE(random_state=42)),
                 ("model", LogisticRegression(max_iter=1000))])
honest = cross_val_score(pipe, X, y, cv=cv, scoring="f1")

X_all, y_all = SMOTE(random_state=42).fit_resample(X, y)
leaked = cross_val_score(LogisticRegression(max_iter=1000), X_all, y_all, cv=cv, scoring="f1")

print("resampled inside each fold:", honest.round(4))
print("resampled before splitting:", leaked.round(4))
print()
print("mean f1, honest:", round(honest.mean(), 4))
print("mean f1, leaked:", round(leaked.mean(), 4))

resampled inside each fold: [0.2353 0.303  0.274  0.3571 0.2597]
resampled before splitting: [0.9673 0.9773 0.9754 0.9807 0.9725]

mean f1, honest: 0.2858
mean f1, leaked: 0.9746

• The leaked score is about three times the honest one. Nothing about the model changed; only where the resampling happened.
• The leak is mechanical. SMOTE builds each new row from two real rows. Split afterwards and a new row can land in the test fold while both of its parents are in the training fold.
• RandomOverSampler leaks the same way, and more directly: the identical row can land on both sides of the split.
• The honest number is the one that predicts production. A model picked on the leaked score is picked on an illusion. Data leakage covers the other places this happens.

RandomOverSampler against SMOTE

Where oversampling earns its place

• The dataset is small. Nothing can be thrown away, so adding is the only direction left.
• The minority rows differ from one another in a meaningful way. SMOTE between two rows that mean different things invents a row that means nothing.
• You are comparing methods honestly. Put the sampler in an imblearn pipeline and cross-validate, so every fold resamples its own training part.

Related

• Previous: Undersampling
• Next: class_weight
• See also: Data leakage
• Reference: Over-sampling and Common pitfalls in the imbalanced-learn user guide