TopolearnUnscaled features silently degrade k-NN, SVMs, regularised linear models and PCA.
Many models compare features numerically. If income is in tens of thousands and age is in tens, a k-nearest-neighbours distance is dominated by income, and a regularised linear model penalises the two coefficients unfairly. Scaling puts features on comparable ranges.
The common scalers in sklearn.preprocessing:
StandardScaler: subtracts the mean and divides by the standard deviation, giving mean 0 and variance 1. The default choice.MinMaxScaler: maps each feature to [0, 1] (orfeature_range). Sensitive to outliers.RobustScaler: uses the median and interquartile range, so outliers barely affect it.
The crucial rule: fit the scaler on the training data only, then use the same fitted scaler to transform the test data and any future data. fit_transform on the test set would compute new means from test data, both leaking information and producing inconsistent features.
Which models need it? k-NN, SVMs, linear and logistic regression with regularisation, neural networks, PCA and k-means all do. Tree-based models (decision trees, random forests, gradient boosting) split on thresholds and are unaffected.
In practice you rarely call scalers by hand: put them in a Pipeline with the model so the train-only rule is applied automatically, including inside cross-validation.
Example
import numpy as np
from sklearn.preprocessing import StandardScaler, MinMaxScaler
X_train = np.array([[20, 30_000], [35, 60_000], [50, 90_000]], dtype=float)
X_test = np.array([[40, 120_000]], dtype=float)
scaler = StandardScaler().fit(X_train) # learn mean/std from train only
print(scaler.mean_) # [3.5e+01 6.0e+04]
print(scaler.transform(X_train).round(2))
# [[-1.22 -1.22]
# [ 0. 0. ]
# [ 1.22 1.22]]
print(scaler.transform(X_test).round(2)) # [[0.41 2.45]] (outside train range is fine)
print(MinMaxScaler().fit(X_train).transform(X_test)) # [[0.66666667 1.5 ]]Remember
- Fit scalers on training data, then transform train and test with the same object
StandardScaleris the default;RobustScalerfor outliers- Tree-based models do not need scaling
Watch out for
- Calling
fit_transformon the test set.