TopolearnReal tables mix types, and ColumnTransformer is how you build one leak-free pipeline for all of them.
Tabular data usually needs different treatment per column: impute and scale numbers, impute and one-hot encode categories, maybe drop IDs. ColumnTransformer routes columns to transformers and concatenates the results side by side:
ColumnTransformer([("num", num_pipe, ["age", "income"]),
("cat", cat_pipe, ["city"])], remainder="drop")Each entry is (name, transformer, columns). Columns can be names (with a DataFrame), integer positions, a boolean mask, or a callable such as make_column_selector(dtype_include="number"), which picks columns by dtype at fit time. The transformer can itself be a pipeline, or "drop" / "passthrough".
remainder controls columns you didn't list: "drop" (default) or "passthrough". Explicit lists are safer than relying on remainder, because a new column in the input will not silently enter the model.
The usual pattern is a full pipeline: make_pipeline(preprocessor, model). Everything is fitted on the training folds, and you can tune nested parameters such as columntransformer__num__simpleimputer__strategy, though long names are a sign to name your steps explicitly.
get_feature_names_out() returns names prefixed with the transformer name, like num__age or cat__city_Paris; set verbose_feature_names_out=False to drop the prefix. That mapping is what you need to interpret coefficients or importances later.
Example
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler
df = pd.DataFrame({"age": [25, 32, None, 51, 46, 29],
"city": ["Oslo", "Rome", "Oslo", None, "Rome", "Lima"],
"bought": [0, 1, 0, 1, 1, 0]})
X, y = df[["age", "city"]], df["bought"]
pre = ColumnTransformer([
("num", make_pipeline(SimpleImputer(strategy="median"), StandardScaler()), ["age"]),
("cat", make_pipeline(SimpleImputer(strategy="most_frequent"),
OneHotEncoder(handle_unknown="ignore")), ["city"]),
])
clf = make_pipeline(pre, LogisticRegression()).fit(X, y)
print(pre.get_feature_names_out())
# ['num__age' 'cat__city_Lima' 'cat__city_Oslo' 'cat__city_Rome']
print(clf.predict(pd.DataFrame({"age": [60], "city": ["Paris"]}))) # [1]Remember
- Route each column group to its own transformer or sub-pipeline
- Prefer explicit column lists over
remainder="passthrough" get_feature_names_out()maps output columns back to inputs
Watch out for
- Forgetting that unlisted columns are dropped by default.