TopolearnClassification is the most common ML task, and the workflow here is the template for every model.
A classifier predicts a discrete label. The workflow is always the same: split, fit on the training set, predict or score on the test set.
LogisticRegression is the usual first choice. Despite its name it is a classifier: it learns a weight per feature and passes the weighted sum through a sigmoid (or softmax for several classes) to get probabilities. It is fast, works well when classes are roughly linearly separable, and its coefficients are interpretable. Its regularisation strength is C (smaller means stronger regularisation). If you see a ConvergenceWarning, scale the features or raise max_iter.
KNeighborsClassifier predicts the majority label among the n_neighbors closest training points. It has almost no training step but predicts slowly on large data, and it is very sensitive to feature scales.
Every classifier offers:
predict(X): the predicted labels,predict_proba(X): class probabilities, columns ordered as inclasses_(most classifiers),score(X, y): mean accuracy.
Labels can be integers or strings; classes_ tells you which column of predict_proba is which. Always compare against a trivial baseline such as DummyClassifier(strategy="most_frequent"): a 95% accuracy is unimpressive if 95% of rows belong to one class.
Example
from sklearn.datasets import load_breast_cancer
from sklearn.dummy import DummyClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import train_test_split
X, y = load_breast_cancer(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, stratify=y, random_state=0)
for model in [DummyClassifier(strategy="most_frequent"),
LogisticRegression(max_iter=5000),
KNeighborsClassifier(n_neighbors=5)]:
model.fit(X_tr, y_tr)
print(type(model).__name__, round(model.score(X_te, y_te), 3))
# DummyClassifier 0.629
# LogisticRegression 0.937
# KNeighborsClassifier 0.916
print(model.classes_, model.predict_proba(X_te[:1])) # [0 1] [[0.8 0.2]] (k-NN)Remember
- Same workflow for every model: split, fit, then score on the test set
predict_probacolumns follow the order ofclasses_- Always compare with a
DummyClassifierbaseline
Watch out for
- Reading a high accuracy without checking the majority-class baseline.