TopolearnMLflow records machine learning work, and every later lesson uses these few words.
Imagine you want a program that guesses whether a customer will cancel their subscription. Writing the rules by hand ("if they haven't opened the app for 30 days...") gets hard fast. Machine learning takes another route: you show the computer many past examples, with the right answers, and let it find the pattern itself.
Those examples are the training data. Finding the pattern in them is called training.
What comes out of training is a machine learning model, usually just called a model: a bundle of learned numbers plus a formula that turns a new customer's details into a prediction. Using a finished model on new data is called inference.
How do you know a model is any good? You keep some examples out of training and check the model's predictions on them. While you are still trying out changes, you check on a validation set.
A test set is different: a second group of examples kept back the whole time and used only once, at the very end, for the final score. Because nothing was adjusted to suit it, that score shows how the model will do on truly new data.
A score like the share of correct answers (accuracy) is a metric.
Before training you also choose settings that shape how the model learns, such as how deep a decision tree (a model that asks a chain of yes/no questions) may grow, or how big each learning step is (the learning rate). Such settings are hyperparameters. Finding good ones means training many times with different values, and keeping track of all those attempts is exactly what MLflow is for.
In Python, the most common library for this kind of classic machine learning is scikit-learn. Every model in it works the same way: create the model object, call fit(X, y) to train it on inputs X and correct answers y, then call predict(X_new) to get predictions.
The inputs are usually a table: one row per example, one column per input, such as age or number of visits.
In Python such tables usually come as a DataFrame, the table type of the popular pandas library. You'll often see X.head(3), which means "the first three rows of the table".
Example
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
# 150 flowers: 4 measurements each (X, a DataFrame) and the species as the answer (y)
X, y = load_iris(return_X_y=True, as_frame=True)
# keep a quarter of the flowers aside as the validation set
X_train, X_val, y_train, y_val = train_test_split(X, y, random_state=0)
model = DecisionTreeClassifier(max_depth=3) # max_depth is a hyperparameter
model.fit(X_train, y_train) # training
preds = model.predict(X_val) # inference on unseen flowers
accuracy = (preds == y_val).mean() # a metric: share of correct answers
print(round(accuracy, 2)) # 0.97Remember
- Training finds a pattern in example data and produces a model; using the model on new data is inference
- Judge a model with a metric on a validation set it never trained on
- Hyperparameters are settings you choose before training; tuning them means many training attempts
Watch out for
- Measuring accuracy on the same examples the model trained on, which makes it look better than it really is.