TopolearnEvery later lesson talks about models, weights, training, loss and gradients; this one explains those words in plain language first.
Suppose you want a program that looks at a photo and says whether it shows a cat. Writing the rules by hand is hopeless: what exactly makes a grid of pixels a cat? Machine learning takes the other route. You collect many examples, each with the right answer attached (the training data), and let the computer find the pattern itself.
What it finds is a model: a fixed formula with a large number of adjustable numbers inside it. Those adjustable numbers are the model's weights. Feed the model an input (the pixels), it does arithmetic with its weights and produces an output (a guess: cat or not). With random weights the guesses are nonsense; with good weights they are right.
Training is the search for good weights. It repeats one simple loop:
- Run some examples through the model and look at its guesses.
- Score how wrong they are with a loss function: one number that is zero when every guess is perfect and grows the worse the guesses get.
- Work out, for every weight, which way to nudge it to make the loss smaller, and by how much. That set of directions is the gradient.
- Nudge every weight a small step that way. Repeat with more examples until the loss stops falling.
TensorFlow does the heavy parts for you. It stores the data in tensors (grids of numbers, like NumPy arrays, but with any number of dimensions). It works out gradients automatically, so you never derive a formula. And it runs the arithmetic on a GPU when you have one: a graphics chip with thousands of small cores that can do millions of multiplications at the same time.
Most models you will build are neural networks: stacks of layers, where each layer takes the numbers from the one below, does a weighted sum with its own weights, squashes the result a little, and passes it up. Big stacks of simple arithmetic turn out to recognise cats, translate text and much more.
You do not need the maths to start. You need the vocabulary: model, weights, training, loss, gradient. The next lessons introduce the TensorFlow objects that hold each of them.
Example
# A toy "model" with one weight w: guess = w * x. The truth is y = 3 * x.
w = 0.0
examples = [(1, 3), (2, 6), (3, 9)] # (input, correct answer)
def loss(w): # 0 when every guess is perfect
return sum((w * x - y) ** 2 for x, y in examples)
for step in range(20):
# gradient: how much the loss rises if w grows a tiny bit
grad = (loss(w + 0.001) - loss(w)) / 0.001
w -= 0.01 * grad # step the other way, to lower the loss
print(round(w, 2), round(loss(w), 4)) # 3.0 0.0003 -> it learned y = 3x
# TensorFlow does the same with millions of weights, computing the gradient exactly.Remember
- A model is a formula with adjustable numbers (weights); training finds good values for them
- The loss measures how wrong the model is; the gradient says which way to nudge each weight
- TensorFlow stores data in tensors, computes gradients for you and runs on GPUs
Watch out for
- Trusting a model because its loss on the training data got very low: it may have memorised those examples instead of learning a pattern that works on new ones.