TopolearnPyTorch has no built-in fit(); you write this loop yourself, and every training script is a variation of it.
PyTorch leaves the training loop to you. Its standard shape:
1. model.train() to enable training behaviour (dropout, batch-norm statistics). 2. For each batch: compute predictions, compute the loss, opt.zero_grad(), loss.backward(), opt.step(). 3. Periodically model.eval() and measure on validation data inside with torch.no_grad():, which skips building the autograd graph and saves memory.
One pass over the training data is an epoch. Shuffle the data each epoch and use mini-batches (typically 32 to 512 samples), which give noisy but cheap gradient estimates.
Track the loss as a Python float with loss.item() for logging, but not on every step on a GPU, since each call synchronises.
Debugging tips that save hours:
- First try to overfit a single small batch. If the loss does not go to near zero, there is a bug.
- Check the initial loss is plausible (about ln C for C classes).
- Watch both training and validation metrics; a growing gap means overfitting.
Higher-level wrappers such as PyTorch Lightning or Hugging Face Trainer generate this loop for you, but they are much easier to use once you have written it by hand.
Example
import torch
from torch import nn
torch.manual_seed(0)
X = torch.randn(512, 2)
y = (X[:, 0] * X[:, 1] > 0).long() # label: same-sign quadrants
model = nn.Sequential(nn.Linear(2, 32), nn.ReLU(), nn.Linear(32, 2))
opt = torch.optim.AdamW(model.parameters(), lr=1e-2)
loss_fn = nn.CrossEntropyLoss()
for epoch in range(100):
model.train()
perm = torch.randperm(len(X)) # shuffle each epoch
for i in range(0, len(X), 64):
idx = perm[i:i + 64]
loss = loss_fn(model(X[idx]), y[idx])
opt.zero_grad()
loss.backward()
opt.step()
model.eval()
with torch.no_grad():
acc = (model(X).argmax(dim=1) == y).float().mean()
print(f"train accuracy {acc:.2f}") # typically above 0.95Remember
- train() / batches / zero_grad-backward-step / eval() + no_grad
- Overfit one batch first to prove the pipeline works
- Log
loss.item()sparingly on GPUs
Watch out for
- Evaluating without
model.eval(), so dropout and batch-norm behave as in training.