TopolearnMismatched loss and label formats, and poorly chosen learning rates, are the most common reasons a Keras model does not learn.
Losses compare targets and predictions and return a scalar:
SparseCategoricalCrossentropy: integer class labels.CategoricalCrossentropy: one-hot labels.BinaryCrossentropy: binary or multi-label targets in {0, 1}.MeanSquaredError,MeanAbsoluteError,Huber: regression.
All classification losses take from_logits=True when predictions are raw scores. Any function fn(y_true, y_pred) returning per-sample losses also works.
Optimizers live in keras.optimizers: Adam, AdamW (decoupled weight_decay), SGD(momentum=0.9), RMSprop, Lion and more. The learning_rate can be a float or a schedule from keras.optimizers.schedules, such as CosineDecay (with optional warmup) or ExponentialDecay, evaluated every step. Options like clipnorm clip gradients.
Metrics are stateful: update_state(y_true, y_pred) accumulates over batches, result() returns the current value and reset_state() clears it. That is why the accuracy fit() shows is an average over the epoch, not the last batch. Accuracy metrics come in sparse and categorical flavours that match the loss; AUC, Precision and Recall accept thresholds.
Example
import numpy as np
import keras
y_true = np.array([0, 2])
logits = np.array([[2.0, 0.5, -1.0], [0.1, 0.2, 3.0]], dtype="float32")
loss = keras.losses.SparseCategoricalCrossentropy(from_logits=True)
print(round(float(loss(y_true, logits)), 4)) # 0.1755
probs = keras.ops.softmax(logits)
print(round(float(keras.losses.SparseCategoricalCrossentropy()(y_true, probs)), 4)) # 0.1755
acc = keras.metrics.SparseCategoricalAccuracy()
acc.update_state(y_true, logits) # 2 of 2 correct
acc.update_state(np.array([1]), np.array([[0.9, 0.1, 0.0]])) # 0 of 1 correct
print(round(float(acc.result()), 3)) # 0.667 (accumulated)
acc.reset_state()
schedule = keras.optimizers.schedules.CosineDecay(1e-3, decay_steps=10_000)
opt = keras.optimizers.AdamW(learning_rate=schedule, weight_decay=1e-4)Remember
- Sparse losses for integer labels, categorical for one-hot
- Learning rates can be schedules evaluated every step
- Metrics accumulate with update_state / result / reset_state
Watch out for
- Using CategoricalCrossentropy with integer labels, which fails with a shape error (or trains on the wrong targets).