TopolearnBusiness objectives and research ideas rarely match the built-in losses and metrics exactly.
Custom losses: any function loss(y_true, y_pred) built from keras.ops that returns one value per sample works. Keras applies sample weights and reduces to a scalar for you. If the loss has hyperparameters that must be saved, subclass keras.losses.Loss, implement call(y_true, y_pred) and get_config.
Custom metrics must accumulate across batches, so they are classes:
- Subclass
keras.metrics.Metric. - In
__init__, create state withself.add_variable(shape=(), initializer="zeros", name=...). update_state(y_true, y_pred, sample_weight=None): update the variables withassign_add.result(): compute the metric from the state.reset_state(): the default resets every variable to its initializer, which is usually enough.
Why not compute a metric as a mean of per-batch values? Because many metrics (precision, AUC, percentages of totals) are not averages of per-batch results. Keeping sums and counts gives the exact epoch-level value.
For quantities that are not functions of (y_true, y_pred), such as a regularisation term computed inside a layer, use self.add_loss(...) in the layer, or override compute_loss in the model.
Example
import numpy as np
import keras
from keras import ops
def huber_like(y_true, y_pred, delta=1.0): # a plain function is a valid loss
err = ops.abs(y_true - y_pred)
sq = 0.5 * err ** 2
return ops.mean(ops.where(err < delta, sq, delta * (err - 0.5 * delta)), axis=-1)
class MeanAbsPct(keras.metrics.Metric):
def __init__(self, name="mape", **kwargs):
super().__init__(name=name, **kwargs)
self.total = self.add_variable(shape=(), initializer="zeros", name="total")
self.count = self.add_variable(shape=(), initializer="zeros", name="count")
def update_state(self, y_true, y_pred, sample_weight=None):
pct = ops.abs((y_true - y_pred) / y_true) * 100.0
self.total.assign_add(ops.sum(pct))
self.count.assign_add(ops.sum(ops.ones_like(pct)))
def result(self):
return self.total / self.count
m = MeanAbsPct()
m.update_state(np.array([100.0, 200.0], "float32"), np.array([110.0, 190.0], "float32"))
print(float(m.result())) # 7.5 (10% and 5% errors)
model = keras.Sequential([keras.Input((3,)), keras.layers.Dense(1)])
model.compile(optimizer="adam", loss=huber_like, metrics=[MeanAbsPct()])Remember
- A loss can be any
fn(y_true, y_pred)built from keras.ops - Metrics keep sums/counts in variables: update_state, result, reset_state
- Use
add_lossfor terms that do not depend on targets
Watch out for
- Averaging per-batch metric values for metrics that are not batch-decomposable, like precision.