Transfer learning and fine-tuning

Pretrained models give strong results with little data and compute.

Transfer learning reuses a model trained on a large dataset. In Keras:

keras.applications provides ImageNet models (ResNet, EfficientNetV2, ConvNeXt, MobileNetV3...). include_top=False drops the classifier so you can add your own.
KerasHub (keras_hub) provides pretrained presets for text, vision and audio, including LLMs, all built on Keras 3.

The standard two-phase recipe:

1. Feature extraction: set base.trainable = False, build a new head on top, and train only the head. Call the base with training=False so its BatchNormalization layers stay in inference mode even later when you unfreeze them. 2. Fine-tuning: set base.trainable = True (or unfreeze only the top blocks), recompile so the change takes effect, and train a few epochs with a learning rate 10-100x smaller.

Preprocessing must match what the model was trained with. Each application has a preprocess_input function; some, like EfficientNetV2 and ConvNeXt, include preprocessing in the model and expect raw 0-255 pixels.

Setting trainable on a model or layer applies recursively to all its sublayers, and frozen weights appear in non_trainable_weights.

Example

import keras
from keras import layers

base = keras.applications.EfficientNetV2B0(
    include_top=False, weights="imagenet", input_shape=(224, 224, 3))
base.trainable = False                        # 1) freeze pretrained weights

inputs = keras.Input((224, 224, 3))           # raw 0-255 pixels for EfficientNetV2
x = base(inputs, training=False)              # keep BatchNorm in inference mode
x = layers.GlobalAveragePooling2D()(x)
x = layers.Dropout(0.2)(x)
outputs = layers.Dense(5)(x)
model = keras.Model(inputs, outputs)

loss = keras.losses.SparseCategoricalCrossentropy(from_logits=True)
model.compile(optimizer=keras.optimizers.Adam(1e-3), loss=loss, metrics=["accuracy"])
# model.fit(train_ds, validation_data=val_ds, epochs=5)    # trains the head only

base.trainable = True                         # 2) fine-tune everything
model.compile(optimizer=keras.optimizers.Adam(1e-5), loss=loss, metrics=["accuracy"])
# model.fit(train_ds, validation_data=val_ds, epochs=3)    # recompiled, tiny LR
print(len(model.trainable_weights) > 2)       # True

Remember

Freeze, train a new head, then unfreeze and fine-tune with a tiny LR
Call the base with training=False to protect BatchNorm statistics
Recompile after changing trainable

Watch out for

Changing trainable without recompiling, so the change is not applied to training.

Check yourself

Why call the pretrained base with training=False even when fine-tuning?

Official docs: keras.io/guides/transfer_learning/

Example

Remember

Watch out for

Check yourself

Builds on

Unlocks