Running neural networks with cv2.dnn

OpenCV · Advanced · 15 min

cv2.dnn lets you deploy trained classifiers and detectors in a lightweight app with only OpenCV installed.

The dnn module runs pretrained networks for inference. Train in PyTorch or TensorFlow, export to ONNX, and load it with net = cv2.dnn.readNet("model.onnx") (or readNetFromONNX). Training is not supported.

Inputs go in as a 4-D blob in NCHW layout (batch, channels, height, width), float32. cv2.dnn.blobFromImage does the usual preprocessing in one call:

blob = cv2.dnn.blobFromImage(img, scalefactor=1/255, size=(224, 224),
                             mean=(0, 0, 0), swapRB=True, crop=False)
  • scalefactor multiplies pixels (after mean subtraction)
  • size resizes; crop=True centre-crops instead of stretching
  • swapRB=True turns OpenCV's BGR into the RGB most models were trained on

Then net.setInput(blob) and out = net.forward(). The output layout depends on the model: class scores for classifiers, boxes and scores for detectors, which you decode yourself and filter with cv2.dnn.NMSBoxes (non-maximum suppression). Higher-level wrappers such as cv2.dnn.DetectionModel handle some of this.

net.setPreferableBackend and setPreferableTarget choose the engine: the default CPU backend, CUDA, OpenVINO or OpenCL, depending on the build. Preprocessing must match training exactly: wrong scaling, channel order or normalisation silently ruins accuracy. OpenCV 5 reworks the dnn engine with broader ONNX coverage, but the Python workflow stays the same.

Example

import cv2
import numpy as np

img = np.zeros((300, 400, 3), np.uint8)
img[..., 2] = 255                                  # pure red in BGR

blob = cv2.dnn.blobFromImage(img, scalefactor=1 / 255, size=(224, 224),
                             mean=(0, 0, 0), swapRB=True, crop=False)
print(blob.shape, blob.dtype)                      # (1, 3, 224, 224) float32
print(blob[0, :, 0, 0])                            # [1. 0. 0.]  now RGB, scaled

boxes = [[10, 10, 100, 100], [12, 12, 100, 100], [200, 50, 80, 80]]   # x, y, w, h
scores = [0.9, 0.8, 0.75]
keep = cv2.dnn.NMSBoxes(boxes, scores, score_threshold=0.5, nms_threshold=0.4)
print(sorted(np.array(keep).ravel().tolist()))     # [0, 2]  duplicate box removed

# With a real model file:
# net = cv2.dnn.readNet("resnet18.onnx")
# net.setInput(blob); scores = net.forward()      # e.g. shape (1, 1000)

Remember

  • cv2.dnn runs inference only; train elsewhere and export to ONNX
  • blobFromImage makes NCHW float32 input: scale, resize, mean, swapRB
  • Decode outputs yourself and filter boxes with NMSBoxes
  • Preprocessing must match the model's training exactly

Watch out for

  • Forgetting swapRB=True for a model trained on RGB images, quietly lowering accuracy.

Check yourself

  1. What shape does `cv2.dnn.blobFromImage(img, size=(320, 240))` produce for one colour image?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: docs.opencv.org/4.x/d6/d0f/group__dnn.html

Builds on

Unlocks

Nothing builds on this yet.