How cv2 maps NumPy arrays to cv::Mat

OpenCV · Expert · 15 min

Most mysterious cv2 errors in Python come from the array-to-Mat conversion, not from the algorithm itself.

OpenCV is C++; every Python call goes through generated bindings that convert ndarrays to cv::Mat (or cv::InputArray). A Mat is a header (rows, cols, type, step) pointing at a data buffer, so a suitable ndarray is wrapped without copying.

What counts as suitable:

  • Supported dtypes map to Mat depths: uint8 CV_8U, int8, uint16, int16, int32, float32, float64 (OpenCV 5 adds more, such as bool, uint32, int64). int64 arrays, NumPy's default integer, are rejected or converted by many functions, which is why points should be int32 or float32.
  • The last axis becomes channels (limited to 512, CV_CN_MAX) when the array is 3-D.
  • Rows may be strided, but each row's pixels must be contiguous. A view like img[..., ::-1] has a negative channel stride, and functions that write into an argument (drawing, out=/dst= parameters) raise "Layout of the output array is incompatible with cv::Mat". Fix it with np.ascontiguousarray.

In-place functions like cv2.rectangle mutate your array because the Mat shares its memory. Functions returning a new image allocate a fresh ndarray; passing a correctly shaped dst= array reuses memory in hot loops.

Saturation: cv2 arithmetic and conversions use saturate_cast, clamping instead of wrapping, and rounding float-to-int to nearest rather than truncating as NumPy's astype does.

Read error messages carefully: "Expected Ptr<cv::UMat> for argument" usually means the array argument had an unsupported dtype or shape, since the binding tried every overload and none matched.

Example

import cv2
import numpy as np

img = np.zeros((50, 50, 3), np.uint8)
rgb_view = img[..., ::-1]                        # negative channel stride
print(rgb_view.flags["C_CONTIGUOUS"])            # False
try:
    cv2.rectangle(rgb_view, (5, 5), (20, 20), (255, 0, 0), -1)
except cv2.error as e:
    print("error:", "Layout" in str(e))         # error: True
fixed = np.ascontiguousarray(rgb_view)
cv2.rectangle(fixed, (5, 5), (20, 20), (255, 0, 0), -1)   # works

pts64 = np.array([[1, 1], [10, 1], [10, 10]])    # int64 by default
pts32 = pts64.astype(np.int32)
cv2.fillPoly(img, [pts32], (0, 255, 0))          # int32 is what cv2 expects

f = np.array([[1.6, -0.4, 300.0]], np.float32)
print(f.astype(np.uint8))                        # NumPy: [[1 0 44]] (truncate, wrap)
print(cv2.convertScaleAbs(f))                    # cv2: [[  2   0 255]] round + saturate

Remember

  • Suitable ndarrays are wrapped as cv::Mat without copying
  • Output/in-place arguments must be contiguous within each row: use np.ascontiguousarray
  • Prefer int32 or float32 for points; int64 often fails
  • cv2 conversions round and saturate, NumPy astype truncates and wraps

Watch out for

  • Drawing on img[..., ::-1] or another non-contiguous view and getting a cryptic layout error.

Check yourself

  1. Why can cv2.rectangle fail on `img[:, :, ::-1]` but work on `img.copy()`?
  2. `cv2.convertScaleAbs(np.float32([[300.0]]))` returns...

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: docs.opencv.org/4.x/d3/df2/tutorial_py_basic_ops.html