TopolearnAny measurement in real-world units, 3-D pose estimation or AR overlay needs a calibrated camera.
A camera is described by its intrinsic matrix K (focal lengths fx, fy in pixels and principal point cx, cy) and distortion coefficients (radial k1, k2, k3 and tangential p1, p2) that bend straight lines, especially with wide-angle lenses.
Calibration recipe:
- Print a chessboard and photograph it 10 to 20 times from different angles and distances, covering the whole frame.
- For each image,
cv2.findChessboardCorners(gray, (cols, rows))finds the inner corners; refine them withcv2.cornerSubPix.(cols, rows)counts inner corners, not squares. - Build matching 3-D object points: a grid with z = 0, scaled by the real square size.
ret, K, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, image_size, None, None).
ret is the RMS reprojection error in pixels; under about 0.5 is good. Larger values point to blurry shots, wrong pattern size or too little variety.
Then cv2.undistort(img, K, dist) straightens images; for video, precompute maps once with cv2.initUndistortRectifyMap and apply cv2.remap per frame. cv2.solvePnP(object_pts, image_pts, K, dist) recovers the pose of a known object, and cv2.projectPoints draws 3-D points onto the image.
The intrinsics are only valid for the resolution and zoom/focus you calibrated with. The ChArUco boards in the aruco module are more robust when the board is partly visible.
Example
import cv2
import numpy as np
pattern, square = (7, 5), 30.0 # inner corners, mm per square
obj = np.zeros((7 * 5, 3), np.float32)
obj[:, :2] = np.mgrid[0:7, 0:5].T.reshape(-1, 2) * square
K_true = np.array([[800, 0, 320], [0, 800, 240], [0, 0, 1]], np.float64)
objpoints, imgpoints = [], []
for rx, ry in [(0.3, 0.1), (-0.2, 0.3), (0.1, -0.3), (0.4, 0.4), (-0.3, -0.2)]:
rvec = np.array([rx, ry, 0.05]); tvec = np.array([-90.0, -60.0, 600.0])
pts, _ = cv2.projectPoints(obj, rvec, tvec, K_true, None) # simulated corners
objpoints.append(obj); imgpoints.append(pts.astype(np.float32))
rms, K, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, (640, 480), None, None)
print(rms < 0.01) # True: perfect synthetic data
print(K.round()) # close to fx=fy=800, cx=320, cy=240
print(dist.shape) # (1, 5) k1 k2 p1 p2 k3Remember
- Intrinsics K and distortion coefficients describe the camera
- Calibrate from many varied chessboard views; the pattern size counts inner corners
- The RMS reprojection error should be well under a pixel
- Undistort with undistort, or precomputed maps plus remap for video
Watch out for
- Calibrating at one resolution and using the matrix on frames of another resolution or zoom.