Camera Calibration

Camera calibration is the process of estimating the intrinsic and extrinsic parameters of a camera — focal length, optical center, and lens distortion coefficients — so that we can accurately map 3D points in the world to 2D image coordinates. Calibration is a prerequisite for any vision-based robotics pipeline: visual odometry, SLAM, object pose estimation, and 3D reconstruction all depend on an accurate camera model. In simulation the camera is ideal (no distortion, known focal length), but real cameras always have manufacturing imperfections.

Background reading

Study sections 13.1 and 13.2 of Peter Corke’s textbook Robotics, Vision and Control (RVC3). The book’s GitHub repo and the Chapter 13 notebook script contain reference code.

Zhang’s method

Zhang’s method estimates camera parameters from multiple views of a planar calibration pattern (typically a checkerboard). The method consists of three stages:

DLT (Direct Linear Transform) initialization — compute an initial estimate of the homography between the calibration pattern and each image.
Closed-form intrinsic parameter extraction — recover the camera matrix $K$ from the homographies.
Non-linear refinement — refine all parameters (intrinsics + extrinsics for each view) by minimizing reprojection error using SGD or Levenberg-Marquardt.

For the mathematical details, see the Zhang’s Algorithm page.

Test dataset

Use the pantelism/wide-camera-calibration HuggingFace dataset as test images (Parquet format):

from datasets import load_dataset
dataset = load_dataset("pantelism/wide-camera-calibration")
# Access images: dataset["train"][0]["image"]

Capturing your own calibration images

Print the 9x7 checkerboard pattern and attach it to a rigid flat surface.
Using your smartphone or webcam at a fixed zoom level, capture at least 15 images from different angles, distances, and orientations.

Capture tips:

Ensure good, diffuse lighting; keep the pattern flat and in focus
Disable autofocus and set a fixed focus for the duration
Capture from many positions and angles, up to about 45 degrees
Ensure the pattern fills a significant portion of the field of view
Avoid perfectly frontal views — angled views provide more information
Capture both wide shots and close-ups

Calibration outputs

After calibration, report:

Camera matrix $K$ (focal lengths $f_x$ , $f_y$ and principal point $c_x$ , $c_y$ )
Distortion coefficients ( $k_1$ , $k_2$ , $p_1$ , $p_2$ , $k_3$ )
Reprojection error (RMS, in pixels)
Undistorted vs. distorted image comparisons
Radial distortion profile (distortion factor vs. radial distance from center)

Pose estimation with a calibrated camera (ROS 2)

Once calibrated, the camera parameters enable object pose estimation:

Use OpenCV’s solvePnP to estimate the relative pose of objects from the camera, given known 3D geometry and 2D detections. Alternatively, use ArUco fiducial markers with cv2.aruco.
Establish a world coordinate system using fixed objects to compute the camera pose (rotation + translation) in the world frame.
In ROS 2, calibration parameters are published on the /camera_info topic:

camera_matrix:
  rows: 3
  cols: 3
  data: [fx, 0, cx, 0, fy, cy, 0, 0, 1]
distortion_model: "plumb_bob"
distortion_coefficients:
  data: [0, 0, 0, 0, 0]  # no distortion in sim

OpenCV reference solution

The reference notebook implements the full calibration pipeline using OpenCV — corner detection, calibration, undistortion, reprojection error analysis, and distortion profiling.