Geometric Computer Vision and Camera Calibration

Introduction and Context

Geometric computer vision studies how cameras form images and how measurements in images can be used to infer the structure, position, and motion of objects in the world. Unlike appearance-based methods, which rely primarily on texture, colour, or learned visual patterns, geometric methods use mathematical relationships among points, lines, planes, and cameras.

A camera does not record three-dimensional coordinates directly. It projects a three-dimensional scene onto a two-dimensional image. This projection loses information: many different 3D points can produce the same image point. Geometric computer vision develops models and algorithms for recovering useful information despite this loss.

The central topics of this chapter are coordinate transformations, projective geometry, homographies, epipolar geometry, camera calibration, pose estimation, and robust estimation using RANSAC. These ideas support applications such as robot navigation, augmented reality, 3D reconstruction, industrial inspection, autonomous vehicles, medical imaging, and visual measurement.

Learning Objectives

After studying this chapter, a student should be able to:

1. Represent points and transformations using Cartesian and homogeneous coordinates.

2. Explain the pinhole camera model and distinguish intrinsic from extrinsic parameters.

3. Apply projective transformations and homographies to image measurements.

4. Explain epipolar geometry and the role of the fundamental and essential matrices.

5. Describe methods for intrinsic calibration, pose estimation, and planar scene registration.

6. Use RANSAC conceptually and quantitatively for robust geometric estimation.

7. Evaluate assumptions, errors, and practical limitations in a computer-vision system.

1. Coordinate Systems and Transformations

A geometric vision system commonly uses several coordinate frames:

• World frame: a fixed reference frame describing the environment.
• Camera frame: a frame whose origin is at the camera centre and whose axes describe camera orientation.
• Image frame: a coordinate system on the image sensor, usually measured in pixels.
• Object frame: a local frame attached to an object or calibration target.

A 3D point in Cartesian coordinates is written as

\[

\mathbf{X}=[X,Y,Z]^T.

\]

A rigid transformation preserves distances and angles. It consists of a rotation matrix \(\mathbf{R}\) and a translation vector \(\mathbf{t}\):

\[

\mathbf{X}_c=\mathbf{R}\mathbf{X}_w+\mathbf{t}.

\]

Here, \(\mathbf{X}_w\) is a world point and \(\mathbf{X}_c\) is the same point expressed in camera coordinates. The rotation satisfies

\[

\mathbf{R}^T\mathbf{R}=\mathbf{I}, \qquad \det(\mathbf{R})=1.

\]

The camera centre expressed in world coordinates is

\[

\mathbf{C}_w=-\mathbf{R}^T\mathbf{t}.

\]

This distinction is important: \(\mathbf{t}\) is not generally the camera position in world coordinates. It is the translation that maps world coordinates into the camera frame.

Homogeneous Coordinates

Homogeneous coordinates represent a 2D point \((x,y)\) as

\[

\tilde{\mathbf{x}}=[x,y,1]^T,

\]

and a 3D point \((X,Y,Z)\) as

\[

\tilde{\mathbf{X}}=[X,Y,Z,1]^T.

\]

A homogeneous vector represents an equivalence class: multiplying all its components by any nonzero scalar gives the same geometric point. For example, \([2,4,2]^T\) represents the Euclidean point \((1,2)\).

Homogeneous coordinates allow translations and projective transformations to be represented by matrix multiplication. A rigid 3D transformation can be written as

\[

\begin{bmatrix}\mathbf{X}_c\\1\end{bmatrix}

=

\begin{bmatrix}\mathbf{R}&\mathbf{t}\\\mathbf{0}^T&1\end{bmatrix}

\begin{bmatrix}\mathbf{X}_w\\1\end{bmatrix}.

\]

2. The Pinhole Camera Model

The ideal pinhole camera assumes that light travels in straight lines through a single optical centre. Let a point in camera coordinates be \((X_c,Y_c,Z_c)\), where \(Z_c>0\) indicates that it lies in front of the camera. Its normalized image coordinates are

\[

x=\frac{X_c}{Z_c}, \qquad y=\frac{Y_c}{Z_c}.

\]

This division by depth is the source of perspective effects: objects farther from the camera appear smaller.

Pixel coordinates are obtained using the intrinsic matrix

\[

\mathbf{K}=

\begin{bmatrix}

f_x&s&c_x\\

0&f_y&c_y\\

0&0&1

\end{bmatrix},

\]

where \(f_x\) and \(f_y\) are focal lengths measured in pixel units, \((c_x,c_y)\) is the principal point, and \(s\) is the skew parameter. Most modern cameras have negligible skew, so \(s\) is often set to zero.

The complete projection equation is

\[

\lambda\tilde{\mathbf{x}}=\mathbf{K}[\mathbf{R}\mid\mathbf{t}]\tilde{\mathbf{X}}_w,

\]

where \(\lambda\) is a nonzero scale factor, \(\tilde{\mathbf{x}}\) is a homogeneous image point, and \([\mathbf{R}\mid\mathbf{t}]\) is the extrinsic camera matrix. In the standard convention, \(\lambda\) is proportional to the camera-frame depth \(Z_c\).

The matrix \(\mathbf{K}\) describes how the camera converts normalized coordinates into pixels. The matrices \(\mathbf{R}\) and \(\mathbf{t}\) describe the camera pose relative to the world frame.

Perspective Properties

Perspective projection has several important properties:

• 3D lines project to image lines, except for degenerate cases involving points at infinity.
• Parallel 3D lines may meet at a vanishing point.
• Angles, lengths, and parallelism are not generally preserved.
• Collinearity is preserved.
• Points with different depths can project to the same image location.

These properties explain both the power and limitations of image geometry.

3. Projective Geometry and Homographies

A projective transformation maps homogeneous image points according to

\[

\tilde{\mathbf{x}}'\sim \mathbf{H}\tilde{\mathbf{x}},

\]

where \(\mathbf{H}\) is a nonsingular \(3\times3\) homography matrix and \(\sim\) denotes equality up to scale. Since a homography has nine entries but is defined only up to scale, it has eight independent degrees of freedom.

A homography is valid in several important situations:

1. Two images observe the same physical plane.

2. The camera rotates about its optical centre, with no translation.

3. A planar object is viewed from two different poses.

4. An image is warped by a projective transformation.

For a plane defined in world coordinates by \(Z=0\), the projection equation becomes

\[

\lambda\tilde{\mathbf{x}}=\mathbf{K}[\mathbf{r}_1\ \mathbf{r}_2\ \mathbf{t}]

\begin{bmatrix}X\\Y\\1\end{bmatrix}.

\]

Therefore, the plane-to-image mapping is

\[

\mathbf{H}=\mathbf{K}[\mathbf{r}_1\ \mathbf{r}_2\ \mathbf{t}].

\]

More generally, for a plane with equation \(\mathbf{n}^T\mathbf{X}+d=0\), the homography between two calibrated cameras can be written, under a consistent coordinate convention, as

\[

\mathbf{H}=\mathbf{R}-\frac{\mathbf{t}\mathbf{n}^T}{d}.

\]

The exact signs depend on how the relative transformation and plane equation are defined, so conventions must be stated before implementation.

Estimating a Homography

Each point correspondence contributes two independent equations in the entries of \(\mathbf{H}\). At least four point correspondences are required, provided that no three are collinear and the points are not arranged in a degenerate configuration. In practice, more than four correspondences are used and the matrix is estimated by a least-squares method such as the normalized Direct Linear Transform (DLT).

A good implementation normalizes image coordinates before solving. Normalization improves numerical conditioning by translating points near the origin and scaling their average distance to a conventional value. The estimated homography is then denormalized and validated using reprojection error.

4. Epipolar Geometry

When two cameras observe the same 3D point, the point and the two camera centres define an epipolar plane. The intersection of this plane with each image plane is an epipolar line. Thus, a point in one image constrains the corresponding point in the second image to lie on a line rather than anywhere in the image.

Let \(\tilde{\mathbf{x}}\) and \(\tilde{\mathbf{x}}'\) be corresponding homogeneous image points. The fundamental matrix \(\mathbf{F}\) satisfies

\[

\tilde{\mathbf{x}}'^T\mathbf{F}\tilde{\mathbf{x}}=0.

\]

The corresponding epipolar line in the second image is \(\mathbf{l}'=\mathbf{F}\tilde{\mathbf{x}}\), and the line in the first image is \(\mathbf{l}=\mathbf{F}^T\tilde{\mathbf{x}}'\).

The fundamental matrix is a \(3\times3\) rank-two matrix defined up to scale. It has seven degrees of freedom. In an ideal noiseless setting, corresponding points satisfy the epipolar constraint exactly. Real measurements produce residual errors.

For calibrated cameras, normalized coordinates are related through the essential matrix:

\[

\mathbf{x}'^T\mathbf{E}\mathbf{x}=0,

\]

where

\[

\mathbf{E}=[\mathbf{t}]_\times\mathbf{R}.

\]

The matrix \([\mathbf{t}]_\times\) is the skew-symmetric matrix representing the cross product with \(\mathbf{t}\). If \(\mathbf{t}=[t_x,t_y,t_z]^T\), then

\[

[\mathbf{t}]_\times=

\begin{bmatrix}

0&-t_z&t_y\\

t_z&0&-t_x\\

-t_y&t_x&0

\end{bmatrix}.

\]

The essential matrix determines relative rotation and translation direction, but not the absolute scale of translation. This is a fundamental limitation of two-view reconstruction: a scene can be enlarged while the camera baseline is enlarged proportionally, producing the same image measurements.

Triangulation

Given camera matrices and corresponding image points, triangulation estimates the 3D point whose projections agree with the observations. Since image measurements are noisy, the estimated point usually minimizes a reprojection or algebraic error rather than satisfying both projection equations exactly.

A valid reconstructed point should have positive depth in both camera frames. This requirement is called the cheirality condition and helps select the physically correct relative pose among algebraic alternatives.

5. Camera Calibration

Camera calibration estimates the parameters needed to relate image measurements to rays in space. It normally includes:

• Intrinsic parameters: focal lengths, principal point, skew, and distortion coefficients.
• Extrinsic parameters: rotation and translation of the camera relative to a reference frame.

Calibration is essential when measurements must have metric or geometric meaning. Without calibration, image coordinates cannot reliably be converted into viewing rays or used for accurate pose and 3D reconstruction.

Lens Distortion

Real lenses deviate from the pinhole model. Radial distortion causes straight image lines to bend, especially near the image boundary. A common radial model uses normalized coordinates \((x,y)\), with \(r^2=x^2+y^2\):

\[

x_d=x(1+k_1r^2+k_2r^4+k_3r^6),

\]

\[

y_d=y(1+k_1r^2+k_2r^4+k_3r^6).

\]

Tangential distortion can be represented by

\[

x_d=x+2p_1xy+p_2(r^2+2x^2),

\]

\[

y_d=y+p_1(r^2+2y^2)+2p_2xy.

\]

Here, \(k_i\) are radial coefficients and \(p_1,p_2\) are tangential coefficients. The model is an approximation and may be insufficient for fisheye or highly specialized lenses.

Calibration with a Planar Target

A common procedure uses a checkerboard or dot grid whose point coordinates are known. The target is photographed from multiple orientations and positions. For each image, the planar point correspondences estimate a homography. Constraints derived from these homographies estimate the intrinsic matrix, after which nonlinear optimization refines all parameters by minimizing reprojection error.

A typical objective is

\[

\min_{\mathbf{K},\mathbf{R}_i,\mathbf{t}_i,\boldsymbol{\delta}}

\sum_{i,j}\left\|\mathbf{x}_{ij}-\pi(\mathbf{K},\boldsymbol{\delta},\mathbf{R}_i,\mathbf{t}_i,\mathbf{X}_j)\right\|^2,

\]

where \(\pi\) is the distorted projection function, \(\boldsymbol{\delta}\) contains distortion parameters, \(i\) indexes images, and \(j\) indexes target points. The residual is measured in pixel units.

Good calibration practice includes using a target that fills different parts of the image, capturing substantial viewpoint variation, ensuring sharp images, and validating on images not used for fitting.

6. Pose Estimation

The Perspective-n-Point (PnP) problem estimates camera pose from known 3D points and their 2D image projections. Given \(n\) correspondences \((\mathbf{X}_i,\mathbf{x}_i)\) and known intrinsics, PnP estimates \(\mathbf{R}\) and \(\mathbf{t}\) such that

\[

\mathbf{x}_i\approx\pi(\mathbf{K},\mathbf{R},\mathbf{t},\mathbf{X}_i).

\]

Different algorithms address different correspondence counts and configurations. Minimal methods can use a small number of points, while iterative methods refine an initial estimate. The points should not be arranged in a degenerate configuration, such as an unsuitable collinear arrangement.

Pose quality is commonly evaluated through reprojection error:

\[

e_i=\left\|\mathbf{x}_i-\hat{\mathbf{x}}_i\right\|_2,

\]

where \(\hat{\mathbf{x}}_i\) is the predicted projection. The mean, median, or percentile of these errors can be reported, but the chosen metric should match the application.

7. Robust Estimation and RANSAC

Correspondence data frequently contain outliers caused by repeated textures, incorrect feature matches, occlusions, or detection errors. Ordinary least squares can be severely distorted by even a modest number of outliers.

RANSAC—Random Sample Consensus—handles this problem by repeatedly selecting a minimal random sample, fitting a model, and counting how many observations agree with it.

A generic procedure is:

1. Select a minimal sample of size \(s\).

2. Estimate the model from that sample.

3. Compute each observation's residual.

4. Classify observations with residual below threshold \(\tau\) as inliers.

5. Retain the model with the strongest consensus.

6. Re-estimate the model using all detected inliers.

If the probability that an individual correspondence is an inlier is \(w\), the probability that one sample of size \(s\) consists entirely of inliers is \(w^s\). To obtain confidence \(p\) that at least one all-inlier sample has been selected after \(N\) trials,

\[

1-(1-w^s)^N\ge p.

\]

Solving for \(N\) gives

\[

N\ge\frac{\log(1-p)}{\log(1-w^s)},

\]

provided \(0<w<1\). This estimate assumes independent sampling and a stable inlier ratio. For example, if \(w=0.5\), \(s=4\), and \(p=0.99\), then

\[

N\ge\frac{\log(0.01)}{\log(1-0.5^4)}

=\frac{-4.6052}{\log(0.9375)}\approx71.4.

\]

Thus, at least 72 iterations are required after rounding upward. In practice, adaptive stopping, sample-quality checks, and local optimization may improve performance.

The threshold \(\tau\) must reflect measurement noise and the residual's units. A threshold that is too small rejects genuine observations; one that is too large accepts outliers. Modern variants use robust loss functions, guided sampling, and local optimization.

Worked Example: Estimating a Pixel Projection

Suppose a calibrated camera has

\[

\mathbf{K}=\begin{bmatrix}800&0&320\\0&800&240\\0&0&1\end{bmatrix},

\]

and the camera-frame point is \((X_c,Y_c,Z_c)=(1.0,0.5,2.0)\) metres. Normalized coordinates are

\[

x=1.0/2.0=0.5,\qquad y=0.5/2.0=0.25.

\]

Therefore,

\[

u=f_xx+c_x=800(0.5)+320=720,

\]

\[

v=f_yy+c_y=800(0.25)+240=440.

\]

The predicted pixel location is approximately \((720,440)\). This calculation assumes an ideal pinhole model, zero skew, and no lens distortion.

Applied Case: Visual Registration of a Printed Planar Target

Background

Consider an illustrative warehouse system that uses a camera mounted above a workbench. A printed calibration board is attached to the bench, and the system must estimate the location of parcels relative to the bench coordinate system.

Problem

The camera image contains the board at different viewpoints. The system must determine the board's pose and map image points to physical bench coordinates. Feature matching may produce incorrect correspondences because the board contains repeated visual patterns.

Analysis

Because the board is planar, a homography is an appropriate image-to-board model. However, raw images contain lens distortion, and incorrect matches can corrupt a direct least-squares estimate. The camera is first calibrated using several checkerboard images. Distortion is removed or incorporated into the projection model. Candidate feature correspondences are then evaluated with RANSAC.

Approach and Solution

The system estimates a homography from four-point samples, counts inliers using a pixel reprojection threshold, and repeats the process sufficiently often for the expected inlier ratio. The final homography is refined with all inliers. If metric 3D pose is required, the homography is decomposed using the calibrated intrinsic matrix, and the physically valid solution is selected using positive-depth constraints.

Outcome

The resulting mapping supports parcel localization, robot pick-and-place operations, and quality inspection. Validation images are used to measure median reprojection error and detect failures caused by blur, severe occlusion, or board movement.

Learning Points

The case demonstrates that model selection depends on scene structure: a homography is appropriate for a plane but not for arbitrary 3D objects. Calibration should precede precision measurement, and robust estimation is necessary when correspondences are imperfect.

Limitations, Best Practices, and Professional Implications

Geometric methods depend on assumptions that can fail in practice. The pinhole model may not represent strong lens distortion. Poorly distributed calibration points can make parameters weakly observable. Planar methods cannot recover general 3D structure from a single view. Two-view reconstruction has an unknown scale. Low texture, motion blur, rolling-shutter effects, changing illumination, and dynamic objects can all reduce reliability.

Best practices include:

• State coordinate-frame conventions explicitly.
• Keep units consistent, especially when combining pixel and metric measurements.
• Calibrate with varied target poses and validate independently.
• Use normalized coordinates and numerically stable solvers.
• Reject degenerate samples in RANSAC.
• Report reprojection errors and confidence information rather than only a pose estimate.
• Check positive depth and physical plausibility.
• Recalibrate after changing focus, lens configuration, sensor format, or mechanical mounting.
• Preserve calibration data securely and document its date, camera serial number, and test conditions.

In safety-critical systems, calibration errors can cause incorrect robot motion, unsafe navigation, or inaccurate inspection decisions. Governance therefore includes traceability, periodic validation, uncertainty reporting, and human review for high-consequence decisions. Images may also contain personal or sensitive information, so collection, storage, access control, and retention must comply with relevant privacy and data-protection requirements.

Contemporary Developments

Current geometric vision systems increasingly combine classical geometry with deep learning. Learned feature detectors and descriptors improve correspondence generation, while geometric verification enforces physically meaningful relationships. Differentiable camera models allow calibration and pose estimation to be integrated into trainable pipelines. Neural radiance fields and related scene representations use camera geometry to synthesize views and infer structure, although their reliability still depends on accurate poses and carefully controlled data.

Event cameras, omnidirectional cameras, rolling-shutter models, and multi-camera rigs require extensions beyond the basic pinhole model. In all cases, the fundamental engineering principle remains: select a model that matches the sensor, scene, and measurement objective, then validate it against observations.

Chapter Summary

Geometric computer vision models the relationship between 3D scenes, cameras, and 2D images. Rigid transformations use rotation and translation to relate coordinate frames, while homogeneous coordinates provide a unified representation for translations and projective mappings. The pinhole camera equation separates intrinsic parameters from extrinsic pose and explains perspective projection.

Homographies describe mappings between views of a plane and are estimated from point correspondences. Epipolar geometry constrains two-view correspondences through the fundamental matrix, or the essential matrix when camera intrinsics are known. Calibration estimates focal length, principal point, skew, and distortion, while pose estimation recovers camera orientation and position from known geometric correspondences. RANSAC makes these estimates robust to outliers by searching for models supported by consensus sets.

Practical systems must manage distortion, degeneracy, noise, unknown scale, dynamic scenes, and coordinate-convention errors. Careful calibration, error measurement, robust estimation, independent validation, and responsible data governance are essential for dependable deployment.

Key Terms

Coordinate frame: A reference system used to express positions and orientations.

Rigid transformation: A rotation and translation that preserve distances.

Homogeneous coordinates: Coordinates that represent Euclidean points up to a nonzero scale.

Pinhole camera: An ideal camera model with a single optical centre.

Intrinsic parameters: Camera parameters governing the mapping from normalized coordinates to pixels.

Extrinsic parameters: Camera rotation and translation relative to a reference frame.

Principal point: The pixel location where the optical axis intersects the image plane.

Skew: A parameter describing non-orthogonality of sensor axes.

Radial distortion: Lens distortion that varies with distance from the principal point.

Homography: A projective mapping between two planes or views under special conditions.

Epipolar plane: The plane formed by two camera centres and a scene point.

Epipolar line: The image-line constraint for a corresponding point in another view.

Fundamental matrix: A rank-two matrix encoding uncalibrated two-view epipolar geometry.

Essential matrix: A matrix encoding calibrated relative rotation and translation direction.

Triangulation: Estimation of a 3D point from multiple image observations.

PnP: Perspective-n-Point pose estimation from 3D–2D correspondences.

Reprojection error: The image distance between an observation and its predicted projection.

RANSAC: A robust estimator that fits models using consensus among randomly sampled data.

Cheirality: The requirement that reconstructed points have positive depth in the cameras.

Degeneracy: A configuration in which a model cannot be estimated reliably or uniquely.

Review Questions

Short Answer Questions

1. Distinguish intrinsic and extrinsic camera parameters.

2. Why is a translation vector in the extrinsic matrix not generally the camera centre in world coordinates?

3. State two situations in which a homography is applicable.

4. What constraint does the fundamental matrix impose on corresponding image points?

5. Why is RANSAC useful in feature-based geometric vision?

Long Answer Questions

1. Explain the pinhole camera model and derive the pixel projection equation from normalized image coordinates.

2. Describe a complete planar camera-calibration procedure, including distortion estimation and validation.

3. Explain epipolar geometry and compare the fundamental matrix with the essential matrix.

4. Discuss the assumptions, strengths, and limitations of homography-based registration.

5. Describe RANSAC in detail and explain how the inlier ratio, sample size, confidence, and threshold affect its performance.

Analytical / Application Questions

1. A camera has \(f_x=f_y=600\) pixels, principal point \((320,240)\), and observes the camera-frame point \((0.4,0.2,1.0)\) metres. Calculate its ideal pixel coordinates and state the assumptions used.

2. A homography estimation uses four-point samples, an estimated inlier ratio of 0.6, and desired confidence 0.99. Derive the minimum approximate number of RANSAC iterations using the standard iteration formula.

3. A pose-estimation system gives low training-image reprojection error but high error on new images. Analyse at least three possible causes and propose a validation or engineering remedy for each.

Multiple Choice Questions with Answers

Question 1

Which equation represents the standard world-to-camera rigid transformation?

A. Xc = R Xw + t

B. Xc = K R Xw

C. Xc = Xw / Z

D. Xc = R + t Xw

Correct Answer: A. Xc = R Xw + t

Question 2

Which parameter belongs to the camera intrinsic matrix?

A. World-frame camera position

B. Camera rotation

C. Principal point

D. Object translation

Correct Answer: C. Principal point

Question 3

How many independent degrees of freedom does a general 2D homography have?

A. 4

B. 6

C. 8

D. 9

Correct Answer: C. 8

Question 4

What is the minimum number of nondegenerate point correspondences needed to estimate a general homography?

A. 2

B. 3

C. 4

D. 8

Correct Answer: C. 4

Question 5

Which equation expresses the epipolar constraint using the fundamental matrix?

A. x'ᵀ F x = 0

B. x' = KX

C. Xc = RXw + t

D. H = K[R|t]

Correct Answer: A. x'ᵀ F x = 0

Question 6

What ambiguity remains in two-view reconstruction from an essential matrix?

A. The image width

B. The absolute translation scale

C. The pixel coordinate system

D. The rank of the matrix

Correct Answer: B. The absolute translation scale

Question 7

For a calibrated camera, what does PnP estimate?

A. Lens manufacturing date

B. Camera pose from 3D–2D correspondences

C. Image colour balance

D. The number of image pixels

Correct Answer: B. Camera pose from 3D–2D correspondences

Question 8

What is the primary purpose of RANSAC?

A. To increase image resolution

B. To remove all lens distortion analytically

C. To estimate a model despite outlier correspondences

D. To convert pixels into RGB values

Correct Answer: C. To estimate a model despite outlier correspondences

Question 9

If the inlier ratio is w and the minimal sample size is s, what is the probability that one random sample contains only inliers?

A. w + s

B. w − s

C. w/s

D. w^s

Correct Answer: D. w^s

Question 10

Which condition helps select the physically valid solution during two-view reconstruction?

A. Negative depth in both cameras

B. Positive depth in the cameras

C. Equal focal lengths in all cameras

D. Zero translation between cameras

Correct Answer: B. Positive depth in the cameras