Image Formation, Optics, Sensors, and Illumination

Introduction

Machine vision begins before image processing. A computer can only analyze the information delivered by an imaging system, and that information is determined by scene geometry, illumination, optical design, sensor physics, exposure, digitization, and communication. A poorly formed image cannot generally be rescued by sophisticated software. Conversely, a carefully designed acquisition system can make inspection, measurement, localization, and robot guidance substantially more reliable.

This chapter develops the physical and engineering foundations of image acquisition. It treats image formation in two complementary ways. Geometric image formation describes where scene points appear in an image. Radiometric image formation describes how much measurable energy reaches each sensor element. The chapter then considers lenses, cameras, sensor technologies, filters, exposure, field of view, resolution, depth of field, interfaces, and imaging beyond ordinary visible-light cameras.

Learning Objectives

By the end of this chapter, a student should be able to:

1. Explain perspective projection and the thin-lens model.
2. Relate focal length, working distance, field of view, magnification, and sensor size.
3. Distinguish spatial, radiometric, spectral, temporal, and depth resolution.
4. Select camera sensors, lenses, filters, and illumination for an inspection task.
5. Calculate exposure, pixel scale, depth of field, and sampling requirements under stated assumptions.
6. Compare visible, near-infrared, thermal, depth, and multispectral imaging.
7. Evaluate acquisition interfaces and identify major sources of image degradation.

1. The Image-Formation Chain

A practical imaging system can be represented as a chain:

scene → illumination → surface interaction → lens → optical filter → sensor → analog electronics → digitizer → interface → image-processing software.

Each stage can alter the information. Illumination may create shadows or specular reflections. Surface texture and material properties determine reflected or emitted radiation. The lens introduces magnification, distortion, blur, and diffraction. The sensor converts photons or other physical signals into electrical charge. Electronics introduce noise, and digitization limits the numerical representation.

The image is therefore not a neutral copy of an object. It is a measurement produced by a physical system. In machine vision, the design question is not simply “Which camera has the highest resolution?” but “Which acquisition system produces measurements that separate the relevant classes of objects under realistic operating conditions?”

2. Geometric Image Formation

2.1 Perspective projection

The pinhole camera is a useful ideal model. A three-dimensional point with coordinates (X, Y, Z) is projected onto an image plane at coordinates (x, y). If the optical axis is the Z-axis and the focal length is f, then:

Here X, Y, and Z are measured in a camera-centered coordinate system, while x, y, and f use the same length unit. The model assumes a pinhole, no lens distortion, and an image plane convention in which the image is represented upright. In a physical camera, the sensor is commonly behind the lens and the image is inverted; coordinate conventions can absorb this sign change.

The equations show that image size depends on depth. A point farther from the camera produces a smaller image. This is why perspective projection is useful for depth estimation but also why objects at different distances cannot be compared directly without calibration.

A more complete camera model includes focal lengths in pixel units, principal-point coordinates, and skew:

The vector [u, v] gives pixel coordinates, s is a scale factor, R is a 3 × 3 rotation matrix, t is a translation vector, and K is the intrinsic calibration matrix. In the common zero-skew case:

The parameters fₓ and fᵧ are focal lengths expressed in pixels, (cₓ, cᵧ) is the principal point, and the semicolon separates matrix rows. This model is fundamental for robot vision, camera calibration, measurement, and three-dimensional reconstruction.

2.2 Thin-lens imaging

A real lens forms an image according to the thin-lens equation:

where f is focal length, u is object distance, and v is image distance. The equation assumes a thin lens and paraxial rays; real lenses depart from it because of thickness, aberration, and finite aperture.

The lateral magnification is:

where hₒ is object height and hᵢ is image height. The negative sign indicates inversion. For distant objects, u is much larger than f, so v is approximately f and the magnitude of magnification is approximately f/u.

In machine vision, working distance is the distance from the relevant lens reference plane to the object. It should not be confused with the distance from the front of the housing unless the lens specification defines it that way.

2.3 Field of view and magnification

For a rectilinear lens, the field of view across a sensor dimension is approximately:

where S is the corresponding sensor dimension, f is focal length, and u is object distance. This formula assumes a thin lens, negligible distortion, and a planar object approximately perpendicular to the optical axis.

For close-up inspection, magnification is often more useful:

where W is the object width that fills sensor dimension S. If a sensor is 8.8 mm wide and the required object field is 220 mm wide, m ≈ 8.8/220 = 0.04. A 5.5 mm focal-length lens would then require an approximate working distance of f/m = 137.5 mm under the distant-object approximation. The actual distance must be refined using the thin-lens equation and the manufacturer’s lens model.

2.4 Perspective, telecentricity, and distortion

Perspective causes apparent size to vary with distance. This is acceptable for many recognition tasks but undesirable for precision dimensional measurement. A telecentric lens restricts chief rays so that magnification changes little over a specified depth range. Object-space telecentric lenses are valuable when parts vary in height or when edges must be measured accurately. Their disadvantages include larger size, higher cost, and restricted field of view.

Common lens distortions include:

• Radial distortion, which bends straight lines outward or inward.
• Tangential distortion, caused by lens and sensor misalignment.
• Chromatic aberration, in which different wavelengths focus at different positions.
• Coma and astigmatism, which degrade off-axis points.
• Vignetting, which reduces brightness toward the image periphery.

Calibration using a known pattern can estimate distortion and intrinsic parameters. Software correction is effective for moderate distortion, but it cannot recover information lost through severe blur or saturation.

3. Radiometric Image Formation

Geometric models determine location; radiometric models determine intensity. A simplified pixel measurement can be expressed as:

Here gᵢⱼ is the digital value of pixel (i, j), tₑ is exposure time, L is scene radiance, T represents optical transmission and pixel-area integration, R is sensor spectral responsivity, λ is wavelength, dA represents sensor area, Q is conversion and quantization, and nᵢⱼ represents noise. This is a conceptual model rather than a complete camera equation. It assumes a linear response over the operating range and separates spectral response from other system effects.

Important radiometric concepts include:

• Radiance: optical power leaving or passing through a surface per unit projected area and solid angle.
• Irradiance: optical power arriving per unit area at the sensor.
• Reflectance: the fraction of incident radiation reflected by a surface, often wavelength-dependent.
• Responsivity: sensor output per unit incident optical energy.
• Dynamic range: the usable ratio between the largest nonsaturated signal and the smallest reliably measurable signal.
• Signal-to-noise ratio (SNR): the strength of the desired signal relative to random variation.

Photon arrival is statistical. If N photons are detected and shot noise is dominant, the standard deviation is approximately √N and the shot-noise SNR is approximately:

This relationship assumes independent photon arrivals and neglects read noise, dark noise, quantization, and fixed-pattern effects. Increasing illumination or exposure can improve photon-limited SNR, but may cause motion blur, saturation, heating, or reduced production speed.

4. Lenses and Optical Selection

Lens selection should begin with the required field of view, working distance, object depth, spectral band, allowable distortion, and measurement accuracy. Focal length alone does not identify whether a lens is suitable.

The aperture is commonly specified by the f-number:

where N is the f-number, f is focal length, and D is the effective entrance-pupil diameter. A smaller f-number admits more light and generally reduces the required exposure, but it also reduces depth of field and may increase aberration. A larger f-number increases depth of field until diffraction becomes significant.

Depth of field

Depth of field is the range of object distances over which image blur remains below an acceptable circle of confusion c. A commonly used approximation for total depth of field is:

where N is f-number, c is the acceptable circle of confusion, and m is magnification. This approximation assumes a thin lens, paraxial imaging, and a specified blur criterion. It becomes less reliable for wide-angle lenses, large perspective changes, and complex optical systems.

For example, with N = 8, c = 0.02 mm, and m = 0.1:

DOF ≈ 2 × 8 × 0.02 × 1.1 / 0.01 = 35.2 mm.

The result is an approximate total object-space depth of field. In a real system, manufacturer data and calibration should be used for final design.

Diffraction and optical resolution

Stopping down the lens improves depth of field but eventually increases diffraction blur. A circular aperture produces an Airy disk whose angular radius to the first minimum is approximately:

where λ is wavelength and D is aperture diameter. The formula assumes a circular aperture and Fraunhofer or sufficiently far-field diffraction conditions. Since D = f/N, increasing N decreases aperture diameter and increases diffraction angle. Therefore, lens choice is a compromise among light collection, depth of field, aberration, and diffraction.

5. Sensors and Camera Performance

5.1 CCD and CMOS

CCD sensors transfer charge through the device to a readout region. Historically, they offered excellent uniformity and low noise, but often consumed more power and operated at lower readout speeds. CMOS sensors provide parallel readout, low power, high frame rates, and flexible region-of-interest operation. Modern CMOS devices can offer excellent noise performance, global shutters, and scientific dynamic range.

A global shutter exposes all pixels at substantially the same time and is preferred for moving objects and robot applications. A rolling shutter exposes different rows at different times. It can produce geometric distortion when the camera or object moves, although it is acceptable for static scenes and some low-cost applications.

5.2 Pixel size, sampling, and resolution

If a sensor dimension S contains P pixels and the field of view is W, the object-space pixel scale is:

where pₒ is expressed in object units per pixel. If a feature of width d must be represented by at least k pixels, a basic sampling requirement is:

This is an engineering rule, not a universal theorem. The required k depends on contrast, noise, edge quality, algorithm, and whether the task is detection, classification, or metrology. The Nyquist sampling frequency for pixel pitch Δ is 1/(2Δ) cycles per unit distance, but adequate sampling also requires appropriate optical modulation transfer function and focus.

5.3 Bit depth and dynamic range

An n-bit digitizer represents 2ⁿ nominal intensity levels. For example, 12-bit data provide 4096 code values. Bit depth does not automatically equal true dynamic range because read noise, nonlinearity, dark current, and saturation constrain the usable range.

Important camera specifications include quantum efficiency, full-well capacity, read noise, dark current, pixel size, frame rate, shutter type, spectral response, gain, offset, linearity, and temperature dependence.

6. Exposure, Focus, and Motion

Exposure is controlled primarily by illumination, aperture, exposure time, and sensor gain. A practical design sequence is to use sufficient illumination, select an aperture that provides the required depth of field, choose a short exposure to limit motion blur, and use gain only when necessary because gain also amplifies noise.

For an object moving with image velocity vᵢ, the image displacement during exposure is approximately:

where b is blur length in image units and tₑ is exposure time. If the maximum permitted blur is bₘₐₓ, then:

This assumes approximately constant velocity and linear motion during exposure. Rotational motion and rolling-shutter effects require further analysis.

Focus should be established at the actual operating distance and verified across the field. Autofocus can be unsuitable in industrial systems because changing scene texture or illumination may alter focus unpredictably. Fixed focus, mechanical locking, and periodic verification are often preferable.

7. Illumination and Optical Filtering

Lighting is frequently the most influential design variable. The objective is not simply maximum brightness but stable contrast between relevant classes.

Common arrangements include:

• Ring lighting: general-purpose illumination around the lens.
• Coaxial lighting: useful for flat reflective surfaces and printed features.
• Backlighting: produces silhouettes for robust outline and dimensional measurement.
• Dark-field lighting: emphasizes scratches, edges, and surface scattering.
• Dome lighting: reduces harsh reflections on curved or glossy objects.
• Structured lighting: projects patterns for shape or depth measurement.
• Diffuse area lighting: produces spatially uniform illumination over broad surfaces.

LEDs are widely used because they are efficient, controllable, and available in many wavelengths. Strobing can provide high instantaneous intensity with a short effective exposure, provided the camera trigger, light controller, and safety limits are coordinated.

Filters restrict the spectral content reaching the sensor. A band-pass filter isolates a wavelength interval; a long-pass filter transmits wavelengths above a cutoff; a short-pass filter transmits wavelengths below a cutoff; and a neutral-density filter reduces intensity without strongly changing spectral composition. Polarizers can suppress specular glare when the illumination and viewing polarization geometry are carefully arranged.

A robust system often uses a narrow-band LED and matching filter. This reduces sensitivity to ambient light and material color variation, but it may also remove information needed to distinguish some materials.

8. Imaging Modalities

Visible and near-infrared imaging

Visible cameras typically operate across approximately 400–700 nm, while near-infrared systems commonly extend beyond 700 nm. NIR can reveal differences in moisture, vegetation, inks, plastics, and biological tissue that are weak in visible light. Silicon sensors often retain useful sensitivity into the near infrared, but a camera intended for visible color imaging may include an IR-cut filter.

Thermal imaging

Thermal cameras measure infrared radiation associated with temperature, usually in long-wave or mid-wave infrared bands. They do not measure temperature directly without assumptions. Emissivity, reflected radiation, atmospheric absorption, viewing angle, and calibration influence the result. Thermal imaging is valuable for overheating components, process monitoring, insulation defects, and human-presence detection, but it is usually unsuitable for fine visible-texture inspection.

Depth imaging

Depth cameras estimate distance using stereo triangulation, structured light, or time of flight. Stereo depends on correspondence and baseline geometry. Structured-light systems project known patterns and infer deformation. Time-of-flight systems measure light travel time or phase shift. Depth accuracy depends on surface reflectivity, texture, ambient light, range, baseline, calibration, and multipath reflections.

Multispectral and hyperspectral imaging

A multispectral camera acquires several selected wavelength bands; a hyperspectral system acquires many narrow, contiguous bands. These systems can separate materials using spectral signatures. They require careful radiometric calibration, stable illumination, and increased data-management capacity. Spectral information is useful only when the selected bands contain discriminative information for the application.

9. Acquisition Interfaces and System Integration

An interface must transport image data reliably within timing, bandwidth, distance, and synchronization constraints. Common choices include USB, GigE Vision, Camera Link, CoaXPress, and vendor-specific industrial links.

For an uncompressed stream, approximate data rate is:

where W and H are image dimensions in pixels, B is bytes per pixel, and F is frame rate in frames per second. For a 2048 × 2048 monochrome camera at 2 bytes per pixel and 30 frames/s:

R = 2048 × 2048 × 2 × 30 = 251,658,240 bytes/s ≈ 252 MB/s.

Protocol overhead, packet gaps, headers, and storage behavior require additional capacity. Region of interest, pixel binning, compression, and reduced frame rate can lower the data rate, although compression may be unsuitable for quantitative inspection if it introduces artifacts.

Hardware triggering is preferred when image capture must be synchronized with a conveyor, encoder, robot, or strobe. A complete design should specify trigger latency, jitter, exposure timing, timestamping, cable length, electromagnetic compatibility, and fault behavior.

10. Calibration and Image Quality Assurance

Calibration converts camera output into useful measurements. It may include:

1. Dark-field correction, using images acquired without illumination.
2. Flat-field correction, compensating for pixel sensitivity variation and uneven illumination.
3. Geometric calibration, estimating intrinsic parameters and lens distortion.
4. Radiometric calibration, relating digital values to optical intensity or temperature.
5. Color or spectral calibration, using reference targets or known standards.
6. Depth calibration, aligning depth and color coordinates and estimating scale.

Calibration must be repeated or checked when the lens, focus, illumination, temperature, mounting, or camera settings change. A calibration file is not a substitute for mechanical stability.

Quality assurance should monitor focus, brightness, saturation percentage, defective pixels, image histogram, geometric reference measurements, and trigger timing. Statistical control charts can reveal gradual lamp aging or camera drift before failures become visible in product data.

11. Illustrative Case: Vision Inspection of Machined Shafts

Background

A manufacturer must inspect steel shafts for diameter, surface scratches, and incorrect end geometry on a conveyor. Shafts vary slightly in height and move continuously.

Problem

A first prototype uses a color camera, diffuse overhead lighting, and a wide-angle lens. Diameter measurements vary with shaft position, polished surfaces create glare, and motion produces occasional blur.

Analysis

The dimensional task requires stable edge localization rather than color. A backlight can create a high-contrast silhouette, while a telecentric lens reduces magnification changes caused by height variation. Because the shaft moves, a global-shutter camera and hardware trigger are required. Surface scratches are a separate task and may require dark-field lighting from a different viewing direction.

Suppose the required field width is 120 mm and the sensor has 2400 pixels across its width. The object-space pixel scale is:

pₒ = 120/2400 = 0.05 mm/pixel.

If a 0.20 mm defect must be detected with at least four pixels, the basic requirement is 0.20/4 = 0.05 mm/pixel, so the design is at the minimum sampling level. In practice, the team selects a narrower field or higher-resolution sensor to provide margin for blur, noise, and imperfect contrast.

Approach and solution

The final system uses a monochrome global-shutter camera, telecentric lens, collimated backlight for silhouette measurement, dark-field LED illumination for scratches, and a hardware trigger from the conveyor encoder. A narrow optical filter suppresses ambient light. Separate acquisition recipes are used for dimensional and surface inspection.

Outcome

Diameter repeatability improves because perspective and illumination variation are reduced. Short exposure limits motion blur, while the dark-field arrangement makes scratches appear as bright scattering features against a dark background. The system also records calibration checks and rejects images with saturation or trigger faults.

Learning points

The case demonstrates that camera resolution alone does not solve an inspection problem. Optical geometry, lighting, sensor timing, spectral filtering, and calibration must be designed as one system. Separating incompatible visual tasks into controlled lighting states may produce more reliable results than searching for one universal image.

12. Challenges, Safety, and Contemporary Practice

Major practical challenges include changing ambient light, reflective surfaces, vibration, thermal drift, contamination of optical windows, lens misalignment, bandwidth limitations, and undocumented automatic camera settings. Automatic exposure and white balance can create frame-to-frame inconsistency; industrial systems commonly lock these parameters after commissioning.

High-intensity visible, ultraviolet, and infrared illumination can create eye and skin hazards. Risk assessment should consider exposure duration, accessible radiation, enclosure design, interlocks, and applicable workplace standards. Thermal cameras and multispectral systems may collect sensitive information, especially in human-monitoring applications. Data governance should define purpose, retention, access, and cybersecurity controls.

Contemporary systems increasingly combine high-dynamic-range sensors, event-based cameras, embedded processing, polarization cameras, short-wave infrared sensors, and AI-based inspection. AI does not remove the need for controlled acquisition. Training data must represent expected variation, and changes in illumination or camera settings can create dataset shift. Explainable acceptance criteria, calibration records, version control, and human review are important for safety-critical or regulated applications.

Best-Practice Design Procedure

1. Define the smallest feature, measurement tolerance, object speed, working distance, and environmental conditions.
2. Choose the imaging modality and spectral band based on material physics.
3. Select lighting to maximize task-relevant contrast and suppress irrelevant variation.
4. Determine field of view, pixel scale, sensor format, focal length, and magnification.
5. Check depth of field, diffraction, distortion, and expected blur.
6. Select global or rolling shutter according to motion.
7. Calculate bandwidth, storage, trigger timing, and processing requirements.
8. Calibrate geometry and radiometry under production conditions.
9. Validate with representative worst-case samples, not only ideal parts.
10. Monitor image quality during operation and document all fixed settings.

Chapter Summary

Image formation combines geometry and radiometry. Perspective projection determines where scene points appear, while radiometric processes determine measured intensity. Thin-lens relations connect focal length, object distance, image distance, and magnification. Field of view, pixel scale, depth of field, diffraction, distortion, and motion blur provide the principal optical design constraints.

Camera selection requires attention to sensor type, pixel size, shutter behavior, quantum efficiency, noise, dynamic range, bit depth, spectral response, and frame rate. Illumination is a central part of the measurement system: backlighting supports silhouettes, dark-field lighting reveals scattering defects, diffuse lighting controls reflections, and spectral filtering improves selectivity. Visible, infrared, thermal, depth, and multispectral systems measure different physical phenomena and therefore require different calibration and interpretation.

Reliable machine vision depends on integrated design, hardware synchronization, bandwidth planning, calibration, safety, and quality monitoring. The most effective system is not necessarily the one with the largest number of pixels, but the one that produces stable, task-relevant measurements.

Key Terms

Geometric image formation: The relationship between three-dimensional scene structure and image location.

Radiometric image formation: The process by which optical energy becomes a measured digital value.

Perspective projection: Projection in which apparent size varies with distance.

Focal length: Optical distance that characterizes the converging power of a lens.

Working distance: Distance between the lens reference plane and the object being imaged.

Field of view: Physical region represented by an image.

Magnification: Ratio of image size to object size.

Telecentric lens: Lens designed to reduce magnification change with object distance.

Distortion: Geometric departure of an image from ideal projection.

Aperture: Opening that controls the amount of light entering a lens.

F-number: Ratio of focal length to effective aperture diameter.

Depth of field: Object-space distance range producing acceptable sharpness.

Diffraction: Wave-optical spreading caused by a finite aperture.

Global shutter: Exposure method in which pixels are captured simultaneously.

Rolling shutter: Exposure method in which sensor rows are captured sequentially.

Dynamic range: Ratio between the largest usable and smallest measurable signal.

Radiance: Directional optical power leaving or passing through a surface.

Flat-field correction: Correction for nonuniform illumination and pixel sensitivity.

Multispectral imaging: Imaging using several selected spectral bands.

Hardware trigger: Electrical timing signal used to coordinate acquisition.

Review Questions

Short Answer Questions

1. What is the difference between geometric and radiometric image formation?
2. State the thin-lens equation and define its variables.
3. Why is a global shutter generally preferred for moving objects?
4. What is the purpose of a telecentric lens?
5. Name two methods for reducing the effects of nonuniform illumination.

Long Answer Questions

1. Explain how focal length, sensor size, working distance, and field of view are related.
2. Compare CCD and CMOS sensors in terms of readout, speed, power, and industrial use.
3. Discuss the trade-offs involved in selecting the lens aperture.
4. Compare visible, near-infrared, thermal, depth, and multispectral imaging.
5. Describe a complete calibration and quality-assurance procedure for an industrial camera.

Analytical / Application Questions

1. A camera observes a 160 mm wide field using 3200 pixels across the field. Calculate the object-space pixel scale and assess whether a 0.4 mm feature represented by eight pixels meets the basic sampling requirement.
2. An object has image velocity 180 pixels/s, and allowable motion blur is 0.6 pixels. Determine the maximum exposure time and discuss two ways to implement the result in hardware.
3. Design an imaging arrangement for measuring the outline of glossy metal components moving on a conveyor. Justify the camera shutter, lighting geometry, lens type, filtering, triggering, and calibration strategy.

Multiple Choice Questions with Answers

Question 1

Which equation describes ideal perspective projection for the horizontal coordinate?

A. x = fX/Z

B. x = fZ/X

C. x = XZ/f

D. x = X + f/Z

Correct Answer: A. x = fX/Z

Question 2

A sensor is 10 mm wide and images a 250 mm object field. What is the approximate magnification?

A. 0.004

B. 0.04

C. 0.4

D. 25

Correct Answer: B. 0.04

Question 3

Which shutter type best limits geometric distortion when imaging a fast-moving object?

A. Rolling shutter

B. Global shutter

C. Manual shutter

D. Electronic gain shutter

Correct Answer: B. Global shutter

Question 4

For a lens with focal length 25 mm and aperture diameter 5 mm, what is the f-number?

A. 0.2

B. 5

C. 20

D. 125

Correct Answer: B. 5

Question 5

Which lighting arrangement is usually most suitable for measuring the outer silhouette of an opaque part?

A. Backlighting

B. Coaxial lighting

C. Dome lighting only

D. Front flash lighting

Correct Answer: A. Backlighting

Question 6

If 4096 intensity levels are available from a digitizer, what is its nominal bit depth?

A. 8 bit

B. 10 bit

C. 12 bit

D. 16 bit

Correct Answer: C. 12 bit

Question 7

A 2048 × 2048 monochrome camera uses 2 bytes per pixel at 30 frames/s. What approximate uncompressed data rate is required?

A. 25.2 MB/s

B. 75.5 MB/s

C. 251.7 MB/s

D. 1.26 GB/s

Correct Answer: C. 251.7 MB/s

Question 8

Which modality measures radiation associated primarily with object temperature?

A. Visible imaging

B. Thermal imaging

C. Stereo imaging

D. Ultrasonic imaging

Correct Answer: B. Thermal imaging

Question 9

If an object moves at 200 image pixels/s and allowable blur is 1 pixel, what is the maximum exposure time?

A. 0.5 ms

B. 5 ms

C. 20 ms

D. 200 ms

Correct Answer: B. 5 ms

Question 10

What is the principal purpose of flat-field correction?

A. To estimate object depth from stereo

B. To correct illumination and pixel-response nonuniformity

C. To increase lens focal length

D. To convert a rolling shutter into a global shutter

Correct Answer: B. To correct illumination and pixel-response nonuniformity