Digital Image Representation and Mathematical Foundations

Introduction

Machine vision systems convert light from the physical world into numerical data that can be stored, processed, measured, and interpreted. A camera image is therefore not merely a picture: it is a sampled and quantized representation of an optical signal. The quality of every later task—inspection, robot guidance, metrology, recognition, and defect detection—depends on how faithfully that representation preserves relevant information.

This chapter develops the mathematical foundations required to understand digital images. It examines sampling and quantization, pixel organization, color spaces, neighborhoods, convolution, linear algebra, probability, statistics, Fourier analysis, spatial frequency, aliasing, noise, and information loss.

Learning Objectives

After studying this chapter, you should be able to:

1. Explain how a continuous optical scene becomes a digital image.
2. Distinguish spatial sampling, intensity quantization, resolution, and dynamic range.
3. Describe common grayscale and color representations.
4. Use pixel neighborhoods and convolution for image filtering.
5. Express images and transformations using vectors and matrices.
6. Apply basic probability and statistics to image measurements.
7. Explain spatial frequency, Fourier representations, aliasing, and noise.
8. Select suitable acquisition and preprocessing methods for a machine-vision task.

1. From Scene to Digital Image

A real scene produces an optical irradiance distribution that varies continuously with position. An ideal grayscale image may be represented as a function

where x and y are spatial coordinates, usually measured in metres or millimetres, and f is irradiance or reflected intensity. A camera lens projects the scene onto a sensor. Each sensor element integrates incoming light over a finite area and over an exposure time. The resulting electrical signal is then amplified, digitized, and stored as pixel values.

A digital image is commonly written as

where m and n are integer row and column indices. For a grayscale image, I[m,n] is one scalar. For a color image, it is a vector such as [R,G,B]. The digital image is discrete in position and usually discrete in amplitude.

The complete acquisition chain is therefore:

optical scene → lens and illumination → sensor integration → sampling → quantization → digital processing.

Each stage can introduce blur, distortion, noise, saturation, or information loss.

2. Spatial Sampling

2.1 Sampling grid

If samples are taken at horizontal spacing Δx and vertical spacing Δy, the sample coordinates are

and the sampled image is approximately

This expression assumes ideal point sampling. Real pixels have finite area, so the sensor records an area average rather than a value at a mathematical point.

The sampling frequencies are

with units of samples per unit distance. Smaller pixel spacing gives a higher sampling frequency and potentially greater spatial resolution.

2.2 Resolution and field of view

If a sensor has N_x columns and N_y rows and the field of view is W × H, then the approximate pixel pitches are

For a 2048 × 1536 sensor viewing a 204.8 mm × 153.6 mm area, the pitch is 0.1 mm per pixel in both directions. A feature smaller than approximately one pixel cannot be reliably measured, and a feature only a few pixels wide may be difficult to classify robustly.

Resolution is not determined by pixel count alone. Lens sharpness, focus, motion, diffraction, sensor pixel size, illumination, and image-processing methods also contribute.

2.3 Sampling theorem and aliasing

For a one-dimensional signal containing no frequencies above f_max, ideal uniform sampling can reconstruct the signal if

This is the Nyquist condition. The frequency f_s/2 is called the Nyquist frequency. The theorem assumes appropriate band limitation and ideal sampling and reconstruction. Real imaging systems do not satisfy these assumptions perfectly.

When a signal contains frequencies above the Nyquist frequency, different continuous signals can produce the same samples. This phenomenon is aliasing. In images, aliasing may appear as jagged edges, false patterns, moiré fringes, or incorrect texture orientations.

A practical anti-aliasing strategy is to optically blur or low-pass filter the image before sampling. This removes frequency components that the sensor cannot represent. The trade-off is reduced sharpness.

3. Quantization and Pixel Encoding

3.1 Amplitude quantization

After sampling, each sensor value is mapped to one of a finite number of intensity levels. With b bits per pixel, the number of levels is

For an ideal unsigned b-bit sensor, the code values range from 0 to L − 1. An 8-bit image has 256 possible levels, while a 12-bit image has 4096 levels.

If the analog input range is [a_min,a_max], the ideal quantization step is

Under a uniform quantizer and assuming the signal uses the full range, the quantization error e lies approximately in [−Δ_q/2, Δ_q/2]. Its variance is often modeled as

This model assumes quantization error is approximately uniform and uncorrelated with the signal. It is less accurate for small or highly structured signals.

3.2 Dynamic range and saturation

Dynamic range is the ratio between the largest non-saturating signal and the smallest useful signal. It may be expressed in decibels as

where A_max and A_min are positive amplitudes. A sensor saturates when the signal exceeds its maximum code. Saturation destroys distinctions among bright values and cannot generally be corrected after acquisition.

Bit depth controls amplitude resolution, but it does not recover detail lost through poor focus, motion blur, or inadequate spatial sampling.

Worked Example: Quantization

An ideal 8-bit camera maps 0–1.0 normalized intensity into 256 levels. The quantization interval is

Δ_q = 1/256 = 0.00390625.

The maximum magnitude of the ideal rounding error is approximately 0.001953. Under the uniform-error model, the standard deviation is

σ_q = Δ_q/√12 ≈ 0.001128.

This is an ideal estimate; sensor noise, nonlinearity, and illumination variation may be much larger.

4. Image Representation

4.1 Grayscale images

A grayscale image is an M × N matrix. The value I[m,n] may represent brightness, reflectance, temperature, or another measured quantity. The coordinate convention must be documented because some software uses row-column order while mathematical descriptions often use x-y order.

A binary image contains two classes, commonly represented by 0 and 1. Binary images are useful for masks, connected components, and simple inspection, but thresholding discards intensity information.

4.2 Color images and color spaces

A color pixel is commonly represented by three or more components. The choice of color space affects interpretation, segmentation, storage, and computation.

RGB is an additive representation. It is not automatically device-independent: the same numerical RGB triplet can correspond to different colors on different cameras or displays. Color calibration requires a defined illuminant, sensor characterization, and often a color reference target.

In many tasks, separating brightness from chromatic information is useful. For example, a colored defect may be easier to segment in HSV or Lab than in RGB. However, transformations do not remove the effects of shadows, specular reflection, or changing illumination.

5. Pixels, Neighborhoods, and Connectivity

A pixel has neighbors defined by a chosen spatial relationship. The four-neighborhood of pixel (m,n) is

The diagonal neighbors form the set

The eight-neighborhood is N₄ ∪ N_D. Boundary pixels have fewer neighbors unless padding is introduced.

Connectivity matters in segmentation and object measurement. Four-connectivity prevents diagonal touching pixels from automatically forming one object; eight-connectivity treats diagonal contact as connected. Inconsistent choices can produce different component counts and areas.

Common boundary conditions include:

• zero padding: outside values are set to zero;
• replication: the nearest boundary value is extended;
• reflection: values are mirrored;
• circular or periodic padding: opposite edges are joined.

Padding affects filtered values near image boundaries and must be selected deliberately.

6. Linear Algebra for Images

An image can be treated as a matrix, a vector, or a point in a high-dimensional space. If an M × N image is stacked columnwise, it becomes a vector in R^(MN). This representation supports transformations such as

where x is the input image vector, A is a linear operator, and y is the output vector. A blur, for example, can be represented by a large structured matrix, although explicitly constructing A is usually inefficient.

A color pixel can be represented as

A linear color transformation is

where M is a 3 × 3 matrix. Such transformations are useful for color correction and conversion, but nonlinear operations such as gamma correction and clipping must be treated separately.

The Euclidean norm of a vector is

and the mean-squared error between images x and y containing K corresponding values is

MSE is mathematically convenient but does not always correspond to perceived visual quality or task performance.

7. Convolution and Linear Filtering

For a discrete image I and kernel h of size (2a+1) × (2b+1), two-dimensional convolution is

The kernel is flipped relative to correlation. Many software libraries implement correlation while calling it convolution; this distinction matters for asymmetric kernels.

A normalized averaging kernel is

It performs local smoothing and reduces independent random noise, but it blurs edges. A Gaussian kernel has weights that decrease with distance from the center and is commonly used for scale control and anti-aliasing.

An edge detector may use a derivative-like kernel. For example, the horizontal Sobel operator is

The corresponding vertical operator is

The gradient magnitude is often estimated by

where G_x and G_y are the filtered responses at the same pixel. This is a local approximation to the magnitude of the image gradient.

Worked Example: Local averaging

Suppose a 3 × 3 neighborhood contains eight pixels with value 100 and a central pixel with value 140. A 3 × 3 normalized averaging filter produces

(8×100 + 140)/9 = 940/9 ≈ 104.44.

The impulse-like central variation is reduced, but the result is no longer an exact representation of the original center. This illustrates the fundamental smoothing trade-off.

8. Probability and Statistics in Images

Image measurements are often modeled as random variables because illumination, sensor electronics, surface texture, and transmission introduce variability. For a random variable X, the expected value is

and the variance is

For K observed values x₁,…,x_K, the sample mean and unbiased sample variance are

The denominator K−1 is used when estimating population variance from independent samples. If the complete population is known, division by K is appropriate.

The covariance between variables X and Y measures joint variation:

The correlation coefficient is

provided both standard deviations are nonzero. Correlation indicates linear association, not causation.

Common noise models

• Gaussian noise: additive variation often associated with electronics and readout.
• Poisson noise: signal-dependent photon-counting variation; variance is related to the mean number of detected photons.
• Salt-and-pepper noise: sparse extreme-valued corruption.
• Fixed-pattern noise: spatially repeatable sensor variation.
• Speckle: multiplicative interference-related variation in coherent imaging.

Noise models are approximations. A good preprocessing method should be selected using the sensor and application, not by assuming all noise is Gaussian.

9. Fourier Analysis and Spatial Frequency

The two-dimensional discrete Fourier transform (DFT) of an M × N image is

The inverse DFT is

Here j is the imaginary unit, u and v are discrete frequency indices, and exp denotes the complex exponential. The DFT assumes periodic extension of the finite image, which can create boundary discontinuities and spectral leakage.

The magnitude |F[u,v]| indicates the strength of frequency components, while the phase carries positional and structural information. Low spatial frequencies describe slowly varying illumination and broad shapes. High spatial frequencies describe fine texture, narrow edges, and noise.

The convolution theorem states that convolution in the spatial domain corresponds to multiplication in the frequency domain:

Thus, a low-pass filter attenuates high frequencies, while a high-pass filter emphasizes rapid spatial changes. Fourier analysis is valuable for diagnosing periodic defects, selecting filter scales, and understanding blur.

10. Information Loss and System Design

Information can be lost through:

• undersampling and aliasing;
• quantization and insufficient bit depth;
• sensor saturation;
• clipping during arithmetic;
• lens blur and defocus;
• motion during exposure;
• compression, especially lossy compression;
• thresholding and binary conversion;
• inappropriate color conversion;
• aggressive denoising.

No later algorithm can reliably reconstruct distinctions that were never recorded. Super-resolution and deconvolution can produce useful estimates when supported by a valid model, but they do not guarantee recovery of unique original data.

A robust acquisition design begins with the smallest feature that must be detected, the required measurement accuracy, expected contrast, illumination variation, and motion. Sampling, optics, exposure, gain, bit depth, and filtering should be designed as one system.

11. Applied Case: Vision Inspection of a Moving Conveyor Part

Background

Consider an illustrative automated inspection cell for aluminum parts moving on a conveyor. The system must detect scratches at least 0.25 mm wide over a 100 mm-wide field of view. Parts move during exposure, and the surface contains periodic machining marks.

Problem analysis

A camera with 2000 horizontal pixels gives a nominal sampling pitch of

100/2000 = 0.05 mm per pixel.

A 0.25 mm scratch spans approximately five pixels, which is potentially detectable. However, detection depends on contrast and orientation. If the lens is blurred so that the scratch edge spreads over several pixels, its contrast decreases.

The machining marks may contain spatial frequencies near the sensor’s Nyquist limit. Sampling them without optical filtering could create false patterns. Conveyor motion can produce directional blur, particularly during long exposure.

Approach and solution

The engineering team should:

1. Use diffuse, stable illumination to reduce specular variation.
2. Choose optics that provide adequate sharpness across the 100 mm field.
3. Use an exposure short enough that motion displacement is a small fraction of a pixel, or synchronize illumination with a strobe.
4. Apply an optical low-pass effect sufficient to suppress unresolvable machining frequencies.
5. Acquire more than 8 bits if the scratch contrast is small relative to the background.
6. Estimate background intensity and noise from representative production images.
7. Use a Gaussian filter only if its loss of scratch contrast is acceptable.
8. Detect local edges or intensity deviations using calibrated thresholds.
9. Validate performance on normal parts, defective parts, illumination changes, and sensor drift.

Outcome

The key result is not simply a sharper image. It is a repeatable image in which the smallest required scratch produces a measurable, non-aliased contrast difference. A system with fewer pixels but better lighting and optics may outperform a higher-resolution system with saturation, glare, or motion blur.

Learning points

The case demonstrates that sampling theory, noise statistics, convolution, and practical machine design are inseparable. Image processing cannot compensate indefinitely for poor acquisition.

12. Best Practices and Contemporary Relevance

Good practice includes preserving raw images, recording camera settings, calibrating geometric and radiometric behavior, documenting coordinate conventions, avoiding unnecessary conversions, and testing algorithms against controlled variations. Measurements should be traceable to physical units when the system is used for metrology.

Modern machine-vision systems often combine classical preprocessing with machine learning. Neural networks still depend on image quality: aliasing, saturation, class imbalance, and uncontrolled illumination can reduce generalization. Data governance should include representative data collection, protection of identifiable imagery where applicable, auditability of automated decisions, and human review for safety-critical outcomes.

Chapter Summary

A digital image is a discrete representation of a continuous optical scene. Spatial sampling determines where measurements are taken, while quantization determines the available intensity levels. The Nyquist condition explains why undersampling causes aliasing, and optical prefiltering limits unrepresentable frequencies. Bit depth, dynamic range, saturation, and noise determine amplitude fidelity.

Images can be represented as matrices, vectors, and multichannel color fields. Pixel neighborhoods support connectivity, local statistics, and filtering. Convolution provides a mathematical model for smoothing, sharpening, and edge detection, while linear algebra describes image transformations. Probability and statistics quantify image variation, noise, correlation, and measurement uncertainty. Fourier analysis connects spatial structures to frequency components and explains the behavior of low-pass and high-pass filters.

In machine vision, acquisition and computation must be designed together. Correct sampling, controlled illumination, suitable optics, calibrated sensors, and validated preprocessing are essential because information lost through aliasing, saturation, blur, or quantization cannot be perfectly recovered.

Key Terms

Sampling: Measuring a continuous signal at discrete spatial or temporal locations.

Quantization: Mapping continuous or high-precision amplitudes to finite numerical levels.

Pixel: A discrete image element representing a sampled measurement.

Spatial resolution: Ability to distinguish nearby spatial features.

Dynamic range: Ratio between the largest useful signal and the smallest useful signal.

Aliasing: False representation caused by sampling frequencies beyond the Nyquist limit.

Nyquist frequency: Half the sampling frequency under uniform sampling.

Saturation: Condition in which a sensor or numerical representation reaches its maximum or minimum code.

Color space: A coordinate system used to represent color.

Neighborhood: Set of pixels spatially related to a selected pixel.

Connectivity: Rule determining whether pixels belong to the same region or object.

Kernel: Finite array of coefficients used in filtering.

Convolution: Linear operation combining an image with a spatially shifted and reversed kernel.

Noise: Unwanted random or systematic variation in an image measurement.

Variance: Statistical measure of squared deviation from the mean.

Covariance: Measure of joint variation between two variables.

Spatial frequency: Rate at which image intensity changes over distance.

Fourier transform: Representation of a signal in terms of frequency components.

Low-pass filter: Filter that preferentially retains low spatial frequencies.

Information loss: Irrecoverable reduction of distinguishable signal content.

Review Questions

Short Answer Questions

1. What is the difference between spatial sampling and intensity quantization?
2. Why can increasing bit depth not correct lens blur?
3. Define four-connectivity and eight-connectivity.
4. What type of image structure is represented primarily by high spatial frequencies?
5. Why is sensor saturation a serious problem in machine vision?

Long Answer Questions

1. Explain the complete process by which an optical scene becomes a digital image, identifying sources of information loss.
2. Derive and discuss the practical meaning of the Nyquist sampling condition.
3. Compare RGB, HSV, and CIE Lab* for industrial color inspection.
4. Explain convolution, correlation, and the role of boundary conditions in filtering.
5. Describe how Fourier analysis helps an engineer understand blur, texture, and noise.

Analytical / Application Questions

1. A camera views a 240 mm-wide region using 2400 columns. Calculate the nominal horizontal pixel pitch. If a defect is 0.6 mm wide, approximately how many pixels span it, and what practical factors could still prevent reliable detection?
2. An inspection image contains periodic false bands after resizing. Analyze how aliasing could have caused the bands and propose an acquisition and preprocessing solution.
3. Design a measurement plan for a conveyor inspection system in which illumination changes, motion blur, and sensor noise are all present. Specify what should be measured, calibrated, and validated before deployment.

Multiple Choice Questions with Answers

Question 1

A sensor samples an image at 20 samples per millimetre. What is its one-dimensional Nyquist frequency?

A. 5 cycles/mm

B. 10 cycles/mm

C. 20 cycles/mm

D. 40 cycles/mm

Correct Answer: B. 10 cycles/mm

Question 2

How many discrete intensity levels are available in an ideal 10-bit unsigned image?

A. 10

B. 256

C. 512

D. 1024

Correct Answer: D. 1024

Question 3

Which operation most directly reduces aliasing before spatial sampling?

A. Optical low-pass filtering

B. Increasing image saturation

C. Applying a binary threshold after sampling

D. Increasing display brightness

Correct Answer: A. Optical low-pass filtering

Question 4

Which expression is the sample mean of K image measurements xᵢ?

A. (1/K)∑ᵢ xᵢ

B. ∑ᵢ(xᵢ−x̄)²

C. (1/(K−1))∑ᵢ xᵢ²

D. √(∑ᵢ xᵢ²)

Correct Answer: A. (1/K)∑ᵢ xᵢ

Question 5

A 3 × 3 normalized averaging filter is primarily used to:

A. Increase quantization levels

B. Smooth local variations and reduce some noise

C. Guarantee perfect edge preservation

D. Convert RGB directly to Lab

Correct Answer: B. Smooth local variations and reduce some noise

Question 6

In a discrete image, eight-connectivity includes:

A. Only the left and right pixels

B. Only diagonal pixels

C. Horizontal, vertical, and diagonal neighboring pixels

D. All pixels in the entire image

Correct Answer: C. Horizontal, vertical, and diagonal neighboring pixels

Question 7

In Fourier analysis, low spatial frequencies generally correspond to:

A. Slowly varying illumination and broad shapes

B. Only isolated sensor defects

C. Pixel addresses

D. Quantization codes exclusively

Correct Answer: A. Slowly varying illumination and broad shapes

Question 8

What is the main consequence of sensor saturation?

A. Bright values become more distinguishable

B. Different high signals are mapped to the same maximum code

C. The sampling frequency doubles

D. Noise is mathematically eliminated

Correct Answer: B. Different high signals are mapped to the same maximum code

Question 9

For an M × N image, stacking all pixels into one vector produces a vector with:

A. M + N elements

B. M − N elements

C. MN elements

D. M/N elements

Correct Answer: C. MN elements

Question 10

Which statement about correlation is correct?

A. It proves that one image variable causes another

B. It measures linear association when standard deviations are nonzero

C. It is always equal to covariance

D. It can only be used for binary images

Correct Answer: B. It measures linear association when standard deviations are nonzero