3. Image Enhancement, Restoration, and Transformation

Introduction and Context

Image processing operations are often divided into three related but distinct activities. Image enhancement improves the visual appearance or interpretability of an image, usually according to a human or application-specific objective. Image restoration attempts to recover an image that has been degraded by blur, noise, sampling, or a physical imaging process. Image transformation changes the representation, geometry, scale, or domain of an image so that it can be stored, analyzed, transmitted, or used by a computer-vision system.

These operations are central to computer vision because images acquired from cameras, scanners, satellites, medical instruments, and industrial sensors are rarely ideal. Illumination may be uneven, contrast may be poor, motion may produce blur, sensors may introduce noise, and geometric differences may prevent direct comparison. Enhancement and restoration therefore frequently precede segmentation, feature extraction, recognition, and deep-learning inference. However, preprocessing is not automatically beneficial. An operation that improves visual appearance may remove weak but diagnostically important information or create artificial structures. The appropriate method depends on the image-formation process, the intended task, and the consequences of error.

This chapter develops a unified view of spatial and frequency-domain filtering, histogram methods, denoising, deblurring, morphology, interpolation, geometric transformation, compression, and image-quality assessment. It also considers how these techniques are selected in professional AI and ML pipelines.

Learning Objectives

After completing this chapter, the reader should be able to:

1. Distinguish enhancement, restoration, and transformation and select an appropriate operation for a stated objective.

2. Explain spatial-domain and frequency-domain filtering using mathematical models.

3. Apply histogram equalization, contrast stretching, smoothing, sharpening, and adaptive enhancement appropriately.

4. Compare common noise-reduction and deblurring methods, including their assumptions and limitations.

5. Use morphological operations for shape-based image analysis.

6. Explain interpolation and geometric transformations and evaluate their effect on image information.

7. Describe image compression methods and the difference between objective and perceptual image quality.

8. Design a defensible preprocessing workflow for a computer-vision application.

1. A Working Model of Image Degradation and Improvement

A digital image can be represented as a discrete function \(f(x,y)\), where \(x\) and \(y\) denote spatial coordinates and the function value denotes intensity or colour. A useful degradation model is

\[

g(x,y)=h(x,y)*f(x,y)+n(x,y),

\]

where \(g\) is the observed image, \(h\) is the point-spread function (PSF), * denotes convolution, and \(n\) is noise. The PSF describes how an imaging system spreads a point. Defocus, atmospheric turbulence, lens imperfections, and motion can all produce different PSFs. Restoration seeks an estimate of \(f\) from \(g\), while enhancement may improve \(g\) without attempting to reconstruct the original scene.

The distinction is important. Histogram equalization can increase contrast but cannot reverse motion blur. A Wiener filter uses a degradation model and statistical assumptions to estimate the latent image; it is therefore a restoration method. Resizing an image changes its sampling grid and is a transformation. In practice, the categories overlap: denoising may both restore fidelity and enhance visibility, while geometric registration may be necessary before restoration.

Two principles guide responsible processing. First, operations should be traceable: the practitioner should know which transformation was applied and with what parameters. Second, evaluation must be task-specific. A sharper image is not necessarily more accurate for object detection, and a higher peak signal-to-noise ratio (PSNR) does not always indicate better perceptual quality.

2. Spatial-Domain Enhancement

Spatial processing operates directly on pixels. A point operation changes each pixel independently:

\[

s=T(r),

\]

where \(r\) is the input intensity and \(s\) is the output intensity. Linear contrast stretching maps a limited intensity interval to a wider one. Gamma correction uses \(s=cr^\gamma\); values of \(\gamma<1\) brighten dark regions, whereas \(\gamma>1\) tends to darken them. Logarithmic transformation expands low intensities and compresses high intensities, which is useful when the dynamic range is large.

Histogram methods use the distribution of intensities. A histogram records the number of pixels at each intensity level. Histogram equalization constructs a mapping from the cumulative distribution function (CDF), approximately spreading frequently occurring levels across the available range. It can reveal detail in low-contrast images, but global equalization may over-amplify noise or fail when illumination varies spatially.

Contrast-limited adaptive histogram equalization (CLAHE) divides the image into local tiles, limits histogram peaks, and then interpolates the mappings between tiles. It is often more suitable for uneven illumination, although an excessively high clip limit can create texture that was not present in the scene. Histogram matching, or specification, maps an image toward a selected target distribution; it is useful when images from different sources must have more comparable appearance.

Neighbourhood operations use a window around each pixel. A linear filter computes a weighted sum:

\[

g(x,y)=\sum_{i=-k}^{k}\sum_{j=-k}^{k}w(i,j)f(x-i,y-j).

\]

A mean filter reduces random variation but blurs edges. A Gaussian filter assigns larger weights near the centre and provides controlled low-pass smoothing. A median filter replaces a pixel by the neighbourhood median and is particularly effective against impulse, or salt-and-pepper, noise because it preserves many discontinuities better than averaging.

Sharpening emphasizes intensity transitions. The Laplacian, a second derivative, can be combined with the original image in an unsharp-masking procedure. In a common form, a blurred version is subtracted from the original to obtain a detail mask, which is then added back with a chosen gain. Excessive sharpening produces halos and amplifies noise. Gradient operators such as Sobel and Scharr are often used not only for enhancement but also for edge measurement.

3. Frequency-Domain Reasoning

The Fourier transform represents an image as a combination of spatial frequencies. Slowly varying illumination and broad regions correspond primarily to low frequencies, while edges, fine texture, and noise contribute higher frequencies. In the frequency domain, convolution becomes multiplication:

\[

G(u,v)=H(u,v)F(u,v)+N(u,v).

\]

A low-pass filter attenuates high frequencies and therefore smooths the image. An ideal low-pass filter has an abrupt cutoff but may cause ringing. Butterworth filters provide a smoother transition, while Gaussian filters have no sharp cutoff and generally produce fewer ringing artefacts. High-pass filtering emphasizes rapid changes and can support sharpening or edge analysis. Band-pass filtering isolates a selected range of spatial detail; notch filters can suppress periodic interference at specified frequencies.

Frequency-domain processing is especially useful when degradation is periodic or when the filter has a large, spatially invariant support. Nevertheless, Fourier filtering assumes care with boundaries and often requires padding. An apparently isolated frequency peak may represent interference, but suppressing it can also remove genuine repetitive structure. Frequency analysis should therefore be interpreted rather than applied mechanically.

4. Noise Reduction and Image Restoration

Noise models influence method selection. Gaussian noise is often associated with electronic measurement variation. Salt-and-pepper noise produces isolated extreme values and may arise from transmission or sensor faults. Poisson noise is signal-dependent and common in photon-limited imaging. Speckle is multiplicative and occurs in modalities such as ultrasound and coherent radar. A method designed for additive Gaussian noise may perform poorly on multiplicative or impulse noise.

Non-local means averages pixels with similar surrounding patches, even when they are far apart spatially. It can preserve repeated structures better than local smoothing, but it is computationally more demanding. Bilateral filtering combines spatial proximity with intensity similarity, smoothing within regions while limiting averaging across strong edges. Guided filtering uses a guidance image, often the image itself, to produce edge-aware smoothing. Modern learned denoisers can achieve excellent results, but they may hallucinate texture or fail under a distribution shift between training and deployment data.

Deblurring is an inverse problem. If the PSF is known, inverse filtering attempts to divide by the frequency response of the blur. Frequencies at which \(H(u,v)\) is small make this operation unstable because noise is strongly amplified. Wiener filtering balances inverse restoration against noise using estimates of signal and noise power spectra. Regularized approaches, including Tikhonov methods, impose a preference for stable or smooth solutions. Blind deconvolution estimates both the latent image and the unknown blur, but the problem is underdetermined and requires constraints or priors.

Restoration quality depends on model accuracy. A wrong motion direction, incorrect blur length, or inaccurate noise estimate can produce ringing, double edges, and misleading detail. In scientific and medical contexts, the original and restored images should be retained separately, with metadata recording the algorithm and parameters.

5. Morphological Processing

Mathematical morphology analyzes shapes using a structuring element, such as a square, disk, or cross. It is most naturally introduced for binary images but also has grayscale versions. Erosion shrinks foreground regions and removes small objects; dilation expands regions and can bridge narrow gaps. Opening, erosion followed by dilation, removes small protrusions and isolated foreground components. Closing, dilation followed by erosion, fills small holes and connects nearby components.

Other useful operations include the morphological gradient, which approximates boundaries; the top-hat transform, which extracts small bright objects or local brightness variations; and the black-hat transform, which highlights small dark structures. In document analysis, opening can remove specks while closing can repair broken characters. In industrial inspection, morphology can separate connected defects or estimate background illumination.

The structuring element encodes a shape assumption. A disk is appropriate for approximately circular objects, whereas a line element can detect or preserve oriented structures. Poor selection may eliminate legitimate features or connect unrelated objects. Morphology should therefore be linked to object scale and geometry rather than treated as a universal cleanup operation.

6. Interpolation and Geometric Transformation

Transformations map coordinates from one image grid to another. Translation, rotation, scaling, reflection, and shearing are examples of affine transformations. In homogeneous coordinates, an affine mapping can be written as

\[

\begin{bmatrix}x'\\y'\\1\end{bmatrix}=

\begin{bmatrix}a&b&t_x\\c&d&t_y\\0&0&1\end{bmatrix}

\begin{bmatrix}x\\y\\1\end{bmatrix}.

\]

A projective, or perspective, transformation uses a homography and can map a planar document viewed at an angle to a rectangular view. Registration aligns images from different times, sensors, or viewpoints. It may use control points, feature correspondences, intensity optimization, or learned correspondence models.

Because transformed coordinates usually fall between pixels, interpolation is required. Nearest-neighbour interpolation is fast and preserves categorical labels, making it appropriate for segmentation masks. Bilinear interpolation uses the four nearest pixels and provides smoother results. Bicubic interpolation uses a larger neighbourhood and can produce sharper but occasionally overshooting values. For repeated transformations, resampling should be minimized because every interpolation can remove high-frequency information.

Aliasing occurs when sampling cannot represent image detail adequately. Downsampling should normally be preceded by low-pass filtering. In a machine-learning pipeline, image resizing must be applied consistently to images, masks, bounding boxes, and keypoints; using ordinary interpolation for a class-label mask can create invalid intermediate classes.

7. Compression and Representation

Compression reduces storage or transmission requirements. Lossless methods preserve every pixel and are appropriate for scientific measurements, archival originals, and segmentation labels. Lossy methods exploit perceptual redundancy and usually achieve higher compression ratios. Transform coding commonly divides an image into blocks, transforms them, quantizes coefficients, and entropy-codes the result. Quantization is the principal irreversible step and may produce blocking, ringing, and loss of fine texture.

Compression should be selected according to the information value of image detail. A visually acceptable image may be unsuitable for metrology or diagnosis. For AI datasets, compression artefacts can become unintended cues: a model may learn camera or compression signatures rather than the target phenomenon. Dataset construction should therefore standardize, document, and where possible diversify acquisition and compression conditions.

8. Assessing Image Quality

Objective measures compare a processed image with a reference. Mean squared error is

\[

MSE=\frac{1}{N}\sum_{i=1}^{N}(f_i-\hat f_i)^2,

\]

and PSNR is derived from the ratio between the maximum signal value and MSE. Higher PSNR generally indicates lower error under the assumed pixel-wise model. Structural similarity (SSIM) compares local luminance, contrast, and structure and often corresponds better to perception than MSE. Multi-scale SSIM extends this reasoning across resolutions.

When no pristine reference exists, no-reference metrics estimate quality from natural-image statistics or learned models. For an operational computer-vision system, task-based measures are often more meaningful: segmentation overlap, detection precision and recall, recognition accuracy, calibration, or downstream error. Enhancement should be validated on the final task and not only by visual inspection.

9. An Applied Case: Restoring Images in an Industrial Inspection Line

Background

Consider an illustrative manufacturing line that uses a camera to inspect painted metal components for scratches, incomplete coating, and small dents. Images are captured while parts move on a conveyor. The system must classify defects and localize them for human review.

Situation and Problem

Initial images show uneven illumination, motion blur in some frames, sensor noise in dark regions, and occasional isolated bright pixels. A simple global threshold produces many false defects, while aggressive sharpening creates artificial edges around normal paint texture.

Analysis

The defects are small and edge-based, so preserving boundaries is important. Motion blur is directional, whereas isolated bright pixels are consistent with impulse noise. Illumination varies across the field because of reflections from the metal surface. The team therefore separates the problems instead of applying one universal filter.

Approach

A flat-field correction is first estimated from reference images to reduce systematic illumination variation. A small median filter removes impulse noise. An edge-aware smoothing operation reduces sensor noise while preserving defect boundaries. For frames with measurable conveyor motion, a calibrated directional blur model is used in a regularized deconvolution step; frames with severe blur are rejected rather than aggressively reconstructed. Local contrast enhancement is applied with conservative CLAHE settings. Morphological opening removes isolated threshold components, while closing fills small breaks in candidate defect regions. Images and labels are resized using different interpolation rules: bicubic for intensity images and nearest-neighbour for masks.

Performance is evaluated at three levels: visual artefact inspection, image-quality measures on a controlled reference set, and defect-level precision, recall, and false-alarm rate on held-out production data. The preprocessing parameters are stored with the model version, and raw images remain available for audit.

Outcome

The workflow reduces false alarms caused by illumination and impulse noise without treating every blur case as recoverable. Its most important improvement is not a single filter but the explicit separation of acquisition correction, denoising, restoration, segmentation preparation, and task evaluation.

Learning Points

The case demonstrates that preprocessing should reflect the physical source of degradation. It also shows why a visually sharper image can be operationally worse, why rejection may be preferable to unstable restoration, and why evaluation must include the downstream inspection objective.

10. Challenges, Professional Responsibilities, and Emerging Practice

Common mistakes include applying histogram equalization without checking noise, using a median filter so large that defects disappear, sharpening before denoising, deconvolving with an inaccurate PSF, resizing labels with bilinear interpolation, and evaluating only on the same images used to tune parameters. Data leakage is another serious risk: if preprocessing uses statistics calculated from the entire dataset, information from the test set may influence training decisions.

In medical, forensic, scientific, and safety-critical applications, enhancement must not be presented as evidence of information that was actually measured. A restored image is an estimate, not an unquestionable recovery. Systems should preserve originals, document transformations, expose uncertainty where feasible, and maintain access controls for sensitive imagery. Privacy-preserving processing may include removal of identifying regions, but anonymization must be validated because identity can persist in backgrounds, metadata, or biometric structure.

Contemporary systems increasingly combine classical processing with learned models. Differentiable geometric transformations support spatial transformer networks; neural denoisers learn image priors; diffusion and generative approaches can perform powerful restoration. Their strength creates a corresponding risk: generative restoration may produce plausible but unsupported details. For high-consequence decisions, physically informed models, uncertainty estimation, provenance, and human review remain essential.

Best practice is to begin with the imaging objective, characterize degradation, establish a baseline, change one processing component at a time, and evaluate both image fidelity and downstream task performance. Parameters should be selected on validation data, robustness should be tested across devices and conditions, and deployment monitoring should detect shifts in noise, illumination, compression, and camera geometry.

Chapter Summary

Image enhancement improves usefulness or appearance, restoration estimates an image degraded by a known or assumed process, and transformation changes its representation or geometry. Spatial methods include point operations, histogram processing, neighbourhood smoothing, edge-aware filtering, and sharpening. Frequency-domain methods provide a useful interpretation of low- and high-frequency content and support periodic-noise suppression and large-scale filtering. Denoising and deblurring are model-dependent inverse problems whose success depends on the correctness of their assumptions. Morphological operations use structuring elements to manipulate shape. Interpolation and geometric transformations enable resizing, registration, and perspective correction but can introduce aliasing and label errors. Compression must be matched to the value of preserved information. Finally, PSNR, SSIM, perceptual judgement, and downstream task metrics answer different quality questions. A reliable workflow is physically informed, documented, task-evaluated, and ethically governed.

Key Terms

Enhancement: Processing intended to improve image appearance or usefulness for a specified task.

Restoration: Estimation of an original image from a model of degradation.

Transformation: A change in image coordinates, representation, scale, or domain.

Point-spread function: The response of an imaging system to an ideal point source.

Convolution: An operation combining an image with a kernel to produce a filtered output.

Histogram: A frequency distribution of image intensity values.

CLAHE: Local histogram equalization with a limit on contrast amplification.

Low-pass filter: A filter that preserves slowly varying content and attenuates high frequencies.

High-pass filter: A filter that emphasizes rapid spatial changes.

Impulse noise: Noise appearing as isolated extreme bright or dark pixels.

Bilateral filter: An edge-aware filter that considers spatial and intensity similarity.

Deblurring: Estimation of a sharper image from a blurred observation.

Wiener filter: A restoration filter balancing inverse correction and noise amplification.

Morphology: Shape-based processing using a structuring element.

Opening: Erosion followed by dilation, often used to remove small foreground objects.

Closing: Dilation followed by erosion, often used to fill gaps or small holes.

Interpolation: Estimation of pixel values at non-integer coordinates.

Homography: A projective mapping between planar image views.

PSNR: A logarithmic measure derived from pixel-wise error relative to signal range.

SSIM: A quality measure comparing local luminance, contrast, and structural similarity.

Review Questions

Short Answer Questions

1. Distinguish image enhancement from image restoration.

2. Why can a median filter be effective against salt-and-pepper noise?

3. What does a point-spread function represent?

4. State one appropriate use of morphological closing.

5. Why is nearest-neighbour interpolation preferred for segmentation masks?

Long Answer Questions

1. Explain spatial-domain smoothing and sharpening methods, including their advantages and artefacts.

2. Compare ideal, Butterworth, and Gaussian frequency-domain filters.

3. Discuss the assumptions, strengths, and limitations of Wiener filtering and blind deconvolution.

4. Explain histogram equalization, CLAHE, and histogram matching in the context of uneven illumination.

5. Describe a complete preprocessing and evaluation workflow for an industrial vision system.

Analytical / Application Questions

1. An image contains periodic horizontal interference and moderate Gaussian noise. Propose a processing sequence and justify the domain and filters selected.

2. A medical image becomes visually sharper after a learned restoration model, but a specialist identifies structures absent from the raw image. How should the system be evaluated and governed?

3. A segmentation model performs well on PNG images but poorly on heavily compressed images from a new camera. Analyze possible causes and propose corrective experiments.

22. Multiple Choice Questions with Answers

Question 1

Which operation is most specifically a restoration task rather than a general enhancement task?

A. Gamma correction to brighten dark regions

B. Histogram equalization to increase contrast

C. Regularized deconvolution using an estimated blur model

D. Colour mapping for visual display

Correct Answer: C. Regularized deconvolution using an estimated blur model

Question 2

A neighbourhood filter is required to remove isolated black and white pixels while preserving object boundaries. Which method is most appropriate?

A. Mean filtering

B. Median filtering

C. Ideal low-pass filtering

D. Laplacian sharpening

Correct Answer: B. Median filtering

Question 3

Why can direct inverse filtering amplify noise severely?

A. It removes all low frequencies

B. It divides by small values of the blur transfer function

C. It replaces every pixel with a neighbourhood median

D. It changes an affine transform into a homography

Correct Answer: B. It divides by small values of the blur transfer function

Question 4

Which morphological sequence is generally used to remove small foreground objects?

A. Dilation followed by erosion

B. Erosion followed by dilation

C. Two successive dilations

D. High-pass filtering followed by thresholding

Correct Answer: B. Erosion followed by dilation

Question 5

Which interpolation method is safest for a discrete segmentation-label image?

A. Bilinear interpolation

B. Bicubic interpolation

C. Nearest-neighbour interpolation

D. Lanczos interpolation

Correct Answer: C. Nearest-neighbour interpolation

Question 6

What is a principal advantage of CLAHE over global histogram equalization?

A. It eliminates the need for sampling

B. It adapts contrast enhancement to local image regions

C. It always removes motion blur

D. It guarantees higher PSNR

Correct Answer: B. It adapts contrast enhancement to local image regions

Question 7

In frequency-domain analysis, periodic interference is often identified by:

A. A concentrated pattern of peaks away from the frequency origin

B. A uniform reduction in all frequencies

C. A change in the image colour space only

D. The absence of any high-frequency content

Correct Answer: A. A concentrated pattern of peaks away from the frequency origin

Question 8

Before downsampling an image, low-pass filtering is commonly used primarily to reduce:

A. Aliasing

B. Histogram clipping

C. Projective distortion

D. Impulse response length

Correct Answer: A. Aliasing

Question 9

Which statement best describes PSNR?

A. It measures only object-recognition accuracy

B. It is derived from pixel-wise error and signal range

C. It directly estimates the physical blur kernel

D. It is always more perceptually valid than SSIM

Correct Answer: B. It is derived from pixel-wise error and signal range

Question 10

In the industrial inspection case, why might severely blurred frames be rejected rather than aggressively deblurred?

A. Deblurring cannot be applied to grayscale images

B. An inaccurate model can create unsupported edges and false defects

C. Rejected frames automatically improve histogram equalization

D. Morphological operations require blurred input

Correct Answer: B. An inaccurate model can create unsupported edges and false defects