7. Mathematical Morphology

Introduction and Context

Mathematical morphology is a theory and set of techniques for analysing the geometric structure of objects in images. Unlike methods that primarily model image intensity as a numerical signal, morphology studies shape, connectivity, boundaries, regions, and spatial relationships. Its operations are particularly effective when the objects of interest can be described by their interaction with a small geometric pattern called a structuring element.

Morphological processing originated in set theory and was first developed for binary images. It has since been extended to grayscale images, colour images, three-dimensional data, video, and graph-based representations. In image-processing systems, morphology is commonly used after segmentation to remove noise, repair breaks, separate objects, fill gaps, extract boundaries, and identify topological structures. It is also used before segmentation to simplify an image or enhance features of a particular size and shape.

The central idea is simple but powerful: an image is probed locally by translating a structuring element across it. The result depends on whether the structuring element fits inside, intersects, or is contained by the image region. This geometric interpretation makes morphology highly intuitive, while its algebraic formulation permits rigorous analysis and efficient implementation.

Learning Objectives

After completing this chapter, the learner should be able to:

1. Explain the set-theoretic basis of binary mathematical morphology.

2. Select suitable structuring elements for different image-processing tasks.

3. Derive and interpret dilation, erosion, opening, and closing.

4. Apply hit-or-miss transforms and connected-component methods.

5. Explain skeletonization, thinning, distance transforms, and watershed segmentation.

6. Extend morphological reasoning to grayscale images.

7. Evaluate the advantages, limitations, and risks of morphological processing.

8. Design a morphology-based workflow for a practical image-analysis problem.

The Morphological View of an Image

In a binary image, the foreground is represented as a set A of foreground pixel coordinates, while the background is its complement. A structuring element B is another small set of coordinates, usually centred at an origin. Common examples include a square, disk, diamond, horizontal line, or vertical line. The shape, size, and origin of B determine which spatial features an operation will preserve or remove.

If B is translated so that its origin lies at point z, the translated element is denoted Bz. Morphological operations examine the relationship between Bz and A. This produces a local geometric test rather than a conventional arithmetic filter. A square element is sensitive to block-like structures, a disk is relatively isotropic, and a line element responds strongly to elongated features in a particular orientation.

The choice of origin matters for asymmetric structuring elements. Symmetric elements are often centred, whereas directional elements may be anchored at an endpoint or another application-specific location. In practical systems, boundary handling must also be defined: pixels outside the image may be treated as background, reflected, replicated, or ignored.

The Four Fundamental Operations

Dilation

The dilation of A by B is defined as

A ⊕ B = {z | (B̂)z ∩ A ≠ ∅},

where B̂ is the reflection of B about its origin. For symmetric B, reflection makes no difference. Intuitively, dilation places the structuring element at every foreground pixel and takes the union of all resulting copies. It expands foreground regions, bridges small gaps, connects nearby components, and fills narrow indentations.

For a binary image, dilation generally increases the area of foreground objects. A disk-shaped structuring element produces approximately isotropic expansion; a horizontal line expands more strongly in the horizontal direction. Dilation is not merely a smoothing operation: it changes object size and may merge objects that are closer than the diameter or effective span of the structuring element.

Erosion

The erosion of A by B is defined as

A ⊖ B = {z | Bz ⊆ A}.

A point remains in the eroded image only when the translated structuring element fits entirely inside A. Erosion therefore removes boundary pixels, shrinks foreground regions, eliminates components smaller than the structuring element, and can break narrow connections.

Erosion and dilation are dual under complementation. In particular,

(A ⊖ B)c = Ac ⊕ B̂.

This duality is useful both theoretically and computationally. It also explains why dilation of the foreground corresponds to erosion of the background.

Opening

Opening is erosion followed by dilation using the same structuring element:

A ○ B = (A ⊖ B) ⊕ B.

Opening removes foreground structures that cannot contain the structuring element while preserving larger structures, often with their general position and shape retained. It is useful for removing small isolated objects, eliminating thin protrusions, separating weakly connected regions, and smoothing contours from the inside.

Opening is anti-extensive, meaning A ○ B is contained in A. It is also increasing and idempotent: repeating the same opening does not produce further change after the first application. These properties distinguish opening from arbitrary combinations of erosion and dilation.

Closing

Closing is dilation followed by erosion:

A • B = (A ⊕ B) ⊖ B.

Closing fills small holes and narrow gaps, joins nearby boundary fragments, and smooths contours from the outside. It is extensive, so A is contained in A • B. Like opening, it is increasing and idempotent.

Opening is generally associated with removing small bright objects in a binary foreground interpretation, whereas closing is associated with filling small dark interruptions. The terms bright and dark require care in grayscale images, where foreground and background are defined relative to intensity.

Morphological Properties and Interpretation

Several algebraic properties provide a useful framework for predicting results:

• Increasingness: if A is contained in C, then A ○ B is contained in C ○ B, and similarly for dilation, erosion, and closing under suitable formulations.
• Translation invariance: translating the input translates the output by the same amount.
• Idempotence: opening and closing applied twice with the same element equal one application.
• Duality: erosion and dilation are related through complementation and reflection.
• Scale dependence: increasing the structuring-element size causes stronger geometric modification.

These properties are important in algorithm design. For example, idempotence means that repeated opening is not a substitute for selecting an appropriate structuring-element size. If the first opening removes a feature, repeated application cannot restore it; if it preserves a feature, repetition with the same element will not normally change it further.

Shape-Based Transforms

Hit-or-Miss Transform

The hit-or-miss transform detects a specified configuration of foreground and background pixels. It uses a pair of structuring elements, B1 for the foreground pattern and B2 for the background pattern:

A * B = (A ⊖ B1) ∩ (Ac ⊖ B2).

A location is retained only when B1 fits inside the foreground and B2 fits inside the background. The transform is therefore more selective than erosion, which tests only foreground support. It can detect endpoints, isolated pixels, corners, junctions, or a prescribed local pattern.

The hit-or-miss transform is sensitive to rotation, scale, and noise. To detect a feature in several orientations, rotated versions of the structuring-element pair may be applied and combined. In practice, exact pattern matching may be softened by preprocessing or by using distance-based methods when images are imperfect.

Boundary Extraction

A simple internal boundary can be obtained by

β(A) = A − (A ⊖ B).

This subtracts the eroded object from the original and retains pixels near the inner boundary. External boundaries can be obtained using dilation, while morphological gradients often use the difference between dilation and erosion:

Gm = (A ⊕ B) − (A ⊖ B).

Morphological gradients are useful for emphasizing object transitions, although they are affected by the size and shape of B.

Morphological Reconstruction

Morphological reconstruction uses a marker image and a mask image. Starting from the marker, geodesic dilations are repeatedly applied while restricting the result to the mask, until stability is reached. Reconstruction is valuable when one wants to remove selected structures without unnecessarily distorting the remaining objects.

For example, opening by reconstruction can remove small components while preserving the contours of surviving objects better than ordinary opening. Reconstruction is also used in hole filling, connected-component extraction, regional maxima detection, and marker preparation for watershed segmentation.

Connected Components and Geometric Measurement

A connected component is a maximal set of foreground pixels linked according to a chosen connectivity rule. In two-dimensional images, 4-connectivity links horizontal and vertical neighbours, whereas 8-connectivity also links diagonal neighbours. The choice must be made consistently for both foreground and background; careless mixing can produce topological ambiguities.

Connected-component labelling assigns a distinct label to each component. A typical two-pass algorithm scans the image, assigns provisional labels, records equivalences among neighbouring labels, and resolves them in a second pass. Modern implementations may use union–find structures or parallel algorithms.

After labelling, region properties can be calculated, including area, perimeter, centroid, bounding box, orientation, eccentricity, and moments. Morphological filtering and connected-component analysis are often combined. For instance, all components whose area is below a threshold can be removed, while components satisfying a shape or location criterion can be retained.

Connectivity is not merely an implementation detail. It influences whether diagonal objects are considered joined, how holes are counted, and how skeleton branches are interpreted. In medical, industrial, and remote-sensing applications, an inappropriate connectivity convention can produce systematic measurement errors.

Skeletons, Thinning, and Medial Structure

The skeleton of a region is a thin representation that preserves its topology and conveys its general shape. One classical definition is the union of skeletal subsets obtained from successive erosions and openings:

Sk(A) = ⋃k [(A ⊖ kB) − ((A ⊖ kB) ○ B)].

The skeleton can reconstruct the original object when combined with suitable dilations of its skeletal subsets. It is closely related to the medial axis, which consists of points having more than one closest boundary point. The distance transform, assigning each foreground pixel its distance from the background, provides a useful route to medial-axis extraction.

Thinning removes boundary pixels iteratively while preserving connectivity and selected end points. Algorithms such as Zhang–Suen and Guo–Hall are widely used for binary skeletonization. Pruning is often necessary because small boundary irregularities create spurious branches. Skeletons are useful for character recognition, road-network analysis, vascular modelling, handwriting analysis, and shape matching, but they can be unstable under noise and small contour changes.

A skeleton is not automatically a complete description of shape. It may lose thickness information, and different objects can have similar skeletons. Therefore, practical systems often retain the distance-transform values along the skeleton or combine skeleton features with region measurements.

Watershed Segmentation

The watershed method treats a grayscale image as a topographic surface. Low-intensity regions are valleys, high-intensity regions are hills, and water introduced into the valleys forms catchment basins. As water levels rise, dams are built where basins would merge. These dams define watershed lines and provide segmentation boundaries.

A direct watershed transform frequently produces over-segmentation because every local minimum may generate a basin. Marker-controlled watershed addresses this problem by specifying reliable foreground markers and background markers. Morphological operations, distance transforms, connected components, and regional minima are commonly used to construct these markers.

For separating touching objects, a typical workflow is: threshold the image, clean the binary mask, compute a distance transform, identify maxima as object markers, construct a complementary background marker, and apply marker-controlled watershed to the gradient or distance surface. The quality of the result depends strongly on marker selection. Poor markers can either merge distinct objects or create unnecessary regions.

Watershed is especially useful when boundaries are weak but object centres are distinguishable. It is less reliable when the image contains severe noise, uneven illumination, or objects with highly irregular intensity patterns unless appropriate preprocessing and normalization are performed.

Grayscale Morphology

For a grayscale image f, dilation and erosion can be interpreted using local maximum and minimum operations. With a flat structuring element B,

(f ⊕ B)(x) = max{f(x − b) : b ∈ B},

(f ⊖ B)(x) = min{f(x + b) : b ∈ B}.

Thus, grayscale dilation brightens structures according to local maxima, while grayscale erosion darkens structures according to local minima. Non-flat structuring elements include a height function and can model both spatial position and intensity offset.

Grayscale opening suppresses bright details smaller than the structuring element, and grayscale closing suppresses dark details. Their difference produces useful enhancement operators. The top-hat transform is

Twhite = f − (f ○ B),

which extracts small bright features. The black-hat transform is

Tblack = (f • B) − f,

which extracts small dark features. These transforms are widely used for background correction, text enhancement, defect detection, and illumination normalization.

Morphological filters are nonlinear. They do not obey the linear superposition principle, and their effects cannot generally be predicted by frequency response alone. Their advantage is that the filtering behaviour can be expressed in terms of object size and shape.

A Practical Design Workflow

A robust morphology-based solution normally follows these stages:

1. Define the target structure and distinguish it from irrelevant texture or noise.

2. Determine whether the image should be treated as binary, grayscale, or both.

3. Estimate the characteristic scale and orientation of target objects.

4. Select a structuring element whose geometry reflects that information.

5. Apply the least destructive operation that solves the problem.

6. Validate connectivity, object counts, area measurements, and boundary accuracy.

7. Inspect failure cases and test sensitivity to scale, noise, threshold, and border policy.

The phrase “least destructive” is important. A large opening may remove noise but also erase legitimate small objects. A strong closing may connect objects that should remain separate. Morphology should therefore be treated as a model-based operation rather than a collection of interchangeable filters.

Illustrative Case Study: Separating Touching Cells

Background

Consider an illustrative microscopy system that must estimate the number and size distribution of approximately circular cells. Cells vary in brightness, and some touch one another. The goal is to produce one label per cell without counting a cluster as a single object.

Situation and Problem

Global thresholding produces a binary foreground mask, but the mask contains small holes, isolated noise, and touching cells. Direct connected-component labelling underestimates the cell count because adjacent cells form merged components.

Analysis

A disk-shaped structuring element is appropriate because the cells are approximately isotropic. A small opening can remove isolated bright noise, while a closing can repair narrow gaps in individual cell regions. However, excessive closing would further merge neighbouring cells. The distance transform of the cleaned foreground mask should exhibit local maxima near the centres of individual cells.

Approach

First, illumination is normalized using a grayscale opening or a background-estimation method. The image is thresholded to form a binary mask. Small components are removed by area filtering, and minor holes are repaired by controlled closing or reconstruction. A distance transform is computed on the foreground mask. Regional maxima of the distance map serve as internal markers. The cleaned foreground acts as a constraint, and a marker-controlled watershed is applied to a gradient or inverted distance surface.

Each watershed region is then labelled and measured. Very small regions caused by noise are rejected, while implausibly large regions are inspected as possible under-segmentation. The structuring-element radius, threshold, and marker-minimum distance are validated against manually annotated images.

Outcome

The workflow can separate many touching cells while preserving isolated cells as single regions. Its success depends less on any one operator than on the interaction between cleaning, marker generation, and watershed control. If cells have highly variable sizes, a single fixed structuring element may be inadequate; multiscale processing or adaptive markers may be required.

Learning Points

The case illustrates four general principles: morphology should follow object geometry; preprocessing must not destroy the evidence required for segmentation; watershed requires meaningful markers; and quantitative validation is essential because visually plausible boundaries may still yield inaccurate counts.

Applications in Professional Practice

Morphology is used in industrial inspection to detect scratches, pinholes, cracks, missing components, and surface defects. In document analysis, it enhances printed characters, removes speckle, joins broken strokes, and separates text lines. In medical imaging, it supports vessel extraction, cell counting, gland segmentation, and detection of anatomical structures. Remote-sensing systems use morphological profiles to characterize buildings, roads, vegetation, and terrain features across multiple scales.

In automated production, morphology is attractive because many operations are computationally efficient and interpretable. Its parameters can often be linked to physical dimensions, such as the minimum defect size or expected gap width. This interpretability is useful for quality assurance and auditability, although it does not remove the need for calibration and validation.

Challenges, Limitations, and Common Errors

The most common error is selecting a structuring element without reference to object scale or orientation. A disk is not always preferable simply because it is visually natural; elongated defects may require line elements, and anisotropic structures may require directional processing.

Thresholding errors propagate into all subsequent binary morphology. If an object is missing from the mask, closing may not reconstruct it correctly. If unrelated regions are included, opening may not distinguish them from legitimate small objects. Boundary effects can also create artificial erosion or dilation near the image edge.

Skeletons are sensitive to noise, watershed is vulnerable to over-segmentation, and connected-component measurements depend on connectivity conventions. Repeated use of morphology can introduce bias in object area, perimeter, and count. Computational cost may become substantial for large three-dimensional data, although optimized libraries, separable elements, distance transforms, and GPU implementations reduce this burden.

Morphological operations should not be evaluated only by visual appearance. Suitable metrics include object-count accuracy, intersection over union, boundary deviation, precision and recall of detected defects, and errors in downstream measurements.

Ethical and Professional Considerations

When morphology is used in medical or biometric systems, errors may affect diagnosis, treatment, or access decisions. Parameters should therefore be documented, datasets should represent relevant variation, and results should be reviewed by domain experts. A visually clean segmentation is not evidence of clinical validity.

Industrial systems must distinguish genuine defects from harmless variation without silently changing acceptance criteria. Audit trails should record image conditions, preprocessing, structuring elements, thresholds, and software versions. In research, examples should not be selected only because they produce favourable results; failure cases and sensitivity analyses are part of responsible reporting.

Contemporary Developments

Modern image-analysis systems increasingly combine morphology with machine learning. Neural networks may produce probability maps, while morphological reconstruction, connected components, or watershed impose geometric constraints and convert predictions into usable objects. Differentiable approximations of dilation and erosion are also studied for integrating morphological concepts into trainable networks.

Multiscale morphology, attribute filtering, graph morphology, and component trees extend the basic theory beyond fixed local structuring elements. Three-dimensional morphology is important in volumetric medical imaging and materials science. Nevertheless, learned methods and morphological methods are complementary: morphology supplies geometric priors and interpretability, while learning can model variation that fixed operators cannot capture.

Best-Practice Guidelines

Use a structuring element that reflects the expected geometry and physical scale of the target. Begin with the smallest operation likely to solve the problem. Keep thresholding, connectivity, border treatment, and parameter choices explicit. Use reconstruction when contour preservation matters. Generate watershed markers conservatively and validate them independently. Preserve intermediate images for debugging and auditing. Finally, evaluate the complete workflow on representative normal cases, difficult cases, and known failures.

Chapter Summary

Mathematical morphology analyses image structure through set operations and structuring elements. Dilation expands regions and joins nearby structures; erosion shrinks regions and removes structures that cannot contain the structuring element. Opening removes small or protruding foreground structures, while closing fills small gaps and holes. Hit-or-miss transforms detect specified foreground–background configurations. Connected-component labelling supports object identification and measurement, while skeletonization and thinning provide compact topological representations. Watershed methods interpret images as topographic surfaces and, when controlled by reliable markers, can separate touching objects. Grayscale morphology uses local maxima and minima and supports top-hat and black-hat enhancement. Effective applications depend on geometry, scale, connectivity, validation, and careful control of limitations.

Key Terms

Structuring element: A small geometric pattern used to probe an image during a morphological operation.

Binary morphology: Morphological processing in which an image is represented as foreground and background sets.

Dilation: An operation that expands foreground regions according to the structuring element.

Erosion: An operation that retains locations where the structuring element fits entirely within the foreground.

Opening: Erosion followed by dilation, commonly used to remove small foreground structures.

Closing: Dilation followed by erosion, commonly used to fill small gaps and holes.

Hit-or-miss transform: A transform that detects simultaneous foreground and background configurations.

Connectivity: The rule defining which neighbouring pixels belong to the same component.

Connected component: A maximal connected foreground region.

Morphological reconstruction: A constrained iterative process driven by a marker and limited by a mask.

Skeleton: A thin, topology-preserving representation of a region.

Thinning: Iterative removal of boundary pixels while preserving selected structural properties.

Distance transform: An image assigning each pixel its distance from a specified background or boundary.

Watershed: A segmentation method based on catchment basins and dividing lines in a topographic surface.

Marker-controlled watershed: Watershed segmentation constrained by predefined foreground and background markers.

Top-hat transform: The difference between an image and its morphological opening, used to extract bright details.

Black-hat transform: The difference between morphological closing and the image, used to extract dark details.

Morphological gradient: A boundary-enhancement measure based on dilation and erosion.

Attribute filtering: Morphological filtering that retains or removes connected structures according to attributes such as area or shape.

Review Questions

Short Answer Questions

1. What is the role of a structuring element in mathematical morphology?

2. State the difference between dilation and erosion.

3. Why is opening idempotent?

4. What is the purpose of a distance transform in watershed segmentation?

5. Distinguish 4-connectivity from 8-connectivity.

Long Answer Questions

1. Derive and explain binary dilation, erosion, opening, and closing, including their geometric effects.

2. Explain the hit-or-miss transform and discuss how it can be used to detect endpoints or junctions.

3. Describe skeletonization and thinning, including their uses and sensitivity to noise.

4. Explain grayscale morphology and the construction of white top-hat and black-hat transforms.

5. Describe marker-controlled watershed segmentation and explain why uncontrolled watershed often over-segments an image.

Analytical / Application Questions

1. An industrial image contains long, thin cracks and isolated bright specks. Design a morphology-based workflow and justify the shape and orientation of each structuring element.

2. Two approximately circular objects touch after thresholding. Explain how distance transforms and marker-controlled watershed could separate them, and identify two possible failure modes.

3. A document image has broken character strokes, small dark noise, and uneven illumination. Propose a sequence of grayscale and binary morphological operations, explaining the likely effect of each.

22. Multiple Choice Questions with Answers

Question 1

Which condition must hold for a point z to belong to the erosion A ⊖ B?

A. The translated structuring element intersects A at least once

B. The translated structuring element is entirely contained in A

C. The translated structuring element is entirely outside A

D. The boundary of B coincides with the boundary of A

Correct Answer: B. The translated structuring element is entirely contained in A

Question 2

Which operation is most appropriate for removing small foreground objects while preserving larger objects of similar shape?

A. Opening

B. Closing

C. Dilation only

D. Watershed without markers

Correct Answer: A. Opening

Question 3

The hit-or-miss transform differs from ordinary erosion because it:

A. Uses only grayscale intensity values

B. Tests both a foreground configuration and a background configuration

C. Always expands the detected object

D. Requires no structuring element

Correct Answer: B. Tests both a foreground configuration and a background configuration

Question 4

Why can direct watershed segmentation produce over-segmentation?

A. It ignores all local minima

B. Every relevant image region has exactly one basin

C. Noise and small local minima may create many catchment basins

D. It performs only binary erosion

Correct Answer: C. Noise and small local minima may create many catchment basins

Question 5

Which transform is designed to extract small bright details from a grayscale image?

A. Black-hat transform

B. White top-hat transform

C. Distance transform

D. Hit-or-miss transform

Correct Answer: B. White top-hat transform

Question 6

A horizontal line structuring element is most suitable when the target feature is:

A. A circular isolated object

B. A horizontally elongated structure

C. A random intensity fluctuation

D. A three-dimensional volume only

Correct Answer: B. A horizontally elongated structure

Question 7

In a cell-separation workflow, local maxima of the distance transform are commonly used to:

A. Estimate internal markers near object centres

B. Replace the binary mask with random noise

C. Force all touching objects to remain connected

D. Measure image file size

Correct Answer: A. Estimate internal markers near object centres

Question 8

Which statement about morphological opening is correct?

A. It is generally extensive and always enlarges the foreground

B. It is erosion followed by dilation and is idempotent

C. It detects only background patterns

D. It is equivalent to a linear convolution

Correct Answer: B. It is erosion followed by dilation and is idempotent

Question 9

Why must connectivity be selected carefully in connected-component analysis?

A. It determines whether diagonal pixels are considered connected and affects topology

B. It changes the image from grayscale to colour

C. It eliminates the need for thresholding

D. It determines the camera exposure

Correct Answer: A. It determines whether diagonal pixels are considered connected and affects topology

Question 10

A skeleton may contain many unwanted short branches primarily because:

A. Skeletonization is completely insensitive to boundary variation

B. Small contour irregularities and noise generate spurious structural branches

C. Dilation always removes all endpoints

D. The distance transform has no relation to shape

Correct Answer: B. Small contour irregularities and noise generate spurious structural branches