Feature Extraction, Shape Analysis, and Measurement

Introduction

Machine vision systems rarely make decisions directly from raw pixels. A camera produces an image, but inspection, guidance, identification, and measurement require a more compact representation: edges, contours, regions, geometric primitives, textures, colors, or learned descriptors. Feature extraction converts image data into measurable quantities. Shape analysis interprets the geometry and topology of objects. Metrology uses calibrated image measurements to estimate physical dimensions, usually with an associated uncertainty.

In mechatronics, these operations connect optical sensing with mechanical action. A vision system may locate a component for robotic grasping, reject a casting with an oversized hole, verify the orientation of a connector, or estimate the position of a moving part. The reliability of the final decision depends not only on the feature algorithm but also on illumination, lens distortion, calibration, segmentation, sampling, and uncertainty analysis.

Learning Objectives

By the end of this chapter, a student should be able to:

1. Explain the purpose and assumptions of feature extraction.
2. Extract and interpret contours, corners, blobs, color, texture, and geometric moments.
3. Perform connected-component analysis and calculate region properties.
4. Compare local and global shape descriptors and template-matching methods.
5. Register images or coordinate systems for inspection and alignment.
6. Convert pixel measurements into physical dimensions using calibration.
7. Estimate measurement uncertainty and distinguish accuracy from repeatability.
8. Design a practical vision-metrology workflow and identify its limitations.

1. From Pixels to Features

A digital image is a sampled function. For a grayscale image, the intensity at pixel coordinates (x, y) is written as I(x,y). A feature is a numerical or structural property that is useful for describing an object or making a decision. Features may be:

• Point features, such as corners or keypoints;
• Edge and contour features, such as boundary location and curvature;
• Region features, such as area, centroid, and eccentricity;
• Appearance features, such as color and texture;
• Relational features, such as distances, angles, and alignment errors.

Feature extraction should be designed around the task. A blob area may be adequate for counting pills, while a connector inspection may require pin positions, hole diameters, and orientation. A feature is useful only when it is sufficiently discriminative, repeatable, computationally practical, and robust to expected variation.

A typical pipeline is:

1. Acquire and radiometrically or geometrically correct the image.
2. Segment the object from its background.
3. Clean the segmentation using morphological operations where justified.
4. Extract contours or connected regions.
5. Compute descriptors and measurements.
6. Compare them with specifications, references, or a trained model.
7. Report a decision together with confidence or uncertainty.

Errors early in this chain propagate forward. For example, an incorrectly segmented boundary changes area, perimeter, centroid, moments, and every measurement based on them.

2. Contours, Edges, and Geometric Primitives

An edge is a location of rapid intensity change. A contour is an ordered representation of an object boundary, often obtained after segmentation. Edge detection is useful when boundaries are visible but region intensity is not uniform. Common gradient operators include Sobel and Scharr filters. If the image is I(x,y), the gradient is

where I_x and I_y are horizontal and vertical intensity derivatives. Its magnitude and orientation are

The magnitude G is expressed in intensity units per pixel, and θ is the direction of greatest increase. Derivative estimates are sensitive to noise, so smoothing is normally applied first. Excessive smoothing, however, displaces or removes narrow features.

The Canny method is a widely used multi-stage edge detector involving smoothing, gradient estimation, non-maximum suppression, double thresholding, and edge tracking by hysteresis. Its two thresholds distinguish strong edges from weak edges connected to strong ones. Threshold selection is application-dependent; it should be validated under the full range of lighting and object variation.

A contour can be represented as a sequence of boundary points (x_i,y_i). Its approximate perimeter is

For a closed contour, the final point is connected to the first. This estimate depends on pixel resolution and contour approximation. A digitized diagonal boundary tends to have a biased perimeter unless an appropriate geometric or subpixel method is used.

Contours can be simplified using polygonal approximation. A tolerance specifies the maximum permitted deviation between the original contour and its approximation. Simplification is useful for recognizing rectangles, triangles, and other manufactured shapes, but an excessive tolerance can eliminate defects.

Geometric primitives are often fitted to contours:

• Lines by least squares or robust estimators;
• Circles by algebraic or geometric fitting;
• Ellipses by constrained least squares;
• Planes and cylinders in three-dimensional vision.

Ordinary least squares is efficient but sensitive to outliers. RANSAC repeatedly fits a model to minimal subsets and selects the model supported by the largest consensus set. It is useful when a contour contains occlusions, burrs, or unrelated edges.

3. Corners and Local Keypoints

A corner is a location where image intensity changes significantly in more than one direction. Edges are locally ambiguous along their length, whereas corners are usually more distinctive for localization and matching.

The Harris detector forms a local second-moment matrix, often called the structure tensor:

Here W is a neighborhood, w is a weighting function, and the derivatives are evaluated within the window. The Harris response is

where k is an empirical constant, commonly selected in the approximate range 0.04–0.06. Large positive R indicates a corner, a strongly negative value indicates an edge, and a value near zero indicates a relatively uniform region. The method assumes that the neighborhood is small enough to represent a local structure and that the intensity pattern is reasonably stable.

Other detectors include FAST, Shi–Tomasi, and scale-space methods such as SIFT-like descriptors. A detector identifies keypoints; a descriptor describes the local image around them. Binary descriptors such as ORB are computationally efficient, while gradient-histogram descriptors are often more robust to moderate illumination changes. Keypoint methods are especially useful for registration and object recognition, but they may fail on textureless or repetitive surfaces.

4. Blobs and Connected Components

A blob is a connected image region whose pixels share a property, such as intensity, color, or texture. After thresholding a grayscale image at T,

The threshold may be global, adaptive, or obtained from a histogram-based method. The notation defines a binary image B; it does not imply that higher intensity always corresponds to the object. The inequality must be reversed when the object is darker than the background.

Connected-component labeling assigns a unique label to each connected foreground region. Under 4-connectivity, a pixel is connected to its horizontal and vertical neighbors. Under 8-connectivity, diagonal neighbors are also included. The choice affects whether diagonally touching regions are treated as one object.

For each component, useful properties include area, bounding box, centroid, perimeter, orientation, and equivalent diameter. If a binary component contains N pixels and the pixel dimensions are s_x and s_y, its area is

where A has units of square length. This formula assumes that pixels represent equal-area samples under a locally uniform scale. Perspective distortion or a nonuniform calibration invalidates a single global scale.

Morphological opening, erosion followed by dilation, removes small isolated objects. Closing, dilation followed by erosion, fills small gaps and connects nearby structures. These operations depend on a structuring element; they should not be used merely to make an image look better, because they alter geometry and may remove real defects.

5. Moments and Region Geometry

Image moments summarize the distribution of pixels. For a binary region, the raw moment of order p + q is

where p and q are nonnegative integers. For a grayscale region, B(x,y) may be replaced by intensity I(x,y), provided the interpretation is intensity-weighted rather than purely geometric.

The centroid is obtained from first-order moments:

For a binary image, m_{00} is the number of foreground pixels. Central moments remove dependence on translation:

Normalized central moments reduce dependence on scale. Combinations of normalized moments, such as Hu moments, are designed to be approximately invariant to translation, rotation, and scale. In practice they are affected by segmentation, quantization, occlusion, and noise; invariance is not perfect.

The second-order central moments describe the spread of a region. The covariance-like matrix is

The eigenvectors of C give principal directions, and the eigenvector associated with the larger eigenvalue gives the major axis. The orientation is commonly calculated as

The angle is measured relative to the x-axis. For nearly circular regions, orientation is poorly defined because the two eigenvalues are almost equal.

Useful shape descriptors include circularity and aspect ratio. Circularity is often defined as

For an ideal continuous circle, C_r = 1. Pixelation, perimeter estimation, and boundary roughness normally make the measured value lower. The descriptor is meaningful only when A and P are measured consistently and the object is not strongly fragmented.

6. Shape Descriptors and Matching

A descriptor should represent relevant shape information while suppressing irrelevant variation. Important classes include:

A normalized cross-correlation score between an image patch I and template T can be expressed as

where i indexes corresponding pixels and bars denote sample means. The score lies between −1 and 1 when the denominator is nonzero. A high score indicates similar zero-mean intensity patterns, but not necessarily correct physical identity. Changes in scale, rotation, occlusion, and nonuniform illumination can reduce the score.

Chamfer matching compares an edge template with a distance transform of the target edge image. It is often more tolerant of small edge displacement than raw pixel correlation. Feature-based matching detects correspondences between descriptors and estimates a transformation, usually with outlier rejection.

7. Color and Texture Features

Color features require a color model. RGB values are convenient for acquisition but mix brightness and chromatic information. HSV separates hue, saturation, and value more intuitively for many segmentation tasks. CIE Lab* is designed to be more perceptually uniform and is often useful for color difference measurements.

A color histogram counts pixels in quantized color bins. It is computationally simple but discards spatial arrangement: two objects with different patterns can have the same histogram. Color segmentation must account for illumination, camera white balance, exposure, reflective surfaces, and shadows. Controlled lighting is generally more reliable than attempting to correct all variation algorithmically.

Texture describes spatial variation in intensity or color. Common approaches include:

• Local Binary Patterns, which compare neighboring pixels with a central pixel;
• Gray-level co-occurrence matrices, which estimate joint gray-level statistics at a specified displacement and direction;
• Gabor filters, which respond to selected spatial frequencies and orientations;
• Local variance and gradient statistics.

A co-occurrence matrix depends on quantization, displacement, and direction. Consequently, texture features must be computed under a defined imaging protocol. Texture is useful for detecting surface finish, weave, corrosion, and contamination, but it can be confounded by lighting direction and focus.

8. Registration and Coordinate Systems

Registration aligns two images or coordinate systems so that corresponding points have consistent coordinates. It is needed for image comparison, defect detection, stereo integration, and robot guidance.

Transformation models should match the physical situation:

• Translation: x'=x+t_x, y'=y+t_y;
• Euclidean transformation: rotation plus translation;
• Similarity transformation: rotation, translation, and uniform scale;
• Affine transformation: also permits shear and nonuniform scale;
• Projective homography: models planar perspective changes.

For a planar object, a homography can be written in homogeneous coordinates as

where H is a 3 × 3 matrix defined up to a nonzero scale factor and s is a projective scale. At least four noncollinear point correspondences are required to estimate a general homography, although more points are preferred for least-squares fitting and outlier detection.

Registration quality should be checked using residual errors. If a point predicted by the transformation differs from its observed location, the residual vector is r_i = p_i^observed − p_i^predicted. Large or structured residuals indicate an unsuitable model, incorrect correspondences, lens distortion, or a nonplanar object.

9. Image Metrology and Calibration

Image metrology is the estimation of physical dimensions from images. The simplest case is a calibrated orthographic or telecentric setup with a constant scale k in millimetres per pixel:

where n is a pixel distance and L is the corresponding physical length. This model assumes that the object lies in the calibrated plane, perspective effects are negligible, and lens distortion has been corrected.

A more general pinhole model relates a 3D point to an image point through camera intrinsics and extrinsics. In homogeneous notation,

where P is a world point, p is an image point, K contains focal lengths and principal point, R and t describe camera pose, and s is a scale factor. A single image cannot generally determine absolute depth without additional information. For precision planar measurement, camera calibration followed by plane rectification is often sufficient.

Calibration methods use a target with known geometry, such as a checkerboard or dot grid. They estimate intrinsic parameters, distortion coefficients, and camera pose. Calibration must be performed at the working distance, focus, aperture, and temperature whenever these affect the measurement.

Worked Example: Area and Diameter

Suppose a segmented circular washer occupies 12,560 pixels. The calibrated pixel size is 0.050 mm × 0.050 mm. The area estimate is

If an equivalent circular diameter is desired,

This is an equivalent diameter, not necessarily the measured inner or outer diameter of a washer. It is valid only if the segmentation represents the intended region and the scale is uniform over that region.

For a ring, it is preferable to measure the inner and outer contours separately. If their fitted radii are r_o and r_i, the radial wall thickness is approximately t = r_o−r_i, provided both radii are expressed in the same calibrated coordinate system.

10. Uncertainty, Accuracy, and Decision Limits

A measurement is incomplete without an indication of its reliability. Accuracy describes closeness to a reference value; repeatability describes variation under the same conditions; reproducibility describes variation when conditions such as operator, camera, or setup change.

If a measured length is computed from a pixel distance and a scale factor, L = kn, and k and n are independent, first-order uncertainty propagation gives

where u_L, u_k, and u_n are standard uncertainties. If covariance is significant, the more general form includes 2n k Cov(k,n). The simplified expression assumes small uncertainties and approximate linearity.

Suppose n = 200.0 ± 0.5 pixels and k = 0.0500 ± 0.0005 mm/pixel, with independent standard uncertainties. Then L = 10.00 mm and

The result may be reported as approximately 10.00 ± 0.11 mm for a coverage factor of one standard uncertainty. A larger coverage factor may be selected when a specified confidence level is required, but the distributional assumptions must be stated.

Uncertainty sources include pixel quantization, edge localization, lens distortion, calibration-target error, focus, thermal drift, vibration, lighting variation, segmentation threshold, and model fitting. Repeatability should be measured experimentally by repeated acquisitions, not inferred only from software output.

For pass/fail inspection, avoid setting a limit exactly at the nominal specification when measurement uncertainty is appreciable. A guard band or decision rule should be defined with the quality system. Borderline parts may require reinspection or a higher-precision instrument.

11. Applied Case: Vision Inspection of a Machined Flange

This is an illustrative case.

Background

A production line manufactures circular flanges containing a central bore and four equally spaced bolt holes. The inspection must verify outer diameter, bore diameter, hole count, hole spacing, and angular orientation. Parts move on a conveyor and may be rotated arbitrarily.

Problem

A simple intensity threshold sometimes merges a hole with a dark fixture region. Reflections produce incomplete edges, and a pixel-to-millimetre conversion based on a ruler gives inconsistent results near the image boundary.

Analysis

The required measurements are geometric, so the system should prioritize stable contours and calibrated coordinates rather than whole-image template correlation. The camera is mounted above the part, but mild perspective and radial distortion are present. The inspection specification is tighter than the uncertainty of an uncalibrated setup.

Approach and Solution

1. Use diffuse, coaxial illumination and a matte fixture to reduce reflections.
2. Calibrate the camera with a precision dot target at the part plane.
3. Correct lens distortion and rectify the working plane.
4. Segment the flange using a combination of intensity and geometric constraints.
5. Apply limited morphological opening only to remove isolated noise.
6. Extract the external contour and fit a circle robustly.
7. Detect the central bore and four hole contours using connected components and circularity constraints.
8. Compute each hole centroid and transform centroids into polar coordinates about the flange center.
9. Estimate angular spacing and compare it with the nominal 90° pattern.
10. Repeat the measurement over a representative sample and estimate bias and repeatability against a calibrated reference gauge.

For a hole centroid (x_i, y_i) and flange center (x_c, y_c), the radial distance and angle are

Angles should be unwrapped before calculating successive differences across the 0°/360° boundary. A missing hole is indicated by fewer than four valid components or by a large deviation from the expected radius. Orientation can be reported as the angle of the first hole relative to a datum feature.

Outcome

Calibration removes position-dependent scale error, robust circle fitting reduces the influence of burrs, and polar analysis separates hole count from orientation. The system classifies parts using measured deviations and an uncertainty-aware decision rule. Parts close to a specification limit are diverted for secondary inspection rather than automatically rejected.

Learning Points

The case demonstrates that reliable metrology is a system problem. Better descriptors cannot compensate for uncontrolled illumination, inappropriate calibration, or an unsuitable geometric model. The inspection algorithm must also reflect manufacturing tolerances and the consequences of false acceptance and false rejection.

12. Limitations, Safety, and Professional Practice

Feature algorithms can fail when objects overlap, surfaces are reflective, lighting changes, the object is out of focus, or the appearance of a valid part varies beyond the training or validation data. A method validated on one camera cannot automatically be transferred to another camera with different spectral response or distortion.

AI-based feature extraction and defect classification can be valuable when defects are difficult to describe analytically. However, training data must represent acceptable variation, rare defects, environmental changes, and borderline cases. Performance should be reported using appropriate validation data, confusion matrices, false-acceptance rates, and false-rejection rates. A high average accuracy can hide unacceptable performance on rare but hazardous defects.

In safety-related applications, vision should not be the sole safeguard unless the complete system has been validated to the applicable functional-safety requirements. Access protection, emergency stops, and interlocks require appropriate safety-rated design. Images may contain proprietary products or identifiable people; storage, access control, retention, and privacy should follow organizational policy and applicable law.

Best practice includes controlled illumination, rigid mounting, calibration records, version-controlled algorithms, traceable reference artifacts, periodic verification, independent test sets, and monitoring for drift. The measurement result, image, calibration version, and decision rule should be traceable when quality or regulatory evidence is required.

Contemporary Developments

Current systems increasingly combine classical geometry with machine learning. A detector may locate a component, while calibrated contour fitting supplies the dimensional measurement. Event cameras provide high temporal resolution for motion, multispectral cameras distinguish materials, and three-dimensional sensors support height and volume measurement. Differentiable vision methods and learned local descriptors can improve registration, but they do not remove the need for calibration and uncertainty assessment.

The most dependable architecture is often hybrid: physics-based imaging and metrology for quantities that must be traceable, complemented by learning-based methods for appearance variation and defect detection.

Chapter Summary

Feature extraction transforms images into task-relevant measurements. Edges and contours represent boundaries; corners and keypoints support localization and matching; blobs and connected components represent regions; color and texture describe appearance; and moments summarize region geometry. Shape descriptors such as circularity, principal axes, Hu moments, and fitted geometric primitives enable classification and inspection.

Template matching and local descriptors solve different correspondence problems and have distinct sensitivity to scale, rotation, illumination, and occlusion. Registration aligns image or world coordinates using transformation models selected according to physical geometry. Metrology requires calibration, distortion correction, appropriate feature fitting, and explicit uncertainty assessment. A measurement system is credible only when its imaging conditions, algorithms, reference standards, validation data, and decision rules are controlled and traceable.

Key Terms

Feature extraction: Conversion of image data into numerical or structural properties.

Contour: An ordered representation of a region boundary.

Edge: A location of rapid image-intensity change.

Blob: A connected region sharing a specified image property.

Connected component: A maximal set of mutually connected foreground pixels.

4-connectivity: Connectivity through horizontal and vertical neighbors.

8-connectivity: Connectivity including diagonal neighbors.

Moment: A weighted sum that describes the spatial distribution of a region.

Centroid: The first-moment center of a region.

Central moment: A moment calculated relative to the centroid.

Keypoint: A distinctive local image location used for correspondence.

Descriptor: A numerical representation used to characterize a feature.

Circularity: A perimeter-area descriptor, commonly 4πA/P².

Registration: Alignment of images or coordinate systems.

Homography: A projective transformation relating corresponding planar points.

Calibration: Estimation of image-to-physical geometry and camera parameters.

Repeatability: Agreement of repeated measurements under unchanged conditions.

Uncertainty: Quantified doubt associated with a measurement result.

Guard band: A decision margin that accounts for measurement uncertainty.

Metrology: The science of measurement and its traceability.

Review Questions

5 Short Answer Questions

1. What is the difference between an edge and a contour?
2. Why can excessive smoothing reduce measurement quality?
3. Distinguish 4-connectivity from 8-connectivity.
4. What physical information is represented by the centroid of a binary region?
5. Why is calibration necessary for dimensional measurement?

5 Long Answer Questions

1. Explain a complete feature-extraction pipeline from image acquisition to an inspection decision.
2. Describe image moments and show how zeroth- and first-order moments produce area and centroid.
3. Compare region-based, boundary-based, local-feature-based, and template-based shape descriptors.
4. Explain the assumptions and applications of a planar homography.
5. Discuss the major sources of uncertainty in camera-based metrology and methods for controlling them.

3 Analytical / Application Questions

1. A component has a segmented area of 8,400 pixels, and the calibrated pixel size is 0.08 mm in both directions. Calculate its physical area and equivalent circular diameter. State two assumptions behind the calculation.
2. Design a vision inspection method for a rectangular plate with four holes, including illumination, segmentation, feature extraction, calibration, and pass/fail logic.
3. A production measurement is close to a tolerance limit but has substantial repeatability variation. Explain how you would decide whether to accept, reject, or remeasure the part, and what additional evidence you would collect.

Multiple Choice Questions with Answers

Question 1

Which quantity is obtained directly from first-order raw moments of a binary region?

A. Centroid coordinates

B. Contour curvature

C. Gradient orientation

D. Template correlation

Correct Answer: A. Centroid coordinates

Question 2

In connected-component analysis, 8-connectivity differs from 4-connectivity because it includes which additional neighbors?

A. Only horizontal neighbors

B. Only vertical neighbors

C. Diagonal neighbors

D. Pixels in a different image

Correct Answer: C. Diagonal neighbors

Question 3

For a region with area A and perimeter P, which expression is the commonly used circularity descriptor?

A. A/P

B. 4πA/P²

C. P²/A

D. 2πP/A

Correct Answer: B. 4πA/P²

Question 4

What is the principal purpose of calibration in image metrology?

A. To increase image file size

B. To convert image coordinates into physically meaningful coordinates

C. To remove all segmentation errors

D. To guarantee perfect repeatability

Correct Answer: B. To convert image coordinates into physically meaningful coordinates

Question 5

Which transformation is generally appropriate for perspective changes between views of the same planar object?

A. A projective homography

B. A one-dimensional threshold

C. A color histogram

D. A morphological opening

Correct Answer: A. A projective homography

Question 6

A binary component contains 2,000 pixels, and each pixel represents 0.1 mm × 0.1 mm. What is its estimated area?

A. 2 mm²

B. 20 mm²

C. 200 mm²

D. 2,000 mm²

Correct Answer: B. 20 mm²

Question 7

Which statement about the Harris corner response is correct?

A. It is designed to identify regions with significant variation in two directions

B. It measures only the average color of an image

C. It requires a calibrated depth sensor

D. It always gives a reliable orientation for circular regions

Correct Answer: A. It is designed to identify regions with significant variation in two directions

Question 8

If L = kn, with independent uncertainties in k and n, which is the first-order variance approximation?

A. u_L² = u_k² + u_n²

B. u_L² = n²u_k² + k²u_n²

C. u_L = kn

D. u_L² = k²n²

Correct Answer: B. u_L² = n²u_k² + k²u_n²

Question 9

What is a principal limitation of a color histogram?

A. It cannot count pixels

B. It discards spatial arrangement

C. It requires no illumination control

D. It only works with binary images

Correct Answer: B. It discards spatial arrangement

Question 10

Why might robust circle fitting be preferred to ordinary least squares in flange inspection?

A. It ignores all valid contour points

B. It reduces the influence of outliers such as burrs or occlusions

C. It eliminates the need for calibration

D. It converts RGB directly into depth

Correct Answer: B. It reduces the influence of outliers such as burrs or occlusions