Classical Recognition and Machine Learning

Introduction and Context

Machine vision becomes useful when an image is converted into a reliable decision: identifying a component, locating a defect, estimating a dimension, or determining whether an assembly is correctly positioned. Image acquisition and preprocessing produce measurements, but recognition assigns meaning to those measurements. Classical recognition and machine learning provide the mathematical and engineering tools for this stage.

A recognition system normally follows a pipeline:

1. image acquisition and calibration;
2. preprocessing and segmentation;
3. feature extraction;
4. feature selection or dimensionality reduction;
5. model training;
6. prediction or decision;
7. performance evaluation and monitoring.

The central engineering principle is that model quality depends not only on the classifier. Illumination, optics, segmentation errors, feature stability, class imbalance, data leakage, and the cost of incorrect decisions can dominate the final result.

Learning Objectives

After studying this chapter, the reader should be able to:

• construct a recognition pipeline for an industrial vision problem;
• distinguish engineered features from learned representations;
• apply feature selection and dimensionality reduction appropriately;
• explain classification, regression, clustering, and anomaly detection;
• formulate probabilistic decisions using posterior probabilities and loss;
• divide data into training, validation, and test sets without leakage;
• diagnose overfitting, imbalance, and distribution shift;
• interpret model outputs and communicate limitations;
• select evaluation measures appropriate to the engineering application.

1. Recognition as a Measurement-and-Decision Problem

Let an image be represented by a vector of measured features x ∈ ℝ^p, where p is the number of features. A trained recognition model maps this vector to an output. For classification, the output is a class label ŷ ∈ {1, …, K}. For regression, it is a numerical estimate ŷ ∈ ℝ.

The distinction between measurement and decision is important. A feature such as circularity is a measurement derived from an object boundary; the decision “acceptable bearing” is an interpretation based on a model and a specification. A good pipeline preserves this distinction so that failures can be diagnosed.

A practical feature vector may contain:

• geometric measurements: area, perimeter, bounding-box dimensions, aspect ratio, compactness, and hole count;
• intensity measurements: mean, variance, percentiles, and histogram bins;
• texture measurements: gray-level co-occurrence statistics, local binary patterns, or filter responses;
• shape descriptors: contour moments, Fourier descriptors, or Hu moments;
• spatial information: centroid, orientation, and distances between regions.

Features should be physically meaningful, repeatable under expected variation, and available at the time of prediction. If a feature changes strongly with illumination or camera position, calibration or normalization may be needed before model training.

2. Feature Engineering

2.1 Geometric and photometric features

For a segmented binary object, area A is the number of foreground pixels multiplied by the pixel area. If pixel dimensions are s_x and s_y in millimetres, then

where N_f is the number of foreground pixels. This formula assumes a calibrated image with locally uniform pixel scale and a segmentation mask that represents the object accurately.

The perimeter P can be estimated from a contour. A common compactness measure is

For an ideal circle, C = 1; for less compact shapes, it is generally smaller. The result is dimensionless because area has units of length squared and perimeter squared has the same units. Pixel-grid effects and contour estimation influence the measured value.

For an object with covariance matrix of point coordinates, the eigenvectors describe principal orientations and the eigenvalues describe spread along those directions. Orientation is often useful for pose classification, but it can be unstable for nearly circular objects because their principal directions are poorly defined.

Photometric features must account for illumination. A normalized intensity can be formed using a reference background, but normalization is valid only when the reference is measured consistently. In production, controlled lighting is usually more reliable than attempting to correct severe illumination variation computationally.

2.2 Texture features

Texture describes local spatial variation rather than individual pixel values. A gray-level co-occurrence matrix (GLCM) counts pairs of quantized intensity levels separated by a specified displacement. Common statistics include contrast, homogeneity, energy, and correlation. Their values depend on quantization, direction, distance, and image resolution; therefore these parameters must be documented and held constant during training and deployment.

Local binary patterns compare a central pixel with neighboring pixels. They are computationally efficient and can be useful for repetitive surface textures. However, texture descriptors may be sensitive to scale, focus, and lighting direction.

2.3 Feature selection and leakage

Feature selection removes irrelevant or redundant variables. Filter methods rank features using statistics such as correlation or mutual information; wrapper methods evaluate subsets using a model; embedded methods perform selection during training, as in regularized linear models or tree-based methods.

Feature selection must be fitted using training data only. If the complete dataset is used to select features before the split, information from the test set influences the model and produces an optimistically biased estimate.

3. Dimensionality Reduction

High-dimensional feature vectors can increase computation, amplify noise, and make visualization difficult. Dimensionality reduction maps x ∈ ℝ^p to a lower-dimensional vector z ∈ ℝ^q, where q < p.

3.1 Principal component analysis

Principal component analysis (PCA) finds orthogonal directions of maximum variance. After centering the training data, let X be the matrix whose rows are feature vectors. The sample covariance matrix is

where n is the number of training samples and the columns of X have zero mean. PCA solves

where v_i is the ith unit eigenvector and λ_i is its eigenvalue. The principal-component score is z_i = Xv_i. The eigenvalues represent variances along the corresponding directions.

If features have different units or scales, standardization is normally required:

where μ_j and σ_j are the training mean and standard deviation of feature j. Standardization is inappropriate when physical absolute scale is itself meaningful and the model is designed to use it directly, but it is essential for many distance-based and covariance-based methods.

PCA is unsupervised: it preserves variance, not necessarily class separability. A low-variance direction may be highly discriminative. PCA should therefore be evaluated by downstream recognition performance, not only by the percentage of variance retained.

3.2 Other reductions

Linear discriminant analysis seeks directions that improve separation between known classes, subject to assumptions about class distributions. t-SNE and UMAP are useful for exploratory visualization, but their visual distances and clusters should not automatically be interpreted as production decision boundaries. A visualization method is not necessarily a predictive model.

4. Supervised Classification

Classification learns a relationship between features and discrete labels. Typical applications include defect type identification, component recognition, and pass/fail inspection.

4.1 Linear and logistic models

For binary classification, logistic regression models the probability of class 1 as

where β₀ is the intercept and β is the coefficient vector. A threshold, often 0.5 but not necessarily optimal, converts probability into a class decision. Coefficients are interpretable when features are appropriately scaled and other assumptions are considered.

A linear discriminant classifier uses class means and covariance estimates. It is computationally efficient and often effective when classes are approximately separable by linear boundaries. Quadratic discriminant analysis permits class-specific covariance matrices but requires more data to estimate them reliably.

4.2 Nonlinear classifiers

Decision trees divide feature space using threshold tests. Ensembles such as random forests average many randomized trees, reducing variance. Gradient-boosted trees build successive models that focus on prior errors and can provide strong performance on tabular engineered features.

The k-nearest-neighbour method predicts from nearby training examples. It is simple but sensitive to feature scaling, irrelevant features, the choice of k, and computational cost at inference.

Support vector machines seek a decision boundary with a large margin. Kernel functions can represent nonlinear boundaries, but kernel parameters and regularization require careful validation.

4.3 Evaluation measures

For binary classification, the confusion matrix contains true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN). Important measures are

Precision answers “when the system predicts positive, how often is it correct?” Recall answers “of the actual positives, how many were found?” The F1 score is

These definitions assume nonzero denominators. Accuracy can be misleading under class imbalance. Receiver operating characteristic (ROC) curves show true-positive rate against false-positive rate over thresholds; precision-recall curves are often more informative when the positive class is rare.

5. Regression and Estimation

Regression predicts a continuous quantity such as crack length, object diameter, pose angle, or conveyor position. Linear regression has the form

where y is the measured response, ε is an error term, and the coefficients are estimated from training data. The ordinary least-squares objective is

This objective penalizes large errors more strongly. Mean absolute error, MAE = (1/n)∑ᵢ |y_i − ŷ_i|, is less sensitive to outliers than mean squared error, while root mean squared error has the same units as the response:

Regression models should be evaluated against engineering tolerance, not merely an abstract average. A mean error of 0.2 mm may be acceptable for one task and unacceptable for another.

6. Clustering and Anomaly Detection

Unsupervised methods operate without labelled outcomes. Clustering can reveal product variants, operating regimes, or previously unknown patterns.

In k-means, each sample is assigned to one of k clusters and centroids are updated to minimize within-cluster squared distance:

where C_r is cluster r and μ_r is its centroid. The algorithm can converge to a local minimum, so multiple initializations are advisable. It assumes that Euclidean distance and centroid representation are meaningful; it is not automatically appropriate for elongated, nested, or uneven-density clusters.

Anomaly detection models normal observations and flags samples with low conformity. Methods include distance from a normal centroid, one-class support vector machines, isolation forests, and autoencoders. Anomaly detection is particularly useful when defective examples are scarce. However, “anomaly” means unusual relative to the training distribution, not necessarily defective. Changes in lighting, camera focus, or product mix can create false alarms.

7. Probabilistic Decision Models

A probabilistic model estimates uncertainty rather than only returning a label. Bayes’ rule is

where P(y | x) is the posterior probability, P(x | y) is the likelihood, P(y) is the prior probability, and P(x) is the evidence. The evidence normalizes the probabilities across possible classes.

The minimum-risk decision selects action a minimizing expected loss:

where L(a,y) is the loss of taking action a when the true class is y. This framework explains why a threshold should depend on consequences. If missing a dangerous defect is much more costly than rejecting a good part, the system should favour sensitivity, possibly with a human review category.

Probabilities must be calibrated before being interpreted as frequencies. Reliability diagrams, calibration curves, and methods such as Platt scaling or isotonic regression can be used on validation data.

8. Training, Validation, and Generalization

A training set estimates model parameters; a validation set supports model and hyperparameter selection; a test set provides a final, untouched estimate of generalization. The test set must not influence feature selection, normalization, threshold selection, or model choice.

For related images, random splitting can leak information. Frames from the same video, multiple views of one component, or repeated images from one production batch should normally remain in the same split. Grouped or time-based splitting better represents deployment.

Cross-validation partitions the training data into folds. Each fold is used once for validation while the others are used for fitting. It is useful for limited datasets, but all preprocessing steps must be fitted within each training fold. A pipeline object that combines preprocessing and the model helps enforce this rule.

Overfitting occurs when a model captures noise or accidental patterns in training data. Symptoms include excellent training performance and substantially poorer validation performance. Remedies include simpler models, regularization, more representative data, augmentation that reflects physical variation, early stopping, and improved measurement conditions.

Regularized least squares, for example, adds a penalty:

where λ ≥ 0 controls the strength of the L2 penalty. A larger λ generally reduces coefficient magnitude and variance, but excessive regularization can cause underfitting. The intercept is commonly excluded from the penalty.

9. Imbalanced Data and Threshold Design

In inspection, defective parts may be far less common than good parts. A classifier can achieve high accuracy by predicting “good” almost always. Countermeasures include class-weighted loss, balanced sampling, targeted data collection, synthetic examples used cautiously, and threshold adjustment.

The evaluation set should preserve or explicitly report the operational prevalence. If sampling deliberately changes prevalence, precision in deployment cannot be inferred directly without correcting for the new class proportions.

Thresholds should be selected from validation data using a cost matrix, recall requirement, false-alarm limit, or production capacity. Selecting a threshold on the test set invalidates the test estimate.

10. Interpretability and Engineering Governance

Interpretability has several meanings:

• global understanding of which variables influence a model;
• local explanation of one prediction;
• causal understanding of the physical process.

Feature importance is not automatically causality. Correlated features can share or obscure importance, and permutation importance may change when correlated variables are present. Partial-dependence plots describe average model behaviour under assumptions that may be unrealistic for strongly correlated features. Local explanation methods should be treated as approximations, not proof of reasoning.

A production system should log model version, calibration version, camera settings, preprocessing parameters, prediction, confidence, and operator action. Images containing people, labels, or proprietary products require appropriate access control, retention policies, and privacy protection. Safety-related systems need traceability, change control, failure analysis, and human override procedures. A vision model should not silently reject products when its input is outside the validated operating envelope.

11. Applied Case: Illustrative Bearing-Surface Inspection

Background

Consider an illustrative automated inspection cell for small bearings. A monochrome camera views the bearing under diffuse coaxial lighting. The objective is to identify acceptable parts, surface contamination, and edge damage, while estimating the diameter of a critical circular surface.

Problem

Initial trials show high accuracy on images collected during one shift but poor performance after lamp replacement. Defects are rare, and several images are captured from each bearing.

Analysis

The original system used raw mean intensity and a k-nearest-neighbour classifier. Lamp intensity changes altered the intensity distribution. Randomly splitting individual images allowed views of the same bearing to appear in both training and test sets. Accuracy therefore overstated performance. The defect class had a prevalence of approximately 2%, making accuracy particularly misleading.

Approach and solution

The redesigned pipeline first calibrates geometric scale and segments the bearing using an edge-and-circle procedure. Features include fitted inner and outer diameters, circularity, radial intensity profiles, texture statistics, and localized high-contrast regions. Intensity features are normalized using a reference region, while lighting is mechanically controlled.

Images are split by bearing and by production date. The training portion is used to fit standardization, feature selection, and a regularized classifier. A separate validation set determines the classification threshold using a cost matrix in which a missed edge defect is five times as costly as a false rejection. A held-out test set is used only once after design decisions are frozen.

Because defective examples are limited, a one-class anomaly detector trained primarily on verified good parts is evaluated as a supplementary alarm. Its alerts are reviewed by an operator and added to a continually governed data-collection process rather than being treated as automatically labelled defects.

Outcome

The revised system produces slightly lower headline accuracy but substantially higher defect recall and more stable performance after the lamp replacement. Diameter estimates are reported in millimetres with an uncertainty estimate and are checked against the engineering tolerance. Images near the decision threshold are routed to human review.

Learning points

The major improvement came from controlling acquisition, preventing leakage, selecting features connected to the failure mechanism, and choosing a threshold based on consequences. The classifier was important, but it was not the only—or necessarily the largest—source of performance.

12. Best-Practice Recognition Pipeline

A disciplined implementation can follow this sequence:

1. define the operational decision, costs, tolerances, and rejection policy;
2. calibrate optics, geometry, and lighting;
3. collect representative data across products, operators, times, and environmental conditions;
4. label examples using documented criteria and measure label agreement;
5. split data by physical unit, batch, or time where appropriate;
6. build a baseline using transparent features and a simple model;
7. fit preprocessing only on training data;
8. compare models using task-relevant metrics and confidence intervals where feasible;
9. select thresholds on validation data;
10. test once on an untouched set and perform stress tests;
11. monitor drift, false alarms, missed defects, and reject rates in production;
12. maintain versioned records and a retraining/change-control procedure.

Contemporary Developments

Classical engineered features remain valuable where data are limited, measurements must be explainable, and the object geometry is known. Modern systems increasingly combine them with learned visual representations, ensembles, self-supervised pretraining, and vision transformers. Hybrid systems can use a learned detector for localization and calibrated classical measurements for acceptance decisions.

The important engineering question is not whether a method is “classical” or “deep,” but whether it is valid for the available data, failure modes, computational constraints, explainability requirements, and safety obligations. Benchmark performance on unrelated datasets does not guarantee production reliability.

Chapter Summary

Recognition converts image-derived measurements into classifications, estimates, groups, or anomaly decisions. Feature engineering supplies geometric, photometric, texture, and shape information that should be stable and physically relevant. Feature selection and PCA can reduce redundancy, but PCA preserves variance rather than guaranteed class separation.

Classification methods range from logistic regression and discriminant analysis to trees, ensembles, nearest neighbours, and support vector machines. Regression estimates continuous quantities and must be evaluated against engineering tolerances. Clustering discovers structure without labels, while anomaly detection models unusual behaviour relative to a reference population.

Training, validation, and testing must be separated to prevent optimistic results. Grouped or time-based splitting is essential when images are related. Imbalance requires suitable metrics, sampling, weighting, and threshold design. Probabilistic decisions allow unequal costs to be represented explicitly. Interpretability, calibration, logging, privacy, safety, and drift monitoring are part of a professional deployment, not optional additions.

Key Terms

Feature engineering: Designing measurable variables from images for a recognition task.

Feature selection: Choosing a subset of available features.

Feature leakage: Use of information unavailable at prediction time or improperly taken from evaluation data.

Standardization: Transforming a feature using its training mean and standard deviation.

PCA: An unsupervised linear transformation based on covariance eigenvectors.

Classification: Prediction of a discrete category.

Regression: Prediction of a continuous numerical response.

Confusion matrix: Table of correct and incorrect classification outcomes.

Precision: Fraction of predicted positives that are truly positive.

Recall: Fraction of actual positives that are detected.

F1 score: Harmonic mean of precision and recall.

Clustering: Unsupervised grouping of similar observations.

Anomaly detection: Identification of observations that differ from a reference distribution.

Overfitting: Learning accidental training patterns that do not generalize.

Regularization: Penalization or constraint used to control model complexity.

Class imbalance: Unequal representation of target classes.

Calibration: Agreement between predicted probabilities and observed frequencies.

Data drift: Change in the input or outcome distribution after deployment.

Expected loss: Probability-weighted cost of a decision.

Interpretability: Ability to understand or explain model behaviour.

Review Questions

Short Answer Questions

1. Why can accuracy be misleading for a rare-defect inspection problem?
2. What is the purpose of standardizing features before applying PCA?
3. Distinguish precision from recall.
4. Why must preprocessing be fitted using training data only?
5. What does an anomaly detector actually measure?

Long Answer Questions

1. Explain the complete feature-engineering process for a segmented industrial object, including geometric, intensity, and texture features.
2. Compare logistic regression, decision trees, random forests, and k-nearest neighbours for machine-vision classification.
3. Explain PCA mathematically and discuss why maximum variance does not guarantee maximum class separability.
4. Describe how training, validation, and test sets should be constructed when many images come from the same physical component.
5. Explain how expected loss can be used to choose a decision threshold in a safety-relevant inspection system.

Analytical / Application Questions

1. A classifier reports TP = 90, FP = 30, FN = 10, and TN = 870. Calculate accuracy, precision, recall, and F1 score. Which measure would be most important if missed defects are costly?
2. Design a recognition pipeline for estimating the diameter and classifying damage on manufactured circular components. Specify acquisition controls, features, model types, validation strategy, and production monitoring.
3. A model performs well during training but poorly after a camera is replaced. Diagnose at least four possible causes and propose tests or corrective actions for each.

Multiple Choice Questions with Answers

Question 1

Which measure is the fraction of actual positive samples detected by a classifier?

A. Accuracy

B. Precision

C. Recall

D. Specificity

Correct Answer: C. Recall

Question 2

In PCA, an eigenvalue of the covariance matrix represents the:

A. Class prior probability

B. Variance along a principal direction

C. Classification threshold

D. Number of classes

Correct Answer: B. Variance along a principal direction

Question 3

Why should images from the same physical component usually remain in one data split?

A. To increase image resolution

B. To prevent related samples from causing data leakage

C. To guarantee balanced classes

D. To eliminate preprocessing

Correct Answer: B. To prevent related samples from causing data leakage

Question 4

For a perfect circular object, the compactness measure C = 4πA/P² is ideally:

A. 0

B. 0.5

C. 1

D. π

Correct Answer: C. 1

Question 5

Which method is primarily unsupervised?

A. Logistic regression

B. k-means clustering

C. Linear discriminant analysis

D. Support vector classification with labelled data

Correct Answer: B. k-means clustering

Question 6

If false negatives are much more costly than false positives, a practical system should generally:

A. Ignore the decision threshold

B. Choose a threshold using the cost structure

C. Optimize accuracy only

D. Remove the positive class

Correct Answer: B. Choose a threshold using the cost structure

Question 7

What is the main purpose of regularization?

A. To increase image brightness

B. To reduce model complexity and overfitting

C. To create test labels

D. To guarantee perfect calibration

Correct Answer: B. To reduce model complexity and overfitting

Question 8

Which statement about PCA is correct?

A. It always maximizes class separation

B. It is supervised by default

C. It finds orthogonal directions of maximum variance

D. It requires exactly two features

Correct Answer: C. It finds orthogonal directions of maximum variance

Question 9

For a rare-defect problem, which evaluation tool is often more informative than accuracy?

A. Precision-recall analysis

B. Image file size

C. Camera frame rate alone

D. Number of training epochs alone

Correct Answer: A. Precision-recall analysis

Question 10

In expected-risk decision-making, the loss function represents:

A. The number of image pixels

B. The cost of an action when a particular true class occurs

C. The covariance eigenvalue

D. The lighting wavelength

Correct Answer: B. The cost of an action when a particular true class occurs