Convolutional Neural Networks

Introduction

Images contain spatial structure. Nearby pixels usually represent nearby parts of the same object, and meaningful visual patterns—such as edges, corners, textures, and shapes—appear at different locations. A conventional fully connected neural network treats every pixel as an unrelated input and therefore requires many parameters. Convolutional neural networks (CNNs) address this problem by exploiting local connectivity, shared weights, and hierarchical feature learning.

CNNs are central to image classification, object detection, semantic segmentation, medical-image analysis, industrial inspection, facial recognition, autonomous systems, and visual search. This chapter develops the main operations of CNNs: convolution, kernels, padding, stride, feature maps, pooling, and classification. It also explains how to design, train, evaluate, and responsibly deploy a practical CNN.

Learning Objectives

After studying this chapter, you should be able to:

• explain why CNNs are suitable for visual data;
• distinguish kernels, filters, feature maps, channels, padding, and stride;
• calculate convolution output dimensions and perform a simple convolution;
• explain pooling and its effects on representation and computation;
• design a basic CNN for image classification;
• identify overfitting, data leakage, class imbalance, and deployment risks;
• interpret CNN performance using suitable evaluation measures.

1. Why Images Require Specialized Networks

A grayscale image can be represented as a matrix of pixel intensities. A colour image normally has three channels—red, green, and blue—so its shape may be height × width × 3. Pixel values are often scaled to [0, 1] or standardized using the training-set mean and standard deviation.

A fully connected layer connecting an image of height H, width W, and C channels to N neurons requires approximately H × W × C × N weights, excluding biases. For a 224 × 224 RGB image connected to 1,000 neurons, this is 150,528,000 weights. Such a layer is computationally expensive and does not explicitly preserve local spatial relationships.

CNNs reduce this burden through:

1. Local receptive fields: a neuron examines only a small region of the input.
2. Parameter sharing: the same kernel is applied at many spatial positions.
3. Hierarchical representation: early layers learn simple patterns, while deeper layers combine them into shapes and object parts.
4. Controlled spatial reduction: stride and pooling reduce resolution while preserving useful information.

Parameter sharing also gives a useful form of translation tolerance: a learned edge detector can respond to an edge whether it occurs near the top or bottom of an image. This does not provide complete invariance; large changes in viewpoint, scale, illumination, or occlusion may still affect performance.

2. Convolutional Representations

2.1 Kernels, filters, and feature maps

A kernel (or kernel matrix) is a small array of learnable weights, such as 3 × 3. It is moved across the input. At each position, corresponding input values and kernel weights are multiplied and summed, usually with a bias. The resulting values form an output feature map.

For a two-dimensional single-channel input X and kernel K of height K_h and width K_w, the cross-correlation operation commonly called convolution in deep-learning libraries is:

Here, X is the input, Y is the output feature map, b is the bias, s is the stride, and i and j identify an output position. Deep-learning implementations generally use cross-correlation, meaning the kernel is not flipped. In mathematical convolution, K would be reversed spatially; the distinction does not change the learning capability because the weights are learned.

For a multi-channel input, each output filter has a separate kernel slice for every input channel. The values are summed across channels:

Here, C_in is the number of input channels and m identifies one of C_out output filters. A convolution layer therefore produces C_out feature maps. Its parameter count is:

The added 1 represents one bias per output filter. This formula assumes a standard dense convolution, with no grouped or depthwise structure.

2.2 Example of a small convolution

Suppose a 3 × 3 image patch is:

and the kernel is:

With zero bias and stride 1, the output is:

1×1 + 2×0 + 0×(−1) + 2×1 + 1×0 + 1×(−1) + 0×1 + 1×0 + 2×(−1) = 2

This kernel responds to a difference between the left and right sides of the patch and can therefore act as a simple vertical-edge detector. Learned kernels are not usually assigned manually; gradient descent adjusts them using training data.

3. Padding, Stride, and Output Dimensions

3.1 Padding

Padding adds values around the input border, most commonly zeros.

• Valid convolution: no padding. The output is smaller.
• Same padding: padding is selected so that, with stride 1, output height and width equal input height and width.
• Explicit padding: a specified number of rows and columns is added.

Padding allows border pixels to participate in approximately the same number of computations as interior pixels. Excessive padding, however, may introduce artificial boundaries.

3.2 Stride

Stride is the number of pixels by which the kernel moves at each step. Stride 1 evaluates every neighbouring position. Stride 2 skips positions and usually reduces spatial dimensions, lowering computation but discarding some detail.

For input height H, input width W, padding P_h and P_w, kernel height K_h and width K_w, and stride S_h and S_w, the output dimensions are:

The floor function is required when the kernel does not fit an additional complete position. This formula assumes dilation 1. With dilation D, the effective kernel height is K_eff = D(K_h − 1) + 1, and K_h in the formula is replaced by K_eff; the same applies to width.

Worked example

An input is 32 × 32, the kernel is 3 × 3, padding is 1, and stride is 1:

Thus, each output feature map is 32 × 32. If padding is 0 and stride is 2:

The output is 15 × 15.

4. Activation Functions and Feature Hierarchies

A convolution is commonly followed by a nonlinear activation. The rectified linear unit (ReLU) is:

It is computationally simple and helps networks learn nonlinear functions. Its limitation is that a neuron can become inactive for many inputs when its pre-activation remains negative, a condition sometimes called a “dying ReLU.” Variants such as leaky ReLU assign a small negative slope.

A typical hierarchy is:

• early layers: edges, colour contrasts, and simple textures;
• middle layers: corners, motifs, and parts;
• deeper layers: object-specific structures and semantic patterns.

This interpretation is useful but not absolute. A feature map need not correspond to one human-interpretable object, and deeper representations may be distributed across many channels.

5. Pooling and Downsampling

Pooling summarizes a local region without learning a separate weight for each position.

Max pooling returns the largest value:

Average pooling returns the arithmetic mean:

R is the pooling window and |R| is its number of elements. Pooling is applied independently to each channel, so it changes height and width but normally not the number of channels.

For a 2 × 2 max-pooling window with stride 2 applied to:

The output is:

Pooling reduces memory and computation and can make representations less sensitive to small translations. It can also discard precise location information. Many modern architectures use strided convolutions, adaptive pooling, or carefully designed multi-scale methods instead of extensive fixed pooling.

6. A Typical CNN Architecture

A basic image-classification CNN may follow this sequence:

1. input image;
2. convolution → ReLU;
3. convolution → ReLU → pooling;
4. convolution → ReLU → pooling;
5. global average pooling or flattening;
6. fully connected classifier;
7. softmax output.

The flatten operation converts a tensor into a vector. It can create many parameters if feature maps remain large. Global average pooling averages each feature map over its spatial dimensions, producing one value per channel. It reduces parameters and often improves robustness, although it may lose detailed spatial information.

For C class scores z₁, …, z_C, softmax converts logits to probabilities:

The probabilities are nonnegative and sum to 1. For one-hot target vector y, categorical cross-entropy is:

If the correct class is t, this becomes L = −log(p_t). Softmax and cross-entropy are normally used together for mutually exclusive classes. For multi-label classification, where several labels may be true simultaneously, independent sigmoid outputs and binary cross-entropy are more appropriate.

Batch normalization and regularization

Batch normalization standardizes intermediate activations using batch statistics during training and running estimates during inference. It can improve optimization, but its behaviour depends on batch size and implementation. Dropout randomly removes activations during training, while weight decay penalizes large weights. Data augmentation—such as random crops, flips, colour changes, and small rotations—can reduce overfitting when transformations preserve the class meaning.

7. Building a CNN for Image Classification

A professional workflow begins before model construction.

Data preparation

• Define classes precisely and document labelling rules.
• Remove corrupt, duplicated, or inappropriate images.
• Split data into training, validation, and test sets before repeated experimentation.
• Keep related images—such as frames from one video or images of one patient—in the same partition to prevent leakage.
• Resize images consistently and normalize using statistics calculated from training data.
• Inspect class frequencies and use stratification where suitable.

Model design

Begin with a small baseline. For example, a 64 × 64 RGB image may pass through convolutions with 32, 64, and 128 filters, using 3 × 3 kernels and ReLU, with downsampling between blocks. Global average pooling followed by a dense output layer is often a reasonable diploma-level design. Increasing depth or width is justified only when validation evidence supports it.

Training loop

For each mini-batch, the network performs a forward pass, calculates loss, backpropagates gradients, and updates weights using an optimizer such as stochastic gradient descent or Adam. Important hyperparameters include learning rate, batch size, number of epochs, kernel sizes, filter counts, augmentation strength, and regularization.

Validation loss and metrics should be monitored during training. Early stopping can terminate training when validation performance has stopped improving. The test set should be used once for final unbiased assessment, not repeatedly for model selection.

Evaluation

Accuracy is:

For a chosen positive class, precision and recall are:

Here TP, TN, FP, and FN denote true positives, true negatives, false positives, and false negatives. Accuracy can be misleading for imbalanced data. A medical-screening system may prioritize recall, while a costly manual inspection process may emphasize precision. Confusion matrices, per-class recall, macro-averaged scores, calibration, and threshold analysis provide a more complete evaluation.

8. Applied Case: Illustrative Industrial Defect Inspection

Background

A manufacturer wishes to identify scratches on metal components using camera images. This is an illustrative case rather than a report of a specific company.

Problem

Manual inspection is slow and variable. Defects are rare, lighting changes across production shifts, and visually similar marks may be acceptable manufacturing variation.

Analysis

A random image split could place images of the same component or production batch in both training and test sets, producing overly optimistic results. Accuracy alone would also be misleading because most parts are defect-free.

Approach and solution

The team first defined a defect taxonomy with inspectors and created a batch-level split. Images were cropped to the inspection region, normalized, and augmented with realistic brightness changes and small translations. A CNN with three convolutional blocks, ReLU activations, moderate dropout, and global average pooling was trained using weighted cross-entropy to address imbalance.

Performance was evaluated using defect recall, precision, confusion matrices, and false alarms per production batch. Grad-CAM or related saliency methods were used as diagnostic aids to check whether the model focused on the component rather than labels, fixtures, or background marks. These explanations were not treated as proof of causal reasoning.

Outcome

The model was accepted only after testing on later production batches and on lighting conditions not represented in training. A human review stage was retained for uncertain predictions. Monitoring was established for changes in camera position, illumination, defect frequency, and performance.

Learning points

The principal challenge was not selecting a fashionable architecture. It was defining labels, preventing leakage, representing operating conditions, choosing appropriate metrics, and integrating the model safely into a human and production workflow.

9. Limitations, Ethics, and Deployment

CNNs can learn spurious correlations. A classifier may associate a disease label with a hospital marker, or a product label with a particular background. Dataset imbalance and under-representation can cause unequal error rates across groups or environments. In sensitive applications, consent, privacy, data minimization, access control, retention policies, and applicable data-protection law must be addressed.

Deployment requires attention to latency, memory, energy use, hardware compatibility, and failure handling. Quantization and pruning can reduce resource requirements, but they may reduce accuracy. A model should have a confidence threshold and an escalation path when predictions are uncertain. Continuous monitoring is necessary because data distributions and camera conditions change. Model cards, dataset documentation, version control, audit logs, and rollback procedures support governance.

CNNs also have security vulnerabilities. Adversarial perturbations, poisoned training data, and distribution shifts can cause incorrect predictions. Testing should include realistic variation, stress tests, out-of-distribution examples, and human review of high-risk cases.

10. Contemporary Developments

Modern vision systems increasingly use residual connections, attention mechanisms, transfer learning, vision transformers, self-supervised pretraining, and hybrid CNN-transformer architectures. Transfer learning is especially valuable when a diploma-level project has limited labelled data: a model pretrained on a large dataset can be adapted to a related task, followed by careful validation.

CNNs remain useful because their locality and parameter sharing are computationally efficient, particularly on edge devices. The correct architecture depends on the amount and type of data, latency requirements, interpretability needs, hardware, and risk of errors—not simply on novelty.

Best-Practice Checklist

• Establish a clear problem definition and label policy.
• Prevent leakage through group- or time-based splitting when necessary.
• Start with a reproducible baseline.
• Track tensor shapes and parameter counts at every layer.
• Use augmentation that reflects real variation, not transformations that change class meaning.
• Monitor training and validation curves.
• Evaluate per-class performance and operational costs.
• Inspect failure cases and possible shortcut features.
• Test on conditions outside the training distribution.
• Document data, preprocessing, model versions, metrics, and limitations.
• Provide human oversight for high-impact decisions.

Chapter Summary

Convolutional neural networks exploit local connectivity and shared weights to learn spatially meaningful image representations efficiently. A kernel scans an input and produces a feature map; multiple filters produce multiple channels. Padding controls border treatment, stride controls movement and often downsampling, and pooling summarizes local regions. Output dimensions depend on input size, kernel size, padding, stride, and, when used, dilation.

Convolutional blocks are commonly combined with nonlinear activations, normalization, regularization, and a classifier. Softmax is suitable for mutually exclusive classes, whereas sigmoid outputs suit multi-label problems. Reliable CNN development requires sound data splitting, normalization, augmentation, validation, appropriate metrics, and careful analysis of errors. In professional settings, privacy, fairness, robustness, monitoring, human oversight, and deployment constraints are as important as model accuracy.

Key Terms

Convolution: A local weighted operation that produces responses from an input.

Kernel: A learnable weight array applied across a spatial input.

Filter: A complete set of kernel weights that produces one output feature map.

Feature map: The spatial responses produced by a filter.

Channel: One depth component of an input or output tensor.

Receptive field: The region of the original input influencing a particular activation.

Padding: Values added around an input boundary.

Stride: The step size used when moving a kernel or pooling window.

Valid convolution: Convolution performed without padding.

Same padding: Padding intended to preserve spatial dimensions under specified conditions.

Pooling: Local summarization used for downsampling.

Max pooling: Pooling that selects the largest value in a region.

ReLU: The activation function max(0, x).

Logit: An unnormalized class score before softmax.

Softmax: A function converting class scores into a probability distribution.

Cross-entropy: A loss measuring disagreement between target labels and predicted probabilities.

Data augmentation: Label-preserving transformations used to improve generalization.

Transfer learning: Adapting knowledge learned from one dataset or task to another.

Data leakage: Unintended use of information from validation or test data during training or selection.

Distribution shift: A change between training conditions and deployment conditions.

Review Questions

Short Answer Questions

1. Why do CNNs generally require fewer parameters than fully connected networks for image inputs?
2. Define padding and stride.
3. What is the difference between a kernel and a feature map?
4. State one advantage and one limitation of max pooling.
5. When is sigmoid output preferable to softmax output?

Long Answer Questions

1. Explain how local connectivity, parameter sharing, and hierarchical learning make CNNs suitable for image analysis.
2. Derive the two-dimensional convolution output-size formula and explain every variable.
3. Describe a complete workflow for training and evaluating a CNN for image classification.
4. Compare flattening followed by dense layers with global average pooling.
5. Discuss the principal ethical, robustness, and governance risks of deploying CNNs in a high-impact application.

Analytical / Application Questions

1. An input image is 64 × 64. A 5 × 5 convolution uses padding 2 and stride 2 with 16 filters. Calculate the spatial dimensions and number of output channels. Explain the effect of this choice.
2. A defect dataset contains 9,500 normal images and 500 defective images. Explain why accuracy is inadequate, recommend suitable metrics, and propose two methods for addressing imbalance.
3. A CNN performs well on randomly divided test images but poorly on images from a later production month. Analyse possible causes and design an investigation and remediation plan.

Multiple Choice Questions with Answers

Question 1

Which CNN mechanism allows the same learned detector to be applied at different image locations?

A. Parameter sharing

B. One-hot encoding

C. Dropout

D. Batch shuffling

Correct Answer: A. Parameter sharing

Question 2

For an input of 32 × 32, a 3 × 3 kernel, padding 1, and stride 1, what is the spatial output size?

A. 28 × 28

B. 30 × 30

C. 32 × 32

D. 34 × 34

Correct Answer: C. 32 × 32

Question 3

What does stride 2 usually do compared with stride 1?

A. Doubles the number of channels

B. Moves the kernel in larger steps and often reduces spatial dimensions

C. Removes all nonlinearities

D. Guarantees higher accuracy

Correct Answer: B. Moves the kernel in larger steps and often reduces spatial dimensions

Question 4

A standard convolution has a 3 × 3 kernel, 8 input channels, and 16 output filters, including one bias per filter. How many parameters does it have?

A. 384

B. 400

C. 1,152

D. 1,168

Correct Answer: D. 1,168

Question 5

Which operation selects the largest value within each local window?

A. Average pooling

B. Max pooling

C. Softmax

D. Batch normalization

Correct Answer: B. Max pooling

Question 6

Which output formulation is generally appropriate when one image may have several true labels simultaneously?

A. Independent sigmoid outputs

B. A single linear output only

C. Mutually exclusive softmax classes

D. Max pooling outputs

Correct Answer: A. Independent sigmoid outputs

Question 7

In a heavily imbalanced defect-detection dataset, which metric is especially useful for measuring how many actual defects were found?

A. Recall

B. True-negative count alone

C. Image width

D. Parameter count

Correct Answer: A. Recall

Question 8

What is the main purpose of data augmentation?

A. To create label-preserving training variation

B. To guarantee zero training loss

C. To replace the test set

D. To remove all convolution layers

Correct Answer: A. To create label-preserving training variation

Question 9

Why is a random image split potentially unsafe for frames extracted from the same video?

A. It always reduces image resolution

B. It can place highly related frames in both training and test sets, causing leakage

C. It prevents convolution

D. It changes RGB values to labels

Correct Answer: B. It can place highly related frames in both training and test sets, causing leakage

Question 10

For mutually exclusive classes, what does softmax produce from class logits?

A. A set of probabilities that sum to one

B. Binary masks with no probabilities

C. Only negative values

D. A larger input image

Correct Answer: A. A set of probabilities that sum to one