12. Deep Learning and Generative AI for Images

Introduction and Context

Image processing has moved from manually designed filters and handcrafted features toward systems that learn representations directly from data. This transition has been driven by deep learning, particularly convolutional neural networks (CNNs), vision transformers, autoencoders, generative adversarial networks (GANs), and diffusion models. These methods can classify images, locate objects, delineate anatomical or industrial regions, restore degraded observations, and synthesize images that were not present in the training set.

The change is not merely a replacement of one algorithm by another. It represents a change in how visual information is modelled. Classical image-processing pipelines typically separated feature extraction, inference, and post-processing. Deep models learn several of these stages jointly, often optimizing millions or billions of parameters through gradient-based learning. Generative models go further: instead of only predicting a label or a boundary, they learn a distribution from which new visual content can be generated or an incomplete observation can be reconstructed.

For postgraduate students, the central challenge is to select an appropriate model, formulate the learning problem correctly, construct reliable data pipelines, and evaluate outputs beyond a single numerical score. A model may achieve high classification accuracy while failing on rare classes, produce visually convincing but factually incorrect content, or perform well on a benchmark but poorly under domain shift. Consequently, technical design must be combined with statistical reasoning, domain knowledge, computational awareness, and responsible governance.

Learning Objectives

After completing this chapter, the reader should be able to:

1. Explain the principles and architectural components of CNNs, vision transformers, autoencoders, GANs, and diffusion models.

2. Formulate image classification, detection, segmentation, restoration, and synthesis tasks as learning problems.

3. Compare supervised, transfer, self-supervised, foundation-model, and generative learning strategies.

4. Select suitable losses, metrics, data-splitting methods, and validation procedures.

5. Analyse model efficiency, robustness, interpretability, safety, and deployment constraints.

6. Design and evaluate an image-learning workflow for a professional or research application.

From Pixels to Learned Representations

An image can be represented as a tensor whose dimensions correspond to height, width, and channels. A learning system maps this tensor to an output such as a class probability vector, a set of bounding boxes, a segmentation mask, or a new image. Let a model with parameters theta be written as y-hat = f-theta(x). Training seeks parameters that minimize an objective over a dataset:

L(theta) = (1/N) sum of l(f-theta(xi), yi) + lambda R(theta).

Here, l is a task loss, R is a regularization term, and lambda controls the penalty on model complexity. In practice, optimization uses stochastic gradient descent or variants such as Adam. The training set is used to estimate parameters, the validation set supports model selection, and the test set provides an approximately unbiased final estimate. Data leakage occurs when information from validation or test data influences training or design decisions.

Representation learning is central to deep vision. Early layers may learn edges, colour transitions, or local textures; intermediate layers combine these into motifs and parts; deeper layers represent object-level or semantic relationships. These descriptions are useful intuitions rather than strict rules: features may be distributed across layers, and some models learn representations that do not correspond neatly to human concepts.

Convolutional Neural Networks

A convolutional layer applies learnable kernels over local image neighbourhoods. For an input feature map X and kernel K, a simplified output is

Y(i,j) = sum over u,v,c K(u,v,c) X(i+u,j+v,c) + b.

Weight sharing allows the same detector to be applied at different locations and substantially reduces the number of parameters compared with a fully connected layer. Nonlinear activation functions, commonly ReLU or its variants, allow multiple layers to express complex mappings. Pooling or strided convolution reduces spatial resolution and increases the effective receptive field.

Modern CNN design uses residual connections, normalization, depthwise separable convolution, feature pyramids, and attention modules. A residual block learns F(x) and returns x + F(x), helping gradients pass through deep networks. Encoder-decoder CNNs preserve high-resolution information through skip connections and are particularly effective for segmentation and restoration. U-Net is a prominent example: an encoder captures context, while a decoder reconstructs spatial detail using features copied from corresponding encoder levels.

CNNs have strong inductive biases for images: locality, translation-related structure, and hierarchical composition. These biases make them data-efficient in many settings. Their limitations include sensitivity to distribution changes, dependence on large labelled datasets for difficult tasks, and possible overreliance on texture or spurious background cues.

Vision Transformers

Vision transformers divide an image into fixed-size patches, flatten or project each patch into an embedding, add positional information, and process the resulting sequence using self-attention. For queries Q, keys K, and values V, attention is commonly expressed as

Attention(Q,K,V) = softmax(QK-transpose / square root of dk)V.

Self-attention allows one region to interact directly with distant regions. This is useful for modelling long-range relationships, although its computational cost can grow substantially with the number of tokens. Hierarchical transformers, windowed attention, patch merging, and efficient attention mechanisms address this issue.

Transformers often benefit from large-scale pretraining. Without sufficient data or suitable pretraining, a transformer may not outperform a carefully designed CNN. Thus, architecture choice should consider dataset size, transfer-learning availability, resolution, computational budget, and the importance of global context.

Learning Paradigms and Model Adaptation

Supervised Learning

In supervised learning, each image is associated with a target. Classification uses cross-entropy for mutually exclusive classes, binary cross-entropy for independent labels, and specialized losses such as focal loss when hard or minority examples require greater emphasis. Detection combines classification and localization losses. Segmentation may use a combination of cross-entropy, Dice loss, boundary loss, or focal loss.

The loss function encodes what the system is encouraged to value. A pixelwise mean squared error loss may produce smooth restoration results, whereas a perceptual or adversarial loss may produce sharper images but introduce hallucinated detail. Loss selection is therefore a modelling decision rather than a routine implementation detail.

Transfer Learning and Fine-Tuning

Transfer learning begins with a model pretrained on a broad dataset and adapts it to a target domain. A common workflow freezes early layers, trains a new task head, and then gradually unfreezes layers using a smaller learning rate. Full fine-tuning can improve performance when sufficient target data are available, while parameter-efficient methods such as adapters, low-rank updates, or prompt-based adaptation reduce memory and training cost.

The usefulness of transfer depends on similarity between source and target domains. Features learned from natural photographs may transfer effectively to some medical or industrial tasks, but differences in imaging physics, scale, and semantics may require substantial adaptation. Evaluation must therefore be performed on representative target-domain data.

Self-Supervised Learning

Self-supervised learning creates training signals from unlabelled images. In contrastive learning, augmented views of the same image are encouraged to have similar representations, while unrelated images are separated. Masked-image modelling hides image patches and trains the model to predict missing content. Other methods use rotation prediction, instance discrimination, clustering, or teacher-student consistency.

Self-supervised pretraining can reduce annotation requirements, but its success depends on the relationship between pretext task and downstream task. Augmentations must preserve the relevant information. An augmentation that changes disease evidence, manufacturing defects, or geospatial structure may teach the wrong invariance.

Foundation Models

A foundation model is pretrained at scale and adapted to many downstream tasks. Image encoders, vision-language models, segmentation models, and multimodal systems can provide general-purpose representations or zero-shot and few-shot capabilities. However, a foundation model is not automatically reliable in a specialized domain. Its training distribution, licensing conditions, hidden biases, prompt sensitivity, and calibration must be examined before deployment.

Formulating Core Image Tasks

Classification

Classification assigns one or more labels to an image or region. A single-label system may use softmax probabilities, whereas multi-label classification uses independent sigmoid outputs. Accuracy is suitable only when classes and error costs are reasonably balanced. Precision, recall, F1 score, balanced accuracy, receiver operating characteristic area, and precision-recall area provide more informative views. Thresholds should be selected on validation data and may differ by class when the costs of false positives and false negatives are unequal.

Calibration is also important. A probability of 0.8 should correspond approximately to an 80 percent event frequency among comparable predictions. Temperature scaling and other post-training methods can improve calibration without changing the ranking of predictions.

Object Detection

Detection predicts object categories and bounding boxes. Two-stage detectors first generate candidate regions and then classify and refine them; one-stage detectors predict locations and categories more directly. Detection evaluation commonly uses intersection over union (IoU), precision-recall curves, and mean average precision across classes and IoU thresholds.

Small objects, crowded scenes, occlusion, class imbalance, and inaccurate annotations create particular difficulties. Non-maximum suppression removes highly overlapping duplicate predictions, but it can also suppress distinct objects in dense scenes. The appropriate detector depends on accuracy, latency, image resolution, hardware, and the consequences of missed detections.

Semantic and Instance Segmentation

Semantic segmentation assigns a class to every pixel. Instance segmentation additionally distinguishes separate objects belonging to the same class. Panoptic segmentation combines semantic and instance perspectives. Dice coefficient and IoU are common region-overlap measures, but boundary quality, object-level recall, and performance on small regions should also be examined.

Segmentation systems often combine multiscale features, skip connections, attention, and carefully designed decoders. Annotation uncertainty is common, especially at fuzzy boundaries. Treating one annotator's mask as absolute truth can create misleading evaluations; consensus masks, soft labels, or boundary-tolerant metrics may be more appropriate.

Restoration and Inverse Problems

Denoising, deblurring, super-resolution, inpainting, and reconstruction are inverse problems. The observed image y can be represented as y = Hx + n, where x is the unknown clean image, H is a degradation operator, and n is noise. Deep restoration models learn either a direct mapping from y to x or a prior that constrains plausible solutions.

Pixel fidelity metrics such as PSNR and SSIM are useful but incomplete. A visually sharp image may have lower PSNR than a blurred image. In medical or scientific imaging, hallucinated structures are unacceptable even when perceptual quality appears high. Evaluation should include task-specific fidelity, uncertainty, human assessment, and downstream effects.

Generative Models for Images

Generative modelling estimates or approximates a distribution of images. The goal may be unconditional synthesis, conditional generation, image-to-image translation, editing, reconstruction, or data augmentation.

Autoencoders and Variational Autoencoders

An autoencoder contains an encoder that maps x to a latent representation z and a decoder that reconstructs x from z. The reconstruction objective encourages information preservation. A variational autoencoder (VAE) treats z as a random variable and regularizes its distribution toward a prior, commonly a standard normal distribution. Its objective combines reconstruction error with a Kullback-Leibler divergence term.

VAEs provide structured and relatively stable latent spaces, but reconstructions can be overly smooth because likelihood-based objectives average plausible possibilities. They are useful for representation learning, anomaly detection, controlled interpolation, and probabilistic generation.

Generative Adversarial Networks

A GAN consists of a generator G and discriminator D. The generator creates samples intended to resemble real data, while the discriminator distinguishes real from generated samples. In the idealized minimax formulation, the generator and discriminator compete through an adversarial objective.

GANs can generate sharp images and support conditional synthesis, style transfer, super-resolution, and image translation. Training can be unstable because the two networks must remain balanced. Mode collapse occurs when the generator produces limited varieties of outputs. Monitoring diversity, using suitable regularization, and evaluating distributions rather than isolated attractive samples are essential.

Diffusion Models

Diffusion models define a gradual forward process that adds noise to data and learn a reverse process that removes noise. During generation, a model repeatedly estimates the denoising direction, often conditioned on text, class labels, measurements, or another image. Latent diffusion performs the process in a compressed representation, reducing computational cost.

Diffusion models generally offer strong sample quality and diversity, but iterative sampling can be expensive. Conditioning does not guarantee factual correctness. Generated content may contain anatomical, geometric, or semantic errors, and the model can reproduce biases or memorized material. Guidance strength, sampling steps, scheduler choice, resolution, and conditioning quality influence the result.

Evaluation, Reliability, and Experimental Design

A credible experiment begins with a clearly defined population and a split strategy that prevents related samples from appearing in both training and test sets. In medical images, all images from one patient should normally remain in one split. In video, temporal and subject-based separation may be needed. Repeated random splits can provide uncertainty estimates, while external validation measures generalization to another site or acquisition condition.

Evaluation should be multidimensional. Predictive performance, computational cost, latency, memory, robustness, calibration, fairness, and failure modes matter together. Confidence intervals, bootstrap estimates, subgroup analysis, and statistical tests can reveal whether an apparent improvement is meaningful. For generative systems, evaluation may include distributional measures, diversity, condition alignment, human preference, memorization checks, and downstream utility. No single metric captures realism, usefulness, and safety simultaneously.

Interpretability tools such as saliency maps, occlusion tests, feature visualization, counterfactual analysis, and concept-based methods can support diagnosis of model behaviour. They should not be treated as proof of causal reasoning. A heatmap may reflect correlated background information rather than the evidence a domain expert considers valid.

An Illustrative Case Study: Automated Defect Inspection

Background

Consider a fictional electronics manufacturer producing circuit boards. Inspection images are acquired under varying illumination and camera positions. Defects include missing components, solder bridges, scratches, and contamination. Only a small fraction of boards have expert-level pixel annotations, while a larger archive contains image-level accept/reject decisions.

Problem

The company wants a system that flags defective boards, locates the defect for rework, and operates within a short production-cycle time. False negatives are costly because defective boards may reach customers; excessive false positives reduce throughput and create unnecessary manual inspection.

Analysis

A classification-only model would identify suspicious boards but would not explain where the problem occurs. A segmentation model would provide precise regions but would require costly annotations. Image acquisition varies across production lines, creating a domain-shift risk. Simple augmentation may help with brightness variation, but it must not alter the visual geometry of a defect.

Approach

The engineering team first creates a subject- and production-batch-based split. A pretrained CNN is fine-tuned for board-level classification using class-weighted loss. A second-stage detector is trained on the smaller set of localized annotations. Self-supervised masked-image pretraining on unlabelled board images is compared with ordinary supervised transfer learning. Calibration is assessed because the probability threshold controls escalation to human inspection.

For rare defects, precision-recall curves and per-defect recall are emphasized rather than overall accuracy. An uncertainty rule sends low-confidence cases to inspectors. The team tests the system on a later production batch and on images from a second camera line. Visual explanations are reviewed by engineers, but decisions are not based solely on saliency maps.

Outcome

The staged system improves operational usefulness over a single classifier: the first model rapidly screens boards, while the detector supplies locations for likely defects. External-line performance is lower than internal validation, revealing illumination and camera differences. After targeted calibration, additional representative data, and camera standardization, the system becomes suitable for supervised deployment with human oversight.

Learning Points

The case demonstrates that architecture is only one part of an image-AI system. Data splits, annotation quality, threshold selection, external validation, latency, and workflow integration determine practical value. It also illustrates why a generative model should not be used merely because it is technologically fashionable; a discriminative detector is more appropriate when the requirement is reliable localization of known defects.

Challenges, Risks, and Common Mistakes

Deep image models are vulnerable to dataset bias, label noise, shortcut learning, adversarial perturbations, corrupted inputs, and distribution shift. Performance may degrade when sensors, populations, lighting conditions, or operating procedures change. Large models also create substantial training and inference costs, including energy use and hardware dependence.

Common mistakes include allowing augmented versions of one image into different data splits, tuning thresholds on the test set, reporting only average accuracy, ignoring rare classes, using visually attractive generative samples as evidence of correctness, and applying arbitrary transformations that destroy task-relevant information. Another error is assuming that a larger model is always superior. A smaller, calibrated, well-monitored model may be safer and easier to maintain.

Generative systems present additional risks. They may fabricate evidence, reproduce copyrighted or private content, amplify stereotypes, or enable deceptive synthetic media. In high-stakes settings, generated images must be clearly identified, provenance should be recorded where feasible, and human review should be mandatory for consequential decisions.

Professional and Governance Considerations

A production system should have documented intended use, data provenance, model versioning, evaluation protocols, access controls, monitoring, and rollback procedures. Sensitive images require privacy protection, secure storage, and appropriate consent or legal authority. Medical, biometric, financial, and public-sector deployments may be subject to sector-specific regulation and audit requirements.

Governance should address not only model accuracy but also accountability: who approves deployment, who investigates failures, and how affected individuals can challenge an automated decision? Dataset documentation, model cards, risk registers, and incident logs provide practical mechanisms for answering these questions. Human oversight must be meaningful rather than a nominal approval step.

Emerging Directions and Best Practices

Current research increasingly combines vision and language, retrieval with generation, multimodal foundation models, efficient adaptation, neural rendering, 3D and video generation, and interactive segmentation. Diffusion and transformer architectures are being adapted for scientific imaging, remote sensing, robotics, and personalized content creation. At the same time, research is moving toward efficient models, synthetic-data auditing, uncertainty estimation, watermarking, provenance, and evaluation under real-world shift.

A sound development practice is to begin with the application and failure costs rather than the newest architecture. Establish a reproducible baseline, define data governance, inspect labels, and select metrics linked to operational decisions. Use transfer or self-supervised learning when labels are scarce, but verify domain compatibility. Compare models under identical splits and budgets. Evaluate subgroups and external data, calibrate outputs, document limitations, and monitor the deployed system. Generative models should be constrained by conditioning, validated against source information, and used with explicit safeguards against hallucination and misuse.

Chapter Summary

Deep learning transforms image processing by learning hierarchical representations and task mappings from data. CNNs exploit local structure and remain effective under limited or moderate data; vision transformers model long-range relationships and often benefit from large-scale pretraining. Supervised, transfer, self-supervised, and foundation-model learning provide different ways to address annotation and domain challenges. Classification, detection, segmentation, and restoration require distinct outputs, losses, and evaluation measures.

Autoencoders and VAEs learn latent representations and probabilistic reconstructions, GANs generate sharp images through adversarial competition, and diffusion models synthesize images through iterative denoising. Generative quality must be evaluated for fidelity, diversity, conditioning accuracy, and safety rather than visual appeal alone. Reliable systems require careful data splitting, calibration, robustness testing, external validation, governance, privacy protection, and human oversight. Model selection should be guided by the operational problem, error costs, resources, and risks.

Key Terms

Representation learning: Learning useful features directly from data rather than designing them manually.

Convolutional neural network: A neural architecture that uses local, shared convolutional kernels to process spatial data.

Residual connection: A shortcut that adds a block's input to its transformed output to support optimization of deep networks.

Vision transformer: An image model that represents image patches as tokens and processes them with self-attention.

Self-attention: A mechanism that computes interactions among elements of a sequence using queries, keys, and values.

Transfer learning: Adapting knowledge learned from one dataset or task to another task.

Self-supervised learning: Learning from automatically constructed targets derived from unlabelled data.

Foundation model: A broadly pretrained model that can be adapted to multiple downstream tasks.

Data leakage: Unintended use of information from validation or test data during training or model development.

Calibration: Agreement between predicted probabilities and observed event frequencies.

Intersection over union: The area of overlap divided by the area of union between predicted and reference regions.

Semantic segmentation: Assigning a semantic class to each image pixel.

Instance segmentation: Assigning pixels to classes while distinguishing separate object instances.

Autoencoder: An encoder-decoder model trained to reconstruct its input.

Variational autoencoder: A probabilistic autoencoder that regularizes latent variables toward a prior distribution.

Generative adversarial network: A generator-discriminator framework trained through adversarial competition.

Mode collapse: A GAN failure in which generated samples lack diversity.

Diffusion model: A generative model that learns to reverse a progressive noise-addition process.

Domain shift: A change between the training distribution and the deployment distribution.

Model hallucination: Generation of plausible-looking but unsupported or incorrect content.

Review Questions

Short Answer Questions

1. Why do convolutional layers use weight sharing, and what advantage does it provide?

2. Distinguish semantic segmentation from instance segmentation.

3. What is the purpose of calibration in an image classifier?

4. Define mode collapse in a GAN.

5. Why can PSNR alone be inadequate for evaluating image restoration?

Long Answer Questions

1. Explain the architecture and inductive biases of CNNs, and compare them with vision transformers.

2. Describe supervised, transfer, self-supervised, and foundation-model learning, including situations in which each is appropriate.

3. Explain how detection and segmentation systems are trained and evaluated, including relevant losses and metrics.

4. Compare VAEs, GANs, and diffusion models with respect to representation, generation process, quality, diversity, and limitations.

5. Discuss the requirements for reliable deployment of a deep image-processing system in a high-stakes application.

Analytical / Application Questions

1. A medical imaging dataset contains images from 200 patients, with multiple scans per patient. Design a train-validation-test splitting strategy and justify it.

2. A defect detector has high accuracy but misses most rare, costly defects. Recommend evaluation metrics, training changes, and threshold policies.

3. A diffusion-based restoration system produces sharp images but occasionally invents structures absent from the measurement. Analyse the risk and propose an evaluation and governance plan.

22. Multiple Choice Questions with Answers

Question 1

A key reason CNNs are often data-efficient for images is that they encode which inductive bias?

A. Every pixel must interact with every other pixel in the first layer

B. Local connectivity and shared weights

C. Random permutation of image patches

D. A requirement that all classes have equal frequency

Correct Answer: B. Local connectivity and shared weights

Question 2

Which evaluation practice best prevents patient-level leakage in a medical image dataset?

A. Randomly assign individual images to all splits

B. Place all images from the same patient in one split

C. Use only the training accuracy

D. Duplicate rare images across validation and test sets

Correct Answer: B. Place all images from the same patient in one split

Question 3

A detector must identify rare defects where missed detections are very costly. Which evaluation emphasis is most appropriate?

A. Overall accuracy alone

B. Per-class recall and precision-recall analysis

C. Training loss after one epoch

D. Image file size

Correct Answer: B. Per-class recall and precision-recall analysis

Question 4

What is the principal operation that enables a vision transformer to model relationships between distant image patches?

A. Max pooling only

B. Self-attention

C. Histogram equalization

D. Non-maximum suppression

Correct Answer: B. Self-attention

Question 5

Which statement best describes self-supervised learning for images?

A. It requires a manually labelled target for every image

B. It constructs learning signals from the images themselves or their transformations

C. It can only be used after object detection

D. It eliminates the need for downstream evaluation

Correct Answer: B. It constructs learning signals from the images themselves or their transformations

Question 6

In a GAN, mode collapse refers to a situation in which the generator:

A. Produces highly diverse samples from many modes

B. Fails to update because the input image is grayscale

C. Produces limited varieties of samples despite apparently realistic outputs

D. Always reproduces the exact training image

Correct Answer: C. Produces limited varieties of samples despite apparently realistic outputs

Question 7

What is the defining generation procedure of a diffusion model?

A. Learning to reverse a progressive noise-addition process

B. Selecting the most frequent training label

C. Using only a deterministic encoder without a decoder

D. Removing all high-frequency information permanently

Correct Answer: A. Learning to reverse a progressive noise-addition process

Question 8

Why may a restoration model with high PSNR still be unsuitable for a medical application?

A. PSNR measures only the number of model parameters

B. It may produce smooth or visually plausible outputs that hallucinate clinically important structures

C. PSNR cannot be calculated for images

D. High PSNR proves that the model is biased

Correct Answer: B. It may produce smooth or visually plausible outputs that hallucinate clinically important structures

Question 9

Which action is most appropriate when deployment images come from a camera line not represented in training data?

A. Assume internal test performance guarantees success

B. Perform external validation and assess domain shift

C. Remove the validation set

D. Increase the classification threshold without testing

Correct Answer: B. Perform external validation and assess domain shift

Question 10

In the circuit-board case, why was a staged classifier-plus-detector workflow useful?

A. Classification rapidly screened boards while detection supplied defect locations

B. The detector eliminated the need for labelled examples

C. The classifier generated legally verified images

D. The workflow guaranteed zero false positives

Correct Answer: A. Classification rapidly screened boards while detection supplied defect locations