Neural Networks and Deep Learning

Introduction and Context

Neural networks are computational models inspired loosely by the organization of biological nervous systems. In artificial intelligence, they provide a general framework for learning a function from examples. A neural network may approximate a classifier, regressor, ranking function, policy, generative distribution, or representation of complex data. Deep learning refers principally to the use of neural networks with multiple learned layers, allowing hierarchical representations to be formed from raw or minimally processed input.

Neural methods have become central to speech recognition, computer vision, natural-language processing, recommender systems, scientific modelling, robotics, and generative AI. Their success has resulted from the interaction of several factors: large datasets, high-performance parallel hardware, improved optimization algorithms, architectural innovations, and software ecosystems for automatic differentiation.

At postgraduate level, neural networks should not be understood merely as collections of layers. They are parameterized statistical models whose behaviour depends on architecture, objective function, optimization procedure, data distribution, computational resources, and deployment constraints. A technically sound design therefore requires analysis of both predictive performance and the costs, risks, and assumptions underlying the model.

Learning Objectives

By the end of this chapter, a reader should be able to:

1. Explain the structure and mathematical operation of feedforward neural networks.

2. Derive the principal steps of backpropagation and relate them to automatic differentiation.

3. Compare optimization methods and diagnose common training failures.

4. Explain overfitting, regularization, normalization, and generalization.

5. Design and calculate the dimensions of convolutional neural networks.

6. Compare recurrent, gated, attention-based, and transformer sequence models.

7. Estimate data, memory, computation, and deployment requirements.

8. Evaluate neural systems critically in terms of reliability, fairness, privacy, and governance.

1. Neural Computation and Network Architecture

1.1 The artificial neuron

A neuron receives an input vector x = (x1, ..., xd), computes a weighted sum, adds a bias, and applies an activation function:

z = wᵀx + b, y = φ(z).

Here, w is a vector of learnable weights, b is a scalar bias, z is the pre-activation, and φ is the activation function. The bias allows the decision boundary to shift independently of the origin. The neuron is a parametric function; learning changes w and b rather than changing the basic computation.

Common activations include:

• Sigmoid: φ(z) = 1/(1 + exp(-z)). It maps values to (0, 1), but can produce very small derivatives for large positive or negative inputs.
• Hyperbolic tangent: tanh(z), mapping to (-1, 1). It is zero-centred but also saturates.
• ReLU: max(0, z). It is computationally simple and has a constant derivative of 1 for positive inputs, although units can become permanently inactive when their inputs remain negative.
• GELU and SiLU: smooth activations frequently used in modern transformer and vision architectures.

An activation is needed in hidden layers to represent nonlinear functions. Without nonlinear activations, a composition of linear layers is still only one linear transformation and cannot represent hierarchical nonlinear decision functions.

1.2 Layers and the multilayer perceptron

A dense layer maps an input vector to a new vector:

h = φ(Wx + b),

where W contains the weights connecting all input units to all output units. A multilayer perceptron (MLP) composes such transformations:

fθ(x) = fL(... f2(f1(x)) ...),

where θ denotes all weights and biases. The output layer depends on the task. A linear output is typical for unconstrained regression; a sigmoid output is used for binary probability estimation; softmax is used for mutually exclusive multiclass prediction:

p(y = k | x) = exp(zk) / Σj exp(zj).

Softmax probabilities sum to one. For numerical stability, implementations subtract the maximum logit from all logits before exponentiation; this does not change the resulting probabilities.

1.3 Representation learning and capacity

Early layers may detect simple or local patterns, while later layers combine them into task-relevant abstractions. This interpretation is useful but not universal: learned features can be distributed, entangled, and difficult to assign human-readable meanings.

Model capacity is the range and complexity of functions a model can represent. It increases with parameters, depth, width, and architectural flexibility, but parameter count alone does not determine effective capacity. Optimization, data quality, implicit regularization, and the structure of the input all matter. The universal approximation theorem states, under specified conditions, that sufficiently wide networks with suitable nonlinear activations can approximate broad classes of continuous functions on compact domains. It does not say that training is easy, that the required width is practical, or that the resulting function generalizes.

2. Learning Objectives and Loss Functions

Training chooses parameters that minimize an empirical objective. Given training examples (xi, yi), a common objective is

J(θ) = (1/n) Σi L(fθ(xi), yi) + λR(θ),

where L is the data loss, R is a regularization term, λ ≥ 0 controls its strength, and n is the number of training examples. The first term measures fit to observed data; the second expresses a preference for particular parameter values or functions.

For regression with real-valued targets, mean squared error is

MSE = (1/n)Σi(ŷi - yi)².

It is appropriate when squared deviations are meaningful and large errors should be penalized strongly. Mean absolute error is more robust to outliers but has a non-differentiable point at zero, normally handled by a subgradient or a smooth approximation.

For binary classification, binary cross-entropy is

L = -[y log(p) + (1-y)log(1-p)],

where y ∈ {0,1} and p is the predicted probability of class 1. For multiclass classification, cross-entropy is

L = -Σk yk log(pk),

where y is typically a one-hot target and p is the softmax probability vector. Cross-entropy corresponds to maximum likelihood under a categorical model. It evaluates probabilistic quality, not merely classification accuracy.

A distinction is essential: training loss, validation performance, and test performance answer different questions. The training set is used for fitting; the validation set supports model and hyperparameter selection; the test set should be reserved for final unbiased assessment. Repeatedly evaluating and adapting to the test set gradually converts it into another training resource.

3. Backpropagation and Automatic Differentiation

3.1 Gradient-based learning

For differentiable or almost-everywhere differentiable models, parameters are updated using the gradient. In basic gradient descent,

θt+1 = θt - η∇θJ(θt),

where η is the learning rate. The gradient points in the direction of greatest local increase, so subtracting it reduces the objective for a sufficiently small step. In practice, gradients are usually estimated from a mini-batch rather than the entire dataset.

3.2 Chain-rule derivation for a two-layer network

Consider a scalar-output network:

z1 = W1x + b1, h = φ(z1), z2 = W2h + b2, ŷ = z2.

Let the loss be L(ŷ, y). The chain rule gives

∂L/∂W2 = (∂L/∂z2)hᵀ,

∂L/∂b2 = ∂L/∂z2,

∂L/∂W1 = [(W2ᵀ(∂L/∂z2)) ⊙ φ'(z1)]xᵀ,

∂L/∂b1 = (W2ᵀ(∂L/∂z2)) ⊙ φ'(z1),

where ⊙ denotes element-wise multiplication. For mean squared error with one example, L = 1/2(ŷ-y)², so ∂L/∂z2 = ŷ-y. The factor 1/2 is conventionally used because it cancels the derivative coefficient 2; it does not alter the minimizer.

Backpropagation is the efficient application of the chain rule from the output toward the input. It reuses intermediate values from the forward pass, avoiding the cost of independently perturbing every parameter. Automatic differentiation systems generalize this process to computational graphs and calculate exact derivatives up to floating-point arithmetic. Automatic differentiation is distinct from symbolic differentiation and from finite differences.

3.3 Vanishing and exploding gradients

In a deep composition, gradients contain products of derivatives and weight matrices. If typical factors are smaller than one, gradients may shrink; if larger than one, they may grow. Saturating activations, unsuitable initialization, excessive depth, and recurrent time unrolling can intensify these effects.

Practical responses include ReLU-like activations, careful initialization, residual connections, normalization, gated recurrence, gradient clipping, and suitable learning rates. Gradient clipping limits a gradient norm when it exceeds a chosen threshold; it changes the update direction or magnitude and should be treated as an optimization intervention, not as a cure for poor model design.

4. Optimization for Deep Models

4.1 Mini-batch stochastic optimization

A mini-batch gradient estimate uses a subset B of examples:

ĝ = (1/|B|)Σi∈B ∇θL_i.

Smaller batches provide noisier but often cheaper updates; larger batches improve hardware utilization but require more memory and may reduce the number of parameter updates for a fixed number of examples. Batch size should be selected with regard to memory, throughput, convergence behaviour, and generalization.

4.2 Momentum and adaptive methods

Momentum maintains a velocity that smooths successive gradients:

v_t = βv_{t-1} + (1-β)g_t,

θ_{t+1} = θ_t - ηv_t,

where β is commonly between zero and one, g_t is the current gradient, and η is the learning rate. The exact implementation may use an equivalent rescaling convention.

Adam maintains exponential moving averages of gradients and squared gradients, with bias correction during early steps. It often converges rapidly and is useful when parameters have different gradient scales. AdamW separates decoupled weight decay from the adaptive gradient update, making the relationship between regularization and optimization clearer. No optimizer is universally best: stochastic gradient descent with momentum can provide strong generalization in some vision tasks, while adaptive methods may be advantageous for sparse, irregular, or language-model objectives.

4.3 Learning-rate schedules and diagnostics

A learning rate that is too high can cause divergence or oscillation; one that is too low can result in slow progress or apparent stagnation. Warm-up, cosine decay, step decay, one-cycle schedules, and validation-based reduction are common approaches. Diagnostics should include training and validation curves, gradient norms, parameter norms, activation distributions, throughput, and numerical stability. A falling training loss with worsening validation performance is a warning sign for overfitting, distribution shift, or data leakage in the evaluation design.

5. Generalization and Regularization

A network can interpolate its training data yet perform poorly on new data. Generalization depends on the relationship between the data-generating process, sampling procedure, architecture, objective, optimization trajectory, and deployment environment.

5.1 Explicit regularization

L2 regularization adds λ||θ||² to the objective and discourages large parameter values. L1 regularization adds λ||θ||1 and can encourage exact zeros, although its practical effect in deep networks depends on parameterization and optimization. Weight decay is related to L2 penalization but is not identical for all adaptive optimizers; decoupled weight decay applies a direct shrinkage step.

Dropout randomly sets activations to zero during training with a specified probability and scales activations so that expected magnitudes remain consistent. It is disabled or replaced by appropriate inference behaviour at evaluation time. Data augmentation creates plausible variations, such as image crops, noise, or text perturbations, and often improves invariance more directly than parameter penalties.

Early stopping monitors validation performance and stops training when continued fitting no longer improves the selected criterion. It is a form of regularization, but it requires a carefully maintained validation set.

5.2 Normalization and residual learning

Batch normalization normalizes activations using mini-batch statistics during training and running estimates during inference. It can improve optimization but may be problematic for very small batches or non-independent data. Layer normalization normalizes within an individual example and is widely used in sequence and transformer models. Group and instance normalization are useful in particular vision settings.

Residual connections compute a transformation of the form y = F(x) + x when dimensions match, or use a projection when they do not. They create shorter gradient paths and make it easier for a deep block to learn an incremental refinement rather than an entirely new mapping.

6. Convolutional Neural Networks

6.1 Local connectivity and weight sharing

A convolutional layer applies a small kernel across spatial positions. The same weights are reused, reducing parameter count and encoding an inductive bias that nearby pixels or features are related and that useful patterns may occur at multiple locations. Strictly, many deep-learning libraries implement cross-correlation rather than mathematical kernel reversal, but the term convolutional layer is conventional.

For a one-dimensional input of length N, kernel size K, stride S, and padding P on each side, the output length is

O = floor((N + 2P - K)/S) + 1.

For two-dimensional inputs, the formula is applied independently to height and width. If dilation D is used, the effective kernel size is K_eff = D(K-1)+1, and the output formula becomes floor((N+2P-K_eff)/S)+1. These formulas assume integer parameters and the standard sliding-window operation.

The parameter count of a convolution with kernel height Kh, kernel width Kw, Cin input channels, and Cout output channels is

KhKwCinCout + Cout,

including one bias per output channel. It does not depend on the image height and width. Pooling or strided convolution reduces spatial resolution and increases the receptive field of later units, but may discard information.

6.2 Design considerations

A CNN typically alternates convolutions, nonlinearities, normalization, and resolution changes, followed by a task-specific head. Modern designs use residual blocks, depthwise-separable convolutions, dilated convolutions, feature pyramids, and attention modules. Classification, detection, segmentation, and image generation require different output structures and losses.

CNNs encode useful locality and translation-related assumptions, often enabling data-efficient learning relative to an unconstrained dense model. However, they may require additional mechanisms for global relationships, exact spatial reasoning, or highly variable geometric structure.

7. Sequence Models: Recurrence, Attention, and Transformers

7.1 Recurrent neural networks

An RNN processes a sequence one element at a time:

h_t = φ(W_xx x_t + W_hh h_{t-1} + b),

y_t = g(W_yh h_t + c).

The hidden state h_t summarizes prior information. Parameter sharing across time makes RNNs compact, but long sequences produce repeated Jacobian multiplication and therefore difficult credit assignment. Long short-term memory (LSTM) and gated recurrent units (GRU) use gates to regulate writing, retaining, and exposing information. They improve long-range modelling but remain sequential during training and inference.

7.2 Attention

Attention allows a representation to select information from a set of representations. Scaled dot-product attention is

Attention(Q,K,V) = softmax(QKᵀ / √d_k)V,

where Q, K, and V are query, key, and value matrices, and d_k is the key dimension. Scaling reduces the tendency of dot products to become large as dimension increases, which would make softmax gradients very small. A mask may prevent a position from attending to disallowed positions, such as future tokens in autoregressive generation.

Multi-head attention performs several attention operations with learned projections, allowing different heads to model different relationships. Attention has direct paths between distant positions, but standard self-attention has O(T²) pairwise time and memory complexity for sequence length T. This quadratic cost is a major constraint for long contexts.

7.3 Transformer architecture

A transformer replaces recurrence with stacked attention and feedforward blocks. A typical block uses residual connections, normalization, multi-head self-attention, and a position-wise feedforward network. Because attention alone does not identify order, positional information must be supplied through learned positional embeddings, sinusoidal encodings, relative position methods, or rotary position representations.

Encoder-only transformers are common for representation learning and classification; decoder-only transformers are common for autoregressive generation; encoder-decoder transformers are useful for sequence-to-sequence tasks such as translation. Pretraining on broad data followed by fine-tuning or instruction adaptation can transfer general representations to downstream tasks.

Transformers parallelize training over sequence positions more effectively than RNNs, but inference for autoregressive generation remains sequential across newly generated tokens. Key-value caching reduces repeated computation, while increasing memory usage with context length and layer width.

8. Computational and Data Requirements

Neural-network feasibility is a systems question. Important resources include dataset volume and diversity, data-label quality, accelerator memory, arithmetic throughput, storage bandwidth, network communication, energy, and engineering time.

For a dense layer with input width d_in and output width d_out, the parameter count is d_in d_out + d_out. A forward pass requires approximately d_in d_out multiply-accumulate operations per example, with the exact operation count depending on whether a multiply-add is counted as one or two floating-point operations. For a convolution, the approximate multiply count per output is KhKwCin, and this is multiplied by the number of output spatial positions and Cout. These estimates exclude some operations and should be treated as planning approximations.

Memory includes parameters, gradients, optimizer states, activations saved for backpropagation, input batches, and temporary workspaces. Mixed-precision training can reduce memory and increase throughput, but it requires loss scaling or equivalent numerical safeguards in some regimes. Gradient accumulation simulates a larger effective batch by summing gradients across several smaller micro-batches, but it does not reproduce every effect of a genuinely large batch, especially batch-statistics behaviour.

Data requirements depend on task complexity, noise, class imbalance, distribution breadth, and transfer learning. More data is not automatically beneficial if labels are systematically biased or if training and deployment distributions differ. Data pipelines should document provenance, consent and licensing, sampling, annotation policy, deduplication, leakage controls, and protected-group coverage.

Deployment may require pruning, quantization, distillation, compilation, caching, batching, or model parallelism. A model that is accurate in a laboratory but exceeds latency, memory, privacy, or energy limits is not an operationally successful model.

9. Illustrative Case: Defect Detection in Industrial Manufacturing

Background

Consider a fictional manufacturer using cameras to identify surface defects on metal components. The production line operates continuously, and false negatives are costly because defective parts may reach customers. False positives also matter because unnecessary manual inspections reduce throughput.

Problem and analysis

The initial dataset contains 80,000 labelled images from two cameras and three product types. Inspection of the data reveals that most defective images were captured during one shift, while camera background and lighting also differ by shift. A random image split would therefore risk leakage: the model might learn lighting or camera signatures rather than defect characteristics.

The team creates a grouped split by production batch and evaluates separately by product type and lighting condition. It reports precision, recall, false-negative rate, calibration, and inspection workload rather than accuracy alone. Since defects are rare, accuracy could remain high even for a useless classifier.

Approach

A pretrained CNN is adapted with a classification head and moderate geometric and photometric augmentation. The training objective uses weighted cross-entropy initially, but the team also evaluates threshold selection against the economic cost of false negatives and false positives. Augmentations are validated with manufacturing engineers so that they do not create physically impossible defects. Batch normalization statistics are handled carefully because production batches differ from training batches.

The team compares the CNN with a transformer-based vision model. The transformer performs better when sufficient data and compute are available, but the CNN provides lower latency on the edge device. Quantization is tested after calibration, and performance is measured on images from a later month, not merely on the held-out historical set.

Outcome

The selected CNN reduces false negatives substantially relative to the existing rule-based system while meeting the line’s latency limit. A human-review queue is retained for uncertain predictions. Monitoring detects a performance decline after a camera replacement, triggering recalibration and additional labelled data collection.

Learning points

The case illustrates that architecture selection is only one part of an AI system. Grouped evaluation, domain-aware augmentation, threshold economics, calibration, drift monitoring, and deployment constraints are equally important. It also demonstrates why a high benchmark score can be misleading when the split does not represent the intended operational environment.

10. Limitations, Ethics, and Governance

Neural models can encode historical discrimination, amplify sampling bias, and behave unpredictably under distribution shift or adversarial manipulation. Their confidence scores may be poorly calibrated, and explanations such as saliency maps should not automatically be treated as causal evidence. A model can identify a correlation without identifying the mechanism responsible for an outcome.

Privacy risks arise when training data contain personal, sensitive, or identifiable information. Access control, minimization, de-identification where appropriate, secure training infrastructure, retention limits, and documented lawful bases are essential. Differential privacy can reduce information leakage by adding carefully calibrated noise, but it generally introduces a utility trade-off and must be evaluated quantitatively.

Governance should include intended-use documentation, data and model cards, risk assessment, reproducible experiments, version control, audit logs, human oversight, incident response, and post-deployment monitoring. High-impact applications may require sector-specific regulation, impact assessments, transparency obligations, and mechanisms for appeal. Responsible design is not achieved by adding an explanation after deployment; it must be integrated into data collection, objective selection, evaluation, and operational control.

11. Contemporary Directions and Best Practices

Current research includes efficient and sparse attention, mixture-of-experts models, multimodal foundation models, retrieval augmentation, parameter-efficient adaptation, neural operators for scientific computing, diffusion-based generative models, continual learning, and hardware-aware neural architecture search. These directions address capability, efficiency, or adaptation, but each introduces new failure modes. For example, mixture-of-experts systems can reduce active computation while increasing routing and memory complexity; continual learning can cause catastrophic forgetting; and retrieval augmentation can propagate errors from retrieved documents.

Best practice for postgraduate engineering projects includes:

• Define the decision, users, constraints, and failure costs before choosing an architecture.
• Establish simple baselines and conduct ablation studies.
• Use leakage-resistant train, validation, and test partitions.
• Record preprocessing, random seeds, software versions, hardware, and hyperparameters.
• Monitor class-conditional performance, calibration, robustness, and subgroup outcomes.
• Match model size and precision to latency, memory, energy, and privacy requirements.
• Use pretrained models carefully, checking licence, provenance, domain mismatch, and hidden biases.
• Test failure cases manually and quantitatively, including distribution shifts likely in deployment.
• Treat uncertainty and human review as system-design components rather than cosmetic additions.

Chapter Summary

Neural networks are parameterized nonlinear function approximators trained by minimizing a data-dependent objective. Their practical behaviour emerges from architecture, activation functions, loss, optimization, regularization, data, and deployment conditions. Backpropagation efficiently applies the chain rule, while optimizers such as momentum and Adam determine how gradients become parameter updates. Generalization is supported by validation discipline, augmentation, weight control, dropout, normalization, residual connections, and early stopping.

CNNs exploit locality and weight sharing for spatial data. RNNs and gated units model sequences through recurrent state, while attention provides direct content-dependent interactions. Transformers use attention, positional information, residual pathways, and feedforward blocks to achieve powerful parallel sequence modelling, at the cost of substantial memory and quadratic attention complexity in sequence length. Effective engineering requires estimating parameters, operations, memory, data quality, latency, and energy rather than reporting accuracy alone. Ethical and governance concerns—including bias, privacy, explainability, robustness, and monitoring—must be addressed throughout the model lifecycle.

Key Terms

Artificial neuron: A weighted-sum computational unit followed by an activation function.

Activation function: A nonlinear function applied within a neural layer.

Backpropagation: Efficient reverse application of the chain rule to compute parameter gradients.

Batch normalization: Normalization based primarily on statistics of a training mini-batch.

Convolution: A sliding local operation with shared weights, commonly used for spatial data.

Cross-entropy: A probabilistic loss commonly used for classification.

Deep learning: Learning with neural networks containing multiple representational layers.

Dropout: A training regularization method that randomly masks activations.

Embedding: A learned vector representation of an item, token, or category.

Generalization: Performance on previously unseen examples from the target distribution.

Gradient clipping: Limiting gradient magnitude to improve numerical or training stability.

Attention: A mechanism that forms weighted combinations of value representations using queries and keys.

Overfitting: Fitting training-specific patterns that do not transfer well to new data.

Residual connection: A shortcut that adds a block input to its transformed output.

Regularization: A method or constraint intended to improve generalization or control complexity.

Softmax: A transformation of logits into nonnegative probabilities summing to one.

Transformer: An architecture based primarily on attention, feedforward blocks, and positional information.

Weight decay: Direct shrinkage of parameters during optimization, often used as regularization.

Review Questions

Short Answer Questions

1. Why are nonlinear activation functions necessary in multilayer networks?

2. Distinguish training, validation, and test data.

3. What problem is addressed by gradient clipping?

4. Why does a transformer require positional information?

5. State two advantages and one limitation of convolutional layers.

Long Answer Questions

1. Derive the gradients for the weights in a two-layer scalar-output network and explain the role of the chain rule.

2. Compare SGD with momentum, Adam, and AdamW in terms of update behaviour and practical use.

3. Explain the principal causes of overfitting and evaluate several regularization strategies.

4. Compare RNNs, LSTMs, and transformers for long-sequence modelling.

5. Discuss the computational, data, and governance requirements for deploying a neural model in a high-impact application.

Analytical / Application Questions

1. A convolution receives an input of size 64 × 64 with kernel size 3, stride 2, and padding 1. Calculate the spatial output dimensions and explain each term in the calculation.

2. A binary-defect dataset has 99% non-defective images. Explain why accuracy is inadequate as the sole metric and propose an evaluation and threshold-selection strategy.

3. Design an experiment to determine whether a vision model is learning defects or camera-specific artefacts. Specify the data split, controls, metrics, and deployment test.

Multiple Choice Questions with Answers

Question 1

What is the principal purpose of a nonlinear activation in a hidden layer?

A. To make all parameters independent

B. To allow compositions of layers to represent nonlinear functions

C. To guarantee zero training error

D. To eliminate the need for a loss function

Correct Answer: B. To allow compositions of layers to represent nonlinear functions

Question 2

For binary classification, which loss is defined as -[y log(p) + (1-y) log(1-p)]?

A. Mean squared error

B. Hinge loss

C. Binary cross-entropy

D. L1 regularization

Correct Answer: C. Binary cross-entropy

Question 3

Backpropagation primarily computes neural-network gradients by using which mathematical principle?

A. The chain rule

B. Bayes’ theorem

C. The central limit theorem

D. Kernel density estimation

Correct Answer: A. The chain rule

Question 4

A convolution uses kernel size 3, stride 2, padding 1, and an input dimension of 32. What is the output dimension in that direction?

A. 15

B. 16

C. 17

D. 18

Correct Answer: B. 16

Question 5

Which property of a convolution reduces parameters compared with a fully connected spatial operation?

A. Random masking

B. Weight sharing across spatial positions

C. Removal of all nonlinearities

D. Using a separate kernel for every pixel

Correct Answer: B. Weight sharing across spatial positions

Question 6

What is the main computational concern of standard self-attention for sequence length T?

A. It has quadratic pairwise complexity in T

B. It cannot use matrix multiplication

C. It always requires recurrent processing

D. It has no trainable parameters

Correct Answer: A. It has quadratic pairwise complexity in T

Question 7

Why is the factor √d_k used in scaled dot-product attention?

A. To convert probabilities into logits

B. To reduce excessively large dot products as key dimension grows

C. To remove positional information

D. To force all attention weights to be equal

Correct Answer: B. To reduce excessively large dot products as key dimension grows

Question 8

Which split should primarily be used for selecting hyperparameters and monitoring generalization during development?

A. Training split

B. Validation split

C. Final test split only

D. Unlabelled deployment split

Correct Answer: B. Validation split

Question 9

Which method randomly masks activations during training as a regularization technique?

A. Dropout

B. Softmax

C. Pooling

D. Weight tying

Correct Answer: A. Dropout

Question 10

In a highly imbalanced defect-detection problem, why can accuracy be misleading?

A. Accuracy cannot be calculated for classifiers

B. A model predicting the majority class can obtain high accuracy while missing defects

C. Accuracy is always lower than recall

D. Accuracy measures only computational cost

Correct Answer: B. A model predicting the majority class can obtain high accuracy while missing defects