Mathematical and Computational Foundations

Introduction

Deep learning and natural language processing (NLP) are often presented as applications of neural-network software. In practice, reliable models depend on mathematical reasoning, statistical judgment, computational discipline, and reproducible experimentation. Vectors describe observations and features; matrices represent datasets and transformations; probability models uncertainty; derivatives quantify change; loss functions define learning objectives; and optimization algorithms adjust model parameters.

This chapter develops the foundations required to understand and implement these ideas using Python and tensor frameworks such as NumPy and PyTorch. The emphasis is practical: mathematical notation is connected to data preparation, model training, evaluation, and debugging.

Learning Objectives

By the end of this chapter, you should be able to:

1. Represent data using scalars, vectors, matrices, and tensors.
2. Perform and interpret common vector and matrix operations.
3. Explain probability distributions, conditional probability, expectation, variance, and covariance.
4. Calculate descriptive statistics and distinguish training, validation, and test data.
5. Explain derivatives, partial derivatives, gradients, and the chain rule.
6. Construct and interpret common loss functions.
7. Apply gradient descent and identify important hyperparameters.
8. Use Python notebooks and tensor libraries for reproducible experiments.
9. Diagnose numerical, statistical, and implementation problems in model training.

1. Mathematical Objects for Machine Learning

1.1 Scalars, vectors, matrices, and tensors

A scalar is a single number, such as a learning rate or a sentence length. A vector is an ordered list of numbers. A vector containing n elements is written as x ∈ ℝⁿ. A matrix is a rectangular arrangement of numbers with rows and columns. A tensor generalizes these structures to multiple dimensions.

For example, a document represented by three numerical features may be written as:

The symbol T denotes transpose. A collection of m examples, each with n features, can be represented by a matrix X ∈ ℝ^m×n. In NLP, a tensor may represent a batch of sequences, where dimensions correspond to batch size, sequence length, and embedding size.

Shape is not merely a programming detail. A mismatch between tensor dimensions can indicate that a model is mathematically invalid or that data have been arranged incorrectly.

1.2 Vector operations

For vectors a and b of equal length, addition is performed element by element:

Scalar multiplication changes every component:

The dot product is:

It produces a scalar and is central to linear models and neural-network layers. The Euclidean norm measures vector length:

The cosine similarity compares direction rather than magnitude:

This formula assumes neither vector is zero. It is widely used to compare word or document embeddings. A cosine similarity near 1 indicates similar direction, 0 indicates orthogonality, and −1 indicates opposite direction.

Worked example. Let a = (2,1) and b = (1,3). Their dot product is 2×1 + 1×3 = 5. Their norms are √5 and √10, so their cosine similarity is 5/√50 ≈ 0.707.

1.3 Matrices and linear transformations

For A ∈ ℝ^m×n and x ∈ ℝⁿ, the product Ax is defined and produces a vector in ℝ^m. The element in row i is:

A fully connected neural layer commonly computes:

where W is a weight matrix, x is an input vector, b is a bias vector, and z is the pre-activation output. If W has shape (p×n), then x has shape (n) and b has shape (p).

Matrix multiplication is associative but generally not commutative: (AB)C = A(BC) when dimensions permit, but AB ≠ BA in general. This distinction matters when implementing transformations.

The transpose satisfies (AB)ᵀ = BᵀAᵀ. A matrix is symmetric when A = Aᵀ. The identity matrix I leaves a compatible vector unchanged: Ix = x.

1.4 Tensors and broadcasting

Tensor frameworks support elementwise operations, matrix multiplication, automatic differentiation, and GPU execution. Broadcasting permits operations between compatible shapes, such as adding one bias vector to every row of a batch. Broadcasting is convenient but can conceal errors; programmers should inspect shapes explicitly rather than relying on accidental compatibility.

A useful practice is to annotate shapes in comments or documentation. For example:

# X: (batch_size, features)
# W: (features, hidden_units)
# b: (hidden_units,)
Z = X @ W + b

2. Probability and Statistics

2.1 Probability concepts

A random variable represents an uncertain quantity. A discrete random variable takes countable values, while a continuous random variable takes values on an interval. A probability mass function assigns probabilities to discrete outcomes; a probability density function describes continuous distributions. For a valid discrete distribution, probabilities are non-negative and sum to one.

The conditional probability of event A given event B, assuming P(B)>0, is:

Bayes’ theorem follows from the definition of conditional probability:

In classification, P(class|features) is a posterior probability. The prior P(class) reflects prevalence before observing features, while P(features|class) is the likelihood.

Two events are independent when P(A∩B)=P(A)P(B). Independence is an assumption, not a default fact. Words in language, for example, are strongly dependent on context.

2.2 Expectation, variance, and covariance

For a discrete random variable X, expectation is:

For a continuous variable, the corresponding expression is E[X]=∫x f(x)dx, where f(x) is the density and the integral covers the variable’s domain.

Variance measures spread around the mean μ=E[X]:

The standard deviation is σ=√Var(X), expressed in the same units as X. For two variables, covariance is:

Positive covariance indicates that variables tend to increase together; covariance depends on measurement units. Correlation standardizes covariance:

provided both standard deviations are nonzero. Correlation lies between −1 and 1 but does not establish causation.

2.3 Sample statistics

For observations x₁,…,x_n, the sample mean is:

The commonly used unbiased sample variance is:

The denominator n−1 corrects bias in estimating population variance when the mean is estimated from the same sample. For a complete population whose mean is known, division by n is appropriate.

Standardization, often called z-scoring, is:

It is useful when features have different scales. In a machine-learning pipeline, x̄ and s must be calculated on the training set only, then applied unchanged to validation and test data. Otherwise, information from evaluation data leaks into training.

2.4 Common distributions

The Bernoulli distribution models one binary trial with success probability p. If X∈{0,1}, then P(X=1)=p and E[X]=p. A binomial distribution counts successes in n independent Bernoulli trials.

The categorical distribution models one outcome among multiple classes. If a model outputs logits z₁,…,z_K, the softmax function converts them into probabilities:

The outputs are non-negative and sum to one. In practice, numerically stable software subtracts the largest logit before exponentiation; this does not change the probabilities because the same factor cancels from numerator and denominator.

3. Derivatives and Gradients

3.1 Rate of change

A derivative measures the instantaneous rate of change of a function. For a scalar function f(x), the derivative at x is:

The derivative exists when this limit exists. For example, if f(x)=x², then f′(x)=2x. At x=3, the slope is 6.

The derivative of a constant is zero. Important rules include:

3.2 Partial derivatives and gradients

A function of several variables has a partial derivative with respect to each variable while holding the others constant. The gradient collects these derivatives:

The gradient points in the direction of steepest increase. Therefore, negative gradient points locally toward steepest decrease. This geometric interpretation explains gradient descent.

For f(w)=½wᵀw, where w is a vector, the gradient is ∇f(w)=w. The factor ½ is chosen so that the derivative of the squared norm has a simple form.

3.3 Chain rule

Neural networks are compositions of functions. If y=f(g(x)), then:

For multiple variables, the same principle propagates derivatives through each operation. Backpropagation is an efficient application of the chain rule that computes gradients of a loss with respect to many parameters.

Consider y=wx, L=½(y−t)², where t is a target. First, ∂L/∂y=y−t. Since ∂y/∂w=x, the chain rule gives:

This small example captures the logic used in large neural networks.

4. Loss Functions and Learning Objectives

A loss function measures disagreement between predictions and targets. Training typically minimizes the average loss over a dataset:

Here, θ denotes model parameters, f_θ is the model, x_i is an input, y_i is a target, and m is the number of examples. The loss should match the task and target representation.

4.1 Mean squared error

For regression with scalar predictions, mean squared error (MSE) is:

It penalizes large errors more strongly than small errors and is differentiable everywhere. Its units are the square of the target units. MSE is sensitive to outliers.

4.2 Mean absolute error

Mean absolute error is:

It is less sensitive to outliers than MSE. The absolute-value function is not differentiable at zero, although optimization libraries use a valid subgradient or a closely related implementation.

4.3 Binary cross-entropy

For a binary target y∈{0,1} and predicted probability p∈(0,1), binary cross-entropy is:

The loss is low when the model assigns high probability to the correct class. In software, a numerically stable combined sigmoid-and-loss function is preferred over manually computing logarithms of probabilities.

4.4 Multiclass cross-entropy

For one-hot target vector y and predicted class probabilities p, categorical cross-entropy is:

If exactly one class is correct, this reduces to −ln(p_correct). Cross-entropy should not be paired incorrectly with already transformed outputs; for example, many frameworks expect raw logits and internally apply a stable log-softmax operation.

5. Gradient-Based Optimization

5.1 Gradient descent

Given objective J(θ), basic gradient descent updates parameters as:

where η>0 is the learning rate and t is the iteration index. The learning rate has no universal best value. If it is too large, training may diverge; if too small, training may be impractically slow.

Worked example. Let J(w)=½(w−4)², so J′(w)=w−4. Starting at w₀=0 with η=0.25:

w₁=0−0.25(−4)=1.

The next gradient is 1−4=−3, giving w₂=1−0.25(−3)=1.75. The value moves toward the minimum at w=4, although it has not reached it after two iterations.

5.2 Batch, stochastic, and mini-batch methods

Batch gradient descent uses all training examples for each update. Stochastic gradient descent uses one example, producing noisy but frequent updates. Mini-batch training uses a subset and is the standard approach for neural networks because it balances computational efficiency and gradient stability.

The gradient estimated from a mini-batch is not identical to the full-dataset gradient. Random shuffling and sufficiently representative batches help reduce systematic bias.

5.3 Momentum and adaptive methods

Momentum maintains a moving direction that can reduce oscillation:

where 0≤β<1. Adaptive methods such as Adam maintain moving estimates of first and second gradient moments. They can be effective defaults, but they do not eliminate the need for learning-rate selection, validation, and monitoring.

5.4 Regularization and generalization

A model may minimize training loss while performing poorly on unseen data. This is overfitting. Regularization constrains complexity. L2 regularization adds a squared-parameter penalty:

where λ≥0 controls penalty strength. Some conventions use λ/2||θ||₂²; the convention changes the gradient coefficient but not the idea. Other controls include early stopping, dropout, data augmentation, and reducing model capacity.

6. Computational Foundations with Python

6.1 Reproducible notebooks

A reproducible notebook should record the data source, preprocessing steps, package versions, random seeds, model configuration, evaluation metrics, and results. Code cells should be executable in order. Hidden state, manually edited outputs, and undocumented data changes make experiments difficult to audit.

A minimal NumPy example is:

import numpy as np

rng = np.random.default_rng(7)
X = rng.normal(size=(4, 3))
w = np.array([0.5, -0.2, 0.1])
y = X @ w
print(X.shape, w.shape, y.shape)

The seed makes the generated sequence reproducible on the same software conditions, although exact results can still vary across hardware and library versions.

6.2 Tensor operations and automatic differentiation

A PyTorch example illustrates differentiation:

import torch

w = torch.tensor(0.0, requires_grad=True)
target = torch.tensor(4.0)
loss = 0.5 * (w - target) ** 2
loss.backward()
print(w.grad)       # derivative of loss with respect to w

Calling backward() applies the chain rule and stores the derivative in w.grad. In a training loop, parameter updates are normally performed inside a context that disables gradient tracking, and gradients are cleared before the next batch. Failure to clear gradients can cause accumulation across iterations.

6.3 Numerical stability

Floating-point numbers have finite precision. Common problems include overflow in exponentials, underflow in probabilities, division by zero, and loss of precision when subtracting nearly equal values. Stable library functions should be preferred. For softmax, subtracting the maximum logit is mathematically safe:

for any constant c, especially c=max_j z_j.

NaN or infinite values should be treated as diagnostic evidence. Check input ranges, missing values, learning rates, activation outputs, and loss calculations rather than simply replacing the values.

7. A Practical Training Workflow

A sound workflow follows these stages:

1. Define the prediction task and target representation.
2. Inspect and clean the data.
3. Split data into training, validation, and test sets.
4. Fit preprocessing operations on training data only.
5. Establish a simple baseline.
6. Build a model with explicit input and output shapes.
7. Select a task-appropriate loss and optimizer.
8. Monitor training and validation performance.
9. Tune hyperparameters using validation data.
10. Evaluate once on the held-out test set.
11. Record configuration, code, metrics, and limitations.

For NLP, splitting must also consider duplicate documents, authors, time, and near-identical text. Random splitting can produce overly optimistic results when information from the same source appears in multiple partitions.

8. Illustrative Case: Classifying Customer Support Messages

Background

A fictional online retailer receives thousands of support messages labelled as “delivery,” “payment,” or “product.” The retailer wants an NLP classifier to route messages automatically.

Problem

The dataset contains class imbalance: delivery messages are common, while payment disputes are rare. Some messages contain personal information. The team must obtain reliable performance without exposing customer data.

Analysis

The team first counts labels, inspects missing text, removes duplicated messages, and creates a stratified training, validation, and test split. Text is tokenized and represented using learned embeddings. The output layer has three logits, one per class. Cross-entropy is selected because the task is single-label multiclass classification.

Accuracy alone is inadequate. If 80% of messages concern delivery, always predicting delivery gives 80% accuracy but fails on payment disputes. The team therefore reports precision, recall, and F1 score by class. For a class, precision and recall are:

where TP, FP, and FN are true positives, false positives, and false negatives. When precision and recall are both nonzero, the F1 score is:

Approach and solution

The team compares a frequency-based baseline, a linear classifier using bag-of-words features, and a small neural model. Training features are normalized or constructed using training data only. The neural model uses mini-batches, a fixed random seed, early stopping based on validation loss, and gradient clipping if unstable gradients occur.

Because payment disputes may have greater business and customer impact, the team examines class-specific recall and considers class-weighted loss. The weights are documented rather than chosen solely to maximize a single score.

Outcome

The neural model improves payment recall, but manual review remains necessary for messages involving refunds or legal claims. The final system logs model version, preprocessing version, confidence score, and routing decision. It does not automatically reject claims.

Learning points

The case demonstrates that mathematical optimization is only one part of professional NLP. Data splitting, metric selection, privacy protection, error analysis, reproducibility, and human oversight are equally important.

9. Limitations, Ethics, and Governance

Mathematical models are approximations. A low average loss does not guarantee fairness across language varieties, demographic groups, or rare cases. Training data may contain historical bias, personal information, or labels produced inconsistently by humans.

Professionals should minimize data collection, remove unnecessary identifiers, restrict access, encrypt sensitive data, and document retention practices. NLP systems should be tested for discriminatory error patterns and calibrated where probabilities are used for decisions. High-impact decisions require human review, escalation procedures, and an audit trail.

Reproducibility also has an ethical dimension. Undocumented preprocessing, selective reporting, or repeated test-set tuning can make results misleading. A responsible report states data limitations, uncertainty, excluded cases, computational resources, and known failure modes.

10. Best Practices

• Check tensor shapes before debugging numerical results.
• Use stable library implementations of softmax and cross-entropy.
• Fit scalers, token vocabularies, and other learned preprocessing only on training data.
• Use validation data for model selection and preserve the test set for final evaluation.
• Compare against a simple baseline.
• Monitor both loss and task-relevant metrics.
• Set seeds, record versions, and save configurations.
• Inspect examples of false positives and false negatives.
• Treat class imbalance and data leakage as design problems, not merely coding issues.
• Report limitations and maintain human oversight for consequential applications.

Chapter Summary

Vectors, matrices, and tensors provide the language for representing data and neural-network computations. Dot products, norms, cosine similarity, and matrix multiplication support feature comparison and linear transformations. Probability describes uncertainty, while expectation, variance, covariance, and correlation support statistical reasoning. Careful sampling and prevention of data leakage are essential for trustworthy evaluation.

Derivatives and gradients quantify how a loss changes with model parameters. The chain rule enables backpropagation, and gradient-based optimizers update parameters toward lower loss. MSE and MAE are common regression losses; binary and multiclass cross-entropy are suited to probabilistic classification. Learning rate, batch size, regularization, and numerical stability strongly influence training.

Python notebooks and tensor frameworks make these ideas executable, but reproducibility requires documented data, code, versions, seeds, and evaluation procedures. A professional deep-learning workflow combines mathematical correctness with statistical validity, ethical governance, error analysis, and appropriate human oversight.

Key Terms

Scalar: A single numerical value.

Vector: An ordered one-dimensional collection of values.

Matrix: A rectangular two-dimensional array.

Tensor: A multidimensional numerical array.

Dot product: The sum of pairwise products of two equal-length vectors.

Norm: A measure of vector magnitude.

Cosine similarity: Similarity based on the angle between two nonzero vectors.

Random variable: A variable whose value is determined by an uncertain outcome.

Expectation: The probability-weighted average of a random variable.

Variance: The expected squared deviation from the mean.

Covariance: A measure of joint variation between two variables.

Gradient: A vector of partial derivatives.

Chain rule: A rule for differentiating composed functions.

Loss function: A numerical measure of prediction error.

Cross-entropy: A loss measuring disagreement between target and predicted probability distributions.

Gradient descent: An iterative method that moves parameters opposite the gradient.

Regularization: A method for discouraging excessive model complexity.

Data leakage: Use of information during training that would not be available at prediction time.

Backpropagation: Efficient gradient computation through a computational graph.

Reproducibility: The ability to repeat an experiment and obtain consistent results under documented conditions.

Review Questions

Short Answer Questions

1. What is the difference between a vector and a matrix?
2. Why must preprocessing statistics generally be calculated using training data only?
3. What does a gradient represent geometrically?
4. When is cross-entropy preferable to mean squared error?
5. What is the purpose of a validation set?

Long Answer Questions

1. Explain how matrix multiplication is used in a fully connected neural-network layer, including compatible tensor shapes.
2. Compare MSE, MAE, binary cross-entropy, and multiclass cross-entropy in terms of task suitability and behavior.
3. Explain backpropagation using the chain rule and a simple computational example.
4. Describe a reproducible notebook workflow for training and evaluating an NLP classifier.
5. Discuss how class imbalance, data leakage, and metric selection can affect conclusions about model quality.

Analytical / Application Questions

1. A binary classifier achieves 92% accuracy on a dataset in which 90% of examples belong to one class. Design an evaluation strategy that determines whether the classifier is useful.
2. A training loss becomes NaN after several iterations. Propose a systematic diagnostic process involving data, numerical operations, gradients, and optimizer settings.
3. Design an NLP experiment for routing customer messages into three categories. Specify the data split, representation, loss function, metrics, reproducibility controls, and ethical safeguards.

Multiple Choice Questions with Answers

Question 1

Which operation produces a scalar when applied to two equal-length vectors?

A. Elementwise addition

B. Dot product

C. Scalar multiplication

D. Concatenation

Correct Answer: B. Dot product

Question 2

If X has shape (32, 10) and W has shape (10, 4), what is the shape of X @ W?

A. (10, 10)

B. (32, 10)

C. (32, 4)

D. (4, 4)

Correct Answer: C. (32, 4)

Question 3

Why is the training-set mean used when standardizing validation data?

A. To increase the validation sample size

B. To prevent information leakage from validation into training

C. To guarantee zero validation error

D. To remove the need for a test set

Correct Answer: B. To prevent information leakage from validation into training

Question 4

For J(w) = ½(w−4)², what is the gradient at w = 1?

A. −3

B. −1.5

C. 3

D. 5

Correct Answer: A. −3

Question 5

Which loss is most directly appropriate for a single-label multiclass classifier that outputs class probabilities?

A. Mean absolute error

B. Categorical cross-entropy

C. Euclidean norm

D. Sample variance

Correct Answer: B. Categorical cross-entropy

Question 6

In gradient descent, increasing the learning rate excessively can cause what problem?

A. The model always reaches the exact minimum immediately

B. The parameters may overshoot and training may diverge

C. The dataset becomes balanced

D. Gradients become mathematically unnecessary

Correct Answer: B. The parameters may overshoot and training may diverge

Question 7

What does the softmax function produce from a vector of logits?

A. Probabilities that sum to one

B. Binary labels only

C. Unscaled negative values

D. A single variance value

Correct Answer: A. Probabilities that sum to one

Question 8

Which measure is generally less sensitive to outliers than MSE?

A. MAE

B. Softmax

C. Covariance

D. Cross-entropy in every situation

Correct Answer: A. MAE

Question 9

What is the primary purpose of backpropagation?

A. To split data into training and test sets

B. To compute parameter gradients efficiently

C. To remove all model parameters

D. To convert text directly into labels without training

Correct Answer: B. To compute parameter gradients efficiently

Question 10

If a classifier has many false negatives for an important class, which metric should receive particular attention?

A. Recall

B. Parameter count only

C. Training-batch size

D. Input tensor rank

Correct Answer: A. Recall