Mathematical, Statistical and Computational Foundations

Introduction

Artificial intelligence in electrical engineering is not primarily a collection of software libraries. It is an applied mathematical discipline. A model that estimates motor temperature, detects faults in a transformer, forecasts photovoltaic output, or classifies power-quality disturbances depends on representations of signals, uncertainty, optimization, and computation. These foundations determine whether an AI system is physically meaningful, statistically valid, computationally feasible, and reproducible.

This chapter develops the principal foundations required for later work with machine learning and intelligent electrical systems. Linear algebra supplies the language of vectors, matrices, transformations, and state spaces. Probability and statistics provide methods for describing uncertainty, estimating unknown quantities, and evaluating evidence. Calculus explains sensitivity and optimization. Numerical methods make theoretical models computable. Python-based scientific computing connects these ideas to engineering practice.

Learning Objectives

By the end of this chapter, a student should be able to:

1. Represent electrical-engineering data using vectors, matrices, and tensors.
2. Apply norms, inner products, matrix operations, eigenvalue concepts, and singular-value decomposition.
3. Distinguish probability, random variables, distributions, expectation, variance, covariance, and conditional probability.
4. Use descriptive statistics, estimation, confidence intervals, and hypothesis tests appropriately.
5. Explain derivatives, gradients, the chain rule, and gradient-based optimization.
6. Select suitable numerical methods and recognize their stability and conditioning limitations.
7. Implement basic scientific computations in Python using NumPy, SciPy, and related tools.
8. Design reproducible experiments and assess computational complexity.

1. Mathematical Representations of Engineering Data

1.1 Scalars, vectors, matrices, and tensors

A scalar is a single numerical quantity, such as voltage magnitude or ambient temperature. A vector is an ordered list of values. For example, measurements from three sensors may be represented by

where the superscript T denotes transpose. A matrix is a rectangular array. A data matrix X commonly has n rows of observations and p columns of measured features:

Here, ℝ denotes the real numbers. A tensor generalizes vectors and matrices to more dimensions; for example, a three-dimensional tensor may store voltage waveforms from multiple sensors over multiple time windows.

The choice of representation is consequential. A row may represent one time sample, one machine, or one operating event. Features may have different units, such as volts, amperes, hertz, and degrees Celsius. Consequently, preprocessing must account for scale and physical meaning.

1.2 Inner products, norms, and distance

For vectors a and b of equal dimension, the Euclidean inner product is

The Euclidean norm is

and the squared Euclidean distance between x and y is ‖x − y‖₂². These quantities are used in least-squares regression, nearest-neighbour classification, clustering, and signal comparison. The inner product also measures alignment: if two nonzero vectors have a positive inner product, their angle is less than 90°.

Other useful norms include the Manhattan norm

and the maximum norm ‖x‖∞ = maxᵢ |xᵢ|. The 1-norm often promotes sparse solutions in optimization, whereas the 2-norm gives smooth geometric penalties.

1.3 Matrix operations and linear systems

Matrix multiplication is defined only when the inner dimensions agree. If A is m × n and B is n × p, then AB is m × p, with entries

A linear electrical model may be written as

where A is a system matrix, x is an unknown vector, and b is a known excitation or measurement vector. In circuit analysis, A may arise from Kirchhoff's laws; in state estimation, it may describe the relationship between system states and measurements.

If A is square and nonsingular, x can theoretically be written as x = A⁻¹b. In computation, however, explicitly calculating A⁻¹ is usually less efficient and less numerically stable than solving the system directly using LU, QR, or Cholesky factorization. Cholesky applies when A is symmetric positive definite; QR is especially useful for least-squares problems.

1.4 Least squares and projections

When measurements are noisy or the system is overdetermined, an exact solution may not exist. Least squares selects x to minimize the residual:

If A has full column rank, the normal-equation solution is

This formula is mathematically valid under the stated rank condition, but forming AᵀA can worsen conditioning. QR-based numerical solvers are generally preferred in software.

1.5 Eigenvalues and singular-value decomposition

An eigenvalue λ and eigenvector v of a square matrix A satisfy

Eigenvalues are important in stability analysis, vibration modes, state-space systems, and principal component analysis. For a discrete linear system xₖ₊₁ = Axₖ, asymptotic stability requires every eigenvalue of A to have magnitude less than one, assuming a time-invariant finite-dimensional model.

Singular-value decomposition (SVD) factorizes any real m × n matrix as

where U and V are orthogonal matrices and Σ contains nonnegative singular values. SVD supports dimensionality reduction, noise filtering, pseudoinverse calculation, and analysis of ill-conditioned systems. Retaining the largest singular values produces a lower-rank approximation that captures dominant patterns but may discard small yet physically important abnormalities.

Worked Example 1: A two-sensor least-squares estimate

Suppose a simplified model relates an unknown scalar load x to two measurements:

The least-squares estimate minimizes (x − 3)² + (2x − 5)². Differentiating with respect to x gives

Thus 10x − 26 = 0, so x̂ = 2.6. The predicted measurements are [2.6, 5.2]ᵀ, and the residual is [−0.4, 0.2]ᵀ. The estimate does not fit every measurement exactly because the observations are inconsistent with the one-parameter model.

2. Probability and Uncertainty

Electrical measurements contain sensor noise, quantization error, interference, model error, and operating variability. Probability provides a formal language for these uncertainties.

2.1 Events and conditional probability

For events A and B, conditional probability is

The multiplication rule is P(A ∩ B) = P(A|B)P(B). Bayes' theorem is

In fault diagnosis, A may be “bearing fault exists” and B may be “high vibration is observed.” The prior probability P(A), sensor sensitivity represented by P(B|A), and false-alarm behaviour represented by P(B|not A) must all be considered. A highly accurate sensor can still produce many false alarms when the fault is very rare.

2.2 Random variables and distributions

A random variable maps outcomes to numerical values. A discrete variable has a probability mass function; a continuous variable has a probability density function f(x). For a continuous variable, probabilities are obtained by integration:

The cumulative distribution function is F(x) = P(X ≤ x). A valid density is nonnegative and integrates to one.

The expected value and variance are, respectively,

for a continuous variable, where μ = E[X]. The standard deviation is σ = √Var(X), and it has the same units as X. Variance has squared units.

For two variables X and Y, covariance is

and correlation is

provided both standard deviations are nonzero. Correlation measures linear association, not causation.

2.3 Common engineering distributions

The Bernoulli distribution models one binary trial with success probability p. The binomial distribution models the number of successes in n independent Bernoulli trials. The normal distribution is often useful for aggregate measurement noise:

This assumes a continuous variable with mean μ and standard deviation σ. The normal model should not be used automatically for bounded, strongly skewed, or heavy-tailed data.

The central limit theorem states, under standard independence and finite-variance conditions, that the standardized sample mean tends toward a normal distribution as sample size increases. It does not state that every individual observation is normal.

Worked Example 2: Bayesian fault diagnosis

Assume a motor fault has prior probability P(F) = 0.02. A vibration alarm is triggered with probability P(A|F) = 0.90 when a fault exists and P(A|not F) = 0.05 when it does not. Then

Bayes' theorem gives

Thus an alarm corresponds to approximately 26.9% fault probability, not 90%. The base rate is essential to interpreting the sensor output.

3. Statistics for Data and Model Evaluation

3.1 Samples and estimators

A population is the complete set of possible observations; a sample is the observed subset. For observations x₁, …, xₙ, the sample mean is

The unbiased sample variance is

The n − 1 denominator corrects the tendency of the sample mean to be estimated from the same data. If the data constitute the entire finite population of interest, a population variance with denominator n may instead be appropriate.

A sample statistic estimates a population parameter. Bias, variance, consistency, and efficiency describe estimator quality. No estimator is universally best; suitability depends on the data-generating process and objective.

3.2 Confidence intervals and hypothesis tests

If observations are independent and approximately normal, and the population standard deviation is unknown, a confidence interval for the mean may be written

where t is a Student's t critical value, α is the significance level, and n − 1 is the degrees of freedom. The frequentist interpretation is that a procedure constructed this way captures the fixed population mean in approximately 100(1 − α)% of repeated samples. It is not strictly correct to say that the particular unknown mean has a 95% probability of lying in the already calculated interval.

A hypothesis test compares a null hypothesis H₀ with an alternative H₁. The p-value is the probability, assuming H₀, of obtaining a result at least as extreme as the observed result under the specified test statistic. It is not the probability that H₀ is true. Statistical significance also does not establish engineering significance; a tiny effect can be statistically detectable with enough data.

3.3 Regression and error metrics

In supervised learning, regression predicts a numerical target y from features x. A linear model is ŷ = β₀ + β₁x₁ + … + βₚxₚ. Common metrics include mean absolute error

and mean squared error

MAE has the units of the target; MSE has squared units and penalizes large errors more strongly. Metrics must be selected according to engineering consequences. For a protection system, missed faults and false trips may have very different costs, so accuracy alone is inadequate.

Data leakage occurs when information unavailable at deployment enters training or evaluation. Randomly splitting adjacent time-series samples can leak operating conditions across sets. Time-ordered or group-based splitting is often more appropriate for electrical systems.

4. Calculus and Optimization

4.1 Derivatives and gradients

The derivative of a scalar function f(x) is

when the limit exists. It measures local rate of change. For multiple variables, the gradient is

The gradient points in the direction of greatest local increase. Therefore, the negative gradient is used for local descent.

The chain rule explains how derivatives propagate through computational layers. If z = g(y) and y = h(x), then dz/dx = (dg/dy)(dy/dx). In neural networks, repeated application of this rule is backpropagation.

4.2 Optimization and gradient descent

Given a differentiable objective J(θ), gradient descent updates parameters according to

where η > 0 is the learning rate. This method assumes that the gradient is available or can be estimated and that the step is sufficiently suitable for the local geometry. A learning rate that is too large can cause divergence; one that is too small can make progress impractically slow.

For a convex differentiable objective, every local minimum is global, although convergence speed depends on smoothness and conditioning. For nonconvex objectives, gradient methods may reach local minima, saddle points, or other stationary regions.

Worked Example 3: One gradient step

Let J(w) = (w − 3)² and suppose w₀ = 1. Then ∇J(w) = 2(w − 3), so the gradient at w₀ is −4. With learning rate η = 0.1,

The update moves w toward the minimizer w = 3. It does not reach the optimum in one step because the learning rate is finite.

4.3 Regularization

A regularized objective combines data fit with a penalty:

where L is loss, R is a penalty, and λ ≥ 0 controls the trade-off. L2 regularization uses R(θ) = ‖θ‖₂² and discourages large parameters. L1 regularization uses R(θ) = ‖θ‖₁ and can produce exact zeros. Regularization can reduce overfitting but may also remove physically meaningful weak effects if λ is excessive.

5. Numerical Methods and Computational Complexity

5.1 Floating-point computation

Computers represent real numbers with finite precision. Round-off error arises because many real values cannot be represented exactly. Truncation error arises when an infinite process is approximated by a finite one. Conditioning describes sensitivity of the mathematical problem; stability describes how an algorithm responds to errors. A well-conditioned problem can be damaged by an unstable algorithm, and a stable algorithm cannot remove inherent ill-conditioning.

For a linear system Ax = b, the condition number in a chosen matrix norm is

for nonsingular A. A large κ(A) means small relative perturbations may cause large relative changes in the solution. Units and scaling affect numerical behaviour, so feature normalization and physically sensible nondimensionalization are often useful.

5.2 Root finding and integration

To solve f(x) = 0, bisection requires a continuous function and an interval [a,b] with f(a)f(b) < 0. It is robust but converges linearly. Newton's method uses

and can converge quadratically near a simple root when the initial guess is sufficiently good and f′ is nonzero. It may diverge or encounter a zero derivative otherwise.

Numerical integration approximates ∫ₐᵇ f(x)dx. The trapezoidal rule with spacing h is

for equally spaced points, where h = (b − a)/n. The approximation is exact for linear functions and generally has error that decreases with h² for sufficiently smooth functions.

5.3 Complexity

Big-O notation describes asymptotic growth. A loop over n samples is O(n); a dense matrix-vector multiplication for an n × n matrix is O(n²); naive dense matrix multiplication is O(n³). Complexity does not specify actual runtime, because hardware, memory access, vectorization, and implementation matter. Memory complexity is equally important in high-rate sensor systems.

A practical AI engineer balances accuracy, latency, energy consumption, memory, and maintainability. A slightly less accurate model may be preferable for a protective relay if it has deterministic millisecond-level execution and clear failure behaviour.

6. Python-Based Scientific Computing

Python is widely used because it combines readable syntax with mature numerical libraries. NumPy supplies multidimensional arrays and vectorized operations; SciPy provides optimization, linear algebra, integration, and signal-processing routines; pandas supports tabular data; Matplotlib supports visualization; and scikit-learn provides many classical machine-learning methods.

A typical workflow is:

import numpy as np
from scipy import linalg

A = np.array([[3.0, 1.0], [1.0, 2.0]])
b = np.array([9.0, 8.0])
x = linalg.solve(A, b)
print(x)

The code solves Ax = b without explicitly forming A⁻¹. For reproducibility, specify package versions, random seeds where appropriate, data-preprocessing steps, train-test splits, model hyperparameters, units, and evaluation metrics. A random seed improves repeatability but does not guarantee identical results across all hardware and library versions.

Vectorization is generally faster than Python loops because operations are delegated to optimized compiled routines. Nevertheless, vectorization can increase memory use. Profiling should precede optimization; premature optimization can reduce clarity without improving the relevant bottleneck.

7. Experiment Design and Reproducibility

An AI experiment should begin with a precise engineering question. “Can a model classify faults?” is incomplete. A stronger question specifies fault classes, sensors, operating range, prediction horizon, costs of errors, and deployment constraints.

Important design principles include:

• Define the population and sampling procedure.
• Separate training, validation, and final test data.
• Prevent temporal, equipment-level, and preprocessing leakage.
• Use representative operating conditions and document exclusions.
• Repeat experiments or use cross-validation when appropriate.
• Report uncertainty, not only a single performance number.
• Preserve code, data provenance, configuration, and software environment.
• Compare against a simple baseline and an engineering rule-based method.

Cross-validation is useful when observations are sufficiently independent. It is inappropriate to randomly mix strongly correlated samples from the same event across folds. For equipment datasets, leave-one-device-out evaluation may better estimate generalization to a new device.

Reproducibility has several levels: computational reproducibility means the same inputs and code yield the same outputs; statistical reproducibility means repeated sampling supports similar conclusions; external validity means conclusions transfer to relevant new conditions. These are related but not identical.

8. Applied Case: Predictive Maintenance of an Induction Motor

The following is an illustrative case, not a report of a particular industrial installation.

Background and problem

A plant monitors current, vibration, speed, winding temperature, and ambient temperature from induction motors. Maintenance records identify normal operation and several bearing-fault conditions. The engineering objective is to identify developing faults early while limiting false maintenance alarms.

Analysis

The raw signals have different units and sampling rates. Vibration contains high-frequency information, while temperature changes slowly. Each recording is divided into time windows. Features may include RMS current, crest factor, spectral-band energy, temperature slope, and speed-normalized quantities. The feature matrix X is standardized using statistics calculated only from the training data:

where μⱼ,train and σⱼ,train are the training mean and standard deviation for feature j. Applying test-set statistics would leak information.

A baseline classifier using threshold rules is compared with logistic regression and a tree-based model. Data are split by motor rather than by randomly selected windows so that windows from one motor do not appear in both training and testing. Evaluation includes recall for faults, false-alarm rate, precision, detection latency, and computational time.

Approach and solution

A logistic model estimates fault probability using

where Y = 1 denotes the selected fault class. A decision threshold is chosen using maintenance costs rather than automatically setting it to 0.5. If missed faults are much more expensive than inspections, a lower threshold may be justified, subject to operational constraints.

The deployment pipeline includes sensor validation, missing-data handling, feature extraction, scaling, prediction, alarm persistence logic, and human review. Model outputs are logged with software version, input quality indicators, and timestamp.

Outcome

Suppose the model improves fault recall over the rule-based baseline but produces more false alarms under a previously unseen speed range. The appropriate conclusion is not simply that the model “works.” The model has learned a distribution that does not fully cover deployment conditions. Additional data, speed-aware features, calibration analysis, or a controlled fallback strategy is required.

Learning points

The case illustrates that mathematical correctness alone is insufficient. Good AI engineering requires appropriate sampling, leakage control, physical feature interpretation, cost-sensitive evaluation, uncertainty awareness, and monitoring after deployment.

9. Limitations, Governance, and Contemporary Practice

Mathematical models simplify physical systems. A classifier may associate a fault with a particular maintenance crew, sensor location, or season rather than with the fault mechanism. Distribution shift can occur when equipment ages, sensors are replaced, controls change, or operating regimes expand.

Electrical-engineering AI also has safety implications. A model should not silently override protection logic without validated safeguards. Engineers should document intended use, operating limits, fallback behaviour, human authority, and evidence supporting release. Data may contain commercially sensitive information or identifiable operator information; access control, retention policies, and cybersecurity are therefore relevant.

Current practice increasingly combines data-driven models with physical constraints, digital twins, uncertainty quantification, and edge computing. Physics-informed approaches can improve extrapolation when governing equations are known, although they introduce modelling assumptions and calibration demands. Quantization and hardware acceleration can reduce edge latency, but reduced numerical precision must be tested against safety and accuracy requirements.

Best practice is to treat an AI model as one component of a sociotechnical system. Validation should include nominal operation, sensor faults, missing data, adversarial or corrupted inputs, out-of-distribution conditions, and safe degradation. Documentation should make assumptions and limitations visible to future users.

Chapter Summary

Linear algebra represents engineering measurements, models, transformations, and state spaces. Inner products, norms, least squares, eigenvalues, and SVD support estimation, dimensionality reduction, stability analysis, and signal processing. Probability describes uncertainty, while conditional probability and Bayes' theorem support diagnosis. Statistical estimators, confidence intervals, hypothesis tests, regression losses, and careful data splitting support defensible conclusions.

Calculus supplies derivatives, gradients, the chain rule, and optimization methods such as gradient descent. Numerical methods make equations solvable on finite-precision computers, but conditioning, stability, and error must be considered. Computational complexity connects algorithms to latency and resource constraints. Python libraries enable practical implementation, while reproducible experiment design prevents leakage and improves scientific credibility. In electrical-engineering applications, model performance must be interpreted together with physical validity, safety, cost, uncertainty, and governance.

Key Terms

Vector: An ordered array of numerical components.

Matrix: A rectangular array representing data or a linear transformation.

Inner product: A scalar product measuring algebraic alignment between vectors.

Norm: A function measuring vector magnitude.

Least squares: Estimation by minimizing the squared residual norm.

Eigenvalue: A scalar satisfying Av = λv for a nonzero eigenvector v.

Singular-value decomposition: Factorization A = UΣVᵀ into orthogonal factors and singular values.

Random variable: A numerical function of uncertain outcomes.

Expected value: The probability-weighted average of a random variable.

Variance: The expected squared deviation from the mean.

Covariance: A measure of joint linear variation between two variables.

Bayes' theorem: A rule for updating probabilities using evidence.

Estimator: A statistic used to infer a population parameter.

Confidence interval: An interval produced by a procedure with stated long-run coverage.

Gradient: A vector of partial derivatives.

Regularization: A penalty added to an objective to control model complexity.

Condition number: A measure of sensitivity of a problem to perturbations.

Computational complexity: The growth of computational resources with problem size.

Data leakage: Use of information unavailable at deployment during training or evaluation.

Reproducibility: The ability to obtain and support consistent results through documented methods.

Review Questions

Short Answer Questions

1. Why is solving Ax = b generally preferable to explicitly computing A⁻¹b?
2. Distinguish covariance from correlation.
3. What does the learning rate control in gradient descent?
4. Why can random train-test splitting be inappropriate for time-series data?
5. State two sources of numerical error.

Long Answer Questions

1. Explain how least squares, QR factorization, and conditioning are related in engineering estimation.
2. Derive and interpret Bayes' theorem for a sensor-based fault diagnosis problem.
3. Compare MAE and MSE, including their units and sensitivity to large errors.
4. Explain the role of gradients, the chain rule, and regularization in training an AI model.
5. Design a reproducible Python experiment for predicting photovoltaic power and identify possible sources of leakage.

Analytical / Application Questions

1. A fault has prior probability 0.01. A detector has sensitivity 0.95 and false-alarm probability 0.04. Calculate the probability of fault given an alarm and interpret the result.
2. Design an evaluation protocol for a motor-fault classifier when data are collected from 20 motors over six months. Justify the split strategy, metrics, and baseline.
3. A linear system has a large condition number and produces unstable estimates. Propose numerical and engineering remedies, and explain their trade-offs.

Multiple Choice Questions with Answers

Question 1

For a full-column-rank least-squares problem, which expression gives the theoretical normal-equation solution?

A. x̂ = (AᵀA)⁻¹Aᵀb

B. x̂ = AᵀA⁻¹b

C. x̂ = A⁻¹Aᵀb

D. x̂ = (AAᵀ)⁻¹b

Correct Answer: A. x̂ = (AᵀA)⁻¹Aᵀb

Question 2

In the Bayesian fault example, what was the approximate probability of a fault after an alarm?

A. 2.0%

B. 6.7%

C. 26.9%

D. 90.0%

Correct Answer: C. 26.9%

Question 3

Which quantity has the same units as the original measured variable?

A. Variance

B. Standard deviation

C. Squared error

D. Condition number

Correct Answer: B. Standard deviation

Question 4

What is the principal purpose of the negative gradient in gradient descent?

A. To maximize the objective locally

B. To move toward local decrease of the objective

C. To remove all measurement noise

D. To guarantee a global minimum for every objective

Correct Answer: B. To move toward local decrease of the objective

Question 5

Which factorization is especially associated with dimensionality reduction and rank approximation?

A. SVD

B. Euler factorization

C. Bisection factorization

D. Bernoulli factorization

Correct Answer: A. SVD

Question 6

If a dense n × n matrix-vector multiplication is performed directly, its typical asymptotic time complexity is:

A. O(1)

B. O(log n)

C. O(n)

D. O(n²)

Correct Answer: D. O(n²)

Question 7

What condition is required for the basic bisection method to begin on an interval [a,b]?

A. f(a) = f(b) = 0

B. f(a)f(b) < 0 for a continuous function

C. f′(a) = f′(b)

D. The function must be linear

Correct Answer: B. f(a)f(b) < 0 for a continuous function

Question 8

Why should scaling statistics for a test set normally be calculated from training data only?

A. To increase the number of test samples

B. To prevent information leakage

C. To guarantee normality

D. To eliminate all model bias

Correct Answer: B. To prevent information leakage

Question 9

For x₀ = 1, J(w) = (w − 3)², and η = 0.1, what is one gradient-descent update?

A. 0.6

B. 1.4

C. 2.0

D. 3.4

Correct Answer: B. 1.4

Question 10

Which statement about a p-value is correct?

A. It is the probability that the null hypothesis is true.

B. It is the probability that the alternative hypothesis is true.

C. Assuming the null hypothesis, it measures how extreme the observed result is.

D. It measures engineering importance directly.

Correct Answer: C. Assuming the null hypothesis, it measures how extreme the observed result is.