Artificial Intelligence and Machine Learning Fundamentals

Introduction and Context

Artificial intelligence (AI) refers to computational systems that perform tasks commonly associated with human intelligence, such as perception, reasoning, prediction, decision-making and learning. Machine learning (ML) is a major branch of AI in which a model learns relationships from data rather than being programmed with an explicit rule for every situation.

In mechanical engineering, AI and ML are increasingly used with sensors, industrial networks and cyber-physical systems. Typical applications include predictive maintenance, remaining-useful-life estimation, quality inspection, process optimisation, robotics, energy management, fault diagnosis and digital twins. Their value depends not merely on selecting an algorithm, but on formulating the engineering problem correctly, obtaining representative data, validating the model rigorously and managing uncertainty.

A mechanical engineer should therefore understand both the mathematical foundations of learning and the physical context in which predictions will be used. A model that achieves high accuracy on historical laboratory data may fail in a production plant because of sensor drift, changing loads, different materials or previously unseen operating conditions.

Learning Objectives

After studying this chapter, a student should be able to:

1. Distinguish supervised, unsupervised, semi-supervised and reinforcement learning.
2. Formulate regression, classification, clustering and anomaly-detection problems.
3. Explain the operation and limitations of neural-network models.
4. Select appropriate validation procedures and performance metrics.
5. Interpret uncertainty, overfitting, data leakage and distribution shift.
6. Apply ML to a mechanical-engineering case involving condition monitoring.
7. Identify ethical, safety, governance and professional responsibilities in AI deployment.

1. Fundamental Terminology and the Learning Problem

A dataset consists of observations. An observation may contain sensor readings, operating conditions, inspection results or historical outcomes. A feature is an input variable used by a model; examples include vibration RMS, shaft speed, bearing temperature and lubricant pressure. A target, label or response is the quantity to be predicted.

For supervised learning, a dataset is commonly written as:

Here, x_i is the feature vector for observation i, y_i is its target, and n is the number of observations. A model f_θ with parameters θ produces a prediction ŷ_i = f_θ(x_i). Training chooses parameters that minimise a loss function, often with a regularisation term:

The loss L measures prediction error, Ω(θ) penalises model complexity, and λ ≥ 0 controls the strength of regularisation. The equation assumes that observations are sufficiently representative of the deployment population and that the loss reflects the engineering cost of errors.

A model is not the same as an algorithm. Linear regression, for example, is a model family; gradient descent is an optimisation algorithm used to estimate its parameters. A hyperparameter is selected by the practitioner rather than learned directly from the training objective, such as a tree depth, learning rate or number of neighbours.

Generalisation, bias and variance

Generalisation is the ability to perform well on new data. Underfitting occurs when a model is too simple to represent important structure. Overfitting occurs when it learns random variation or noise in the training data. The bias–variance perspective describes expected prediction error as a combination of irreducible noise, squared bias and variance:

Here, σ² represents irreducible noise under the assumed data-generating process. The relationship is a conceptual decomposition under standard squared-error assumptions; it is not a guarantee that each component can be measured uniquely from one dataset.

2. Four Learning Paradigms

2.1 Supervised learning

In supervised learning, each training example has a target. Regression predicts a continuous quantity, such as bearing temperature or remaining useful life. Classification predicts a category, such as healthy, misaligned or bearing-defective.

A linear regression model with p features is:

The ordinary least-squares coefficients minimise the sum of squared residuals:

This model assumes an approximately linear conditional mean, independent and appropriately structured errors, and no severe multicollinearity when coefficient interpretation is important. It can be useful as a transparent baseline even when a nonlinear model is ultimately required.

For binary classification, logistic regression models the probability of class 1:

A threshold, often 0.5 but not necessarily, converts probability into a class decision. The threshold should reflect the relative consequences of missed failures and false alarms.

2.2 Unsupervised learning

Unsupervised learning uses inputs without target labels. Its purposes include discovering operating regimes, compressing data, visualising structure and identifying unusual observations. Clustering assigns observations to groups according to a similarity measure.

In k-means clustering, the objective is:

Here, K is the selected number of clusters, C_k is cluster k, and μ_k is its centroid. The standard algorithm alternates between assigning each point to its nearest centroid and recalculating centroids. It tends to work best for compact, roughly spherical clusters measured on appropriately scaled features. It is sensitive to initialisation, outliers and the choice of K.

Principal component analysis (PCA) transforms correlated variables into orthogonal directions of maximum variance. If the centred data matrix is X, principal directions are eigenvectors of its covariance matrix. PCA can reduce dimensionality and support visualisation, but high variance is not always equivalent to engineering importance; a low-variance fault signature may be safety-critical.

2.3 Semi-supervised learning

Semi-supervised learning combines a small labelled dataset with a larger unlabelled dataset. This is particularly relevant in mechanical systems because fault labels may require destructive testing, expert inspection or long operating periods. Methods include pseudo-labelling, consistency regularisation, graph-based learning and representation learning.

The central assumption is that unlabelled observations contain useful information about the structure of the problem. Common assumptions include smoothness—that nearby points tend to have similar labels—and cluster structure—that points in the same dense region tend to share a class. These assumptions can fail when different failure modes overlap in sensor space. Pseudo-labels must therefore be treated cautiously, especially in safety-critical applications.

2.4 Reinforcement learning

Reinforcement learning (RL) concerns sequential decision-making. An agent observes a state, chooses an action, receives a reward and transitions to a new state. A mechanical example is a controller that adjusts cutting speed or HVAC operation while balancing productivity, energy and equipment constraints.

A Markov decision process is represented by states, actions, transition probabilities, rewards and a discount factor γ, where 0 ≤ γ < 1. The value of a policy π is:

The expectation reflects uncertainty in transitions and actions. RL must balance exploration of alternatives with exploitation of known good actions. Direct exploration on expensive machinery can cause damage; simulation, digital twins, conservative policies and safety constraints are therefore important. RL is not simply supervised learning with sequential data because the agent’s actions influence the data it later observes.

3. Data Preparation for Mechanical Applications

The quality of an ML result is constrained by the quality and relevance of its data. A sound workflow includes:

1. Define the engineering decision and the cost of errors.
2. Identify sensors, sampling rates, operating regimes and failure definitions.
3. Synchronise timestamps and handle missing or corrupted readings.
4. Construct physically meaningful features or use appropriate raw signals.
5. Split data according to deployment conditions.
6. Fit preprocessing steps only on training data.
7. Train, validate, test and document the model.

Feature scaling is important for distance-based algorithms, PCA and many neural networks. Standardisation uses training-set mean μ_j and standard deviation s_j:

The same training values μ_j and s_j must be applied to validation and test data. Recomputing them using the complete dataset leaks information from the evaluation set.

Time-series data require special care. Randomly placing adjacent samples into training and test sets can make performance appear unrealistically high because neighbouring windows share operating conditions or even the same fault event. Splitting by machine, production batch, time period or failure episode is usually more credible.

4. Regression, Classification and Clustering Models

Regression models

Useful regression methods include linear regression, polynomial regression, decision trees, random forests, gradient-boosted trees, support-vector regression and neural networks. Model selection should consider accuracy, extrapolation behaviour, interpretability, computational cost and physical plausibility.

Common regression metrics are:

MAE has the same units as the target and gives equal weight to absolute errors. RMSE also has target units but penalises large errors more strongly. R² compares the residual sum of squares with variation around the sample mean; it can be negative on test data and does not establish causation or engineering adequacy. Percentage metrics should be avoided or interpreted carefully when actual values approach zero.

Classification models

Besides logistic regression, common classifiers include decision trees, random forests, support-vector machines, k-nearest neighbours and neural networks. A confusion matrix records true positives (TP), true negatives (TN), false positives (FP) and false negatives (FN).

Accuracy can be misleading for rare failures. Recall is important when missing a dangerous fault is costly; precision is important when unnecessary maintenance is expensive. Receiver operating characteristic (ROC) curves show true-positive rate against false-positive rate over thresholds. Precision–recall curves are often more informative for strongly imbalanced failure data.

For multiclass tasks, macro averaging gives each class equal weight, while micro averaging aggregates individual predictions. The choice should be reported rather than hidden.

Clustering

Before clustering, engineers should decide what similarity means. Euclidean distance between two scaled feature vectors is:

This measure is meaningful only when feature scales and correlations are handled appropriately. Alternatives include Manhattan distance, cosine distance or domain-specific distances. Cluster quality can be assessed using silhouette analysis, stability under resampling and engineering interpretability—not only a numerical index.

5. Anomaly Detection

Anomaly detection identifies observations that differ from expected behaviour. It is useful when labelled failures are scarce. An anomaly may be a sensor fault, a new operating condition, a process upset or an actual component failure; the algorithm alone cannot determine which interpretation is correct.

Methods include statistical control limits, distance-based methods, isolation forests, one-class support-vector machines and autoencoders. A simple residual rule declares an anomaly when:

where τ is a threshold selected using training data, validation data and the consequences of false alarms. If residuals are approximately normal with standard deviation σ_r, a two-sided threshold may be written as τ = kσ_r, but the normality assumption must be checked and serial correlation must not be ignored.

For time series, persistence rules are often better than reacting to one point. For example, an alert may require several consecutive threshold exceedances, while also allowing rapid escalation for a severe signal. Thresholds should be reviewed after sensor replacement, process changes or changes in operating regime.

6. Neural Networks

A feed-forward neural network consists of layers of artificial neurons. A neuron computes a weighted sum followed by an activation function:

Here, x is the input vector, w is a weight vector, b is a bias, and φ is an activation function. Common hidden-layer activations include the rectified linear unit, ReLU(z) = max(0,z), and the hyperbolic tangent. For multiclass classification, softmax converts logits z_k into class probabilities:

The probabilities are nonnegative and sum to one. Training commonly uses gradient-based optimisation and backpropagation to reduce a loss such as mean squared error for regression or cross-entropy for classification.

Neural networks can learn nonlinear relationships and may process images, vibration spectra, acoustic signals or sequences. Their limitations include data requirements, sensitivity to preprocessing and hyperparameters, reduced interpretability and vulnerability to distribution shift. Dropout, weight decay, early stopping, data augmentation and validation monitoring can reduce overfitting, but none guarantees reliable deployment.

Convolutional neural networks exploit local patterns and weight sharing, making them useful for images and transformed vibration signals. Recurrent and attention-based architectures can represent temporal relationships, although simpler engineered features may be preferable when data are limited and traceability is essential.

7. Validation, Metrics and Uncertainty

Validation strategies

A basic hold-out design separates training, validation and test data. The training set estimates model parameters, the validation set supports model and hyperparameter selection, and the test set provides a final unbiased estimate. The test set must not be repeatedly consulted during development.

k-fold cross-validation divides data into k folds, trains on k−1 folds and evaluates on the remaining fold, repeating until each fold has been used for validation. It is useful for independent observations with limited data. Stratified cross-validation preserves class proportions for classification. Grouped cross-validation keeps observations from the same machine or operating episode together. Time-series validation should preserve temporal order, for example by rolling or expanding windows.

Nested cross-validation is appropriate when both model selection and performance estimation are needed with limited data. Confidence intervals may be estimated using bootstrap resampling, but resampling must preserve dependence structures such as machine identity or failure episodes.

Uncertainty analysis

A point prediction alone can be unsafe. Aleatoric uncertainty arises from inherent variability or measurement noise. Epistemic uncertainty arises from limited knowledge, sparse training data or unfamiliar operating conditions. Useful approaches include prediction intervals, quantile regression, ensembles, Bayesian models, conformal prediction and calibration analysis.

For a regression model, a prediction interval should be interpreted as a range produced under specified assumptions and coverage procedures, not as a guarantee. For classification, calibration examines whether predicted probabilities correspond to observed frequencies. If a group of predictions is assigned probability 0.8, approximately 80% should belong to the stated class over a suitably representative population.

Uncertainty should be communicated to decision-makers. A maintenance system might recommend inspection when failure probability is high, but defer automated action when the model is uncertain or the input lies outside the training distribution. Out-of-distribution detection and monitoring of feature distributions are therefore part of deployment, not optional research additions.

8. Worked Example: Bearing Temperature Regression

Suppose a laboratory dataset contains shaft speed N in revolutions per minute, radial load F in newtons and vibration RMS v in millimetres per second. The target is bearing temperature rise ΔT in °C. A baseline model is:

Assume that the model is fitted only on the training set. On four independent test observations, the measured temperature rises are 20, 25, 30 and 35 °C, while predictions are 22, 23, 33 and 32 °C. The errors y_i − ŷ_i are −2, 2, −3 and 3 °C.

The MAE is:

The RMSE is:

The mean measured value is 27.5 °C. The total sum of squares is 125, and the residual sum of squares is 26. Therefore:

The result indicates that the test predictions explain approximately 79.2% of the sample variation under the squared-error definition. It does not demonstrate that the model is safe, causal or accurate across all loads. Engineers should inspect residuals by speed and load, check whether the test set represents future operation, quantify uncertainty and compare the error with the allowable temperature limit.

9. Applied Case: Illustrative Predictive Maintenance System

Background

The following is an illustrative case. A manufacturing plant operates 48 motor-driven centrifugal pumps. Unexpected bearing failures interrupt production, while routine replacement creates unnecessary cost and downtime.

Problem

The plant has vibration, temperature, speed and load measurements at one-minute intervals. Only 31 historical failure episodes have been confirmed by inspection. Many more records are unlabelled, and operating conditions vary substantially between pumps.

Analysis

A random row-level split would be inappropriate because adjacent measurements from one failure episode are highly correlated. The engineering team groups records by pump and failure episode. It defines a prediction horizon of 14 days and distinguishes three outcomes: healthy, warning and confirmed impending failure. Maintenance personnel identify the cost of a missed failure as substantially greater than the cost of a false alarm.

Exploratory analysis shows that vibration RMS increases under high load even when the pump is healthy. Consequently, raw vibration thresholds generate many false alarms. Features are normalised relative to operating regime, and spectral indicators are calculated from windows whose duration is documented. Sensor outages are represented explicitly rather than silently interpolated across long gaps.

Approach and solution

The team first trains an interpretable logistic-regression baseline. A gradient-boosted tree is then compared with it using grouped, time-aware validation. An unsupervised clustering model identifies normal operating regimes, and a residual-based anomaly detector is used as a supplementary alarm rather than as an autonomous diagnosis. Because only a small fraction of data are labelled, unlabelled records support representation learning, but pseudo-labels are reviewed before entering the final training set.

Performance is assessed using recall, precision, PR-AUC, calibration and alert lead time. The threshold is selected with maintenance personnel, not solely by maximising accuracy. A warning is issued only when an elevated score persists across multiple windows, except when temperature exceeds a separately defined safety limit.

Outcome

In a retrospective, pump-grouped test, the selected model provides useful early warnings and fewer nuisance alarms than the original fixed vibration rule. The plant does not allow the model to shut down pumps automatically. Instead, it creates an inspection work order containing the predicted risk, dominant features, recent trend and uncertainty indicator. Technicians can reject the recommendation and record the reason, creating feedback for monitoring and future improvement.

Learning points

1. The engineering decision and cost structure determine the metric and threshold.
2. Grouped and time-aware validation is more credible than random row splitting.
3. Unlabelled data can help, but weak labels can also propagate error.
4. Anomaly detection identifies deviation, not necessarily a physical fault.
5. Human oversight and traceable records are important for safety and accountability.

10. Limitations, Governance and Professional Practice

AI systems may reproduce sampling bias, discriminate between operating groups or fail for machines not represented in training. Sensor placement, calibration and maintenance affect both inputs and labels. Data leakage can arise when a feature is recorded after a fault has already been diagnosed. Engineers must establish a clear prediction time so that every feature would genuinely be available when the decision is made.

Safety-critical applications require hazard analysis, fail-safe behaviour, independent verification and validation, cybersecurity controls and change management. A model should have an owner, documented version, training-data description, intended-use statement, known limitations and rollback procedure. Continuous monitoring should examine performance, calibration, input drift, missingness, alert rates and changes in failure modes.

Privacy may matter even in industrial systems if worker activity, biometric information or identifiable maintenance records are collected. Data minimisation, access control, retention limits and secure transmission are appropriate governance practices. Regulatory obligations vary by jurisdiction and application, but professional responsibility remains: an engineer must not present uncertain predictions as facts or deploy a model beyond its validated operating envelope.

Contemporary developments

Current research includes physics-informed ML, hybrid models combining conservation laws with data-driven components, self-supervised learning for unlabelled sensor streams, foundation models for time series and edge AI for low-latency inference. These developments are promising, but larger or more complex models do not automatically improve engineering reliability. Hybrid approaches are particularly valuable when physical knowledge is strong but measurements are sparse, because physics can constrain implausible predictions and improve extrapolation.

Best-Practice Checklist

• Begin with a measurable engineering decision, not an algorithm.
• Establish the prediction horizon, target definition and error costs.
• Use representative, quality-controlled data and document sensor provenance.
• Separate machines, episodes or future periods during validation when appropriate.
• Fit preprocessing only on training data.
• Compare complex models with transparent baselines.
• Report several relevant metrics, not accuracy alone.
• Examine calibration, uncertainty and out-of-distribution behaviour.
• Use domain knowledge to test physical plausibility.
• Monitor the deployed system and retain human authority for high-consequence decisions.

Chapter Summary

Machine learning learns useful relationships from data, while AI includes the broader capabilities of perception, reasoning and decision-making. Supervised learning uses labelled examples for regression and classification; unsupervised learning discovers structure such as clusters or principal directions; semi-supervised learning combines limited labels with abundant unlabelled data; and reinforcement learning learns sequential policies through rewards.

Mechanical-engineering applications require careful data preparation, physically meaningful features and validation that reflects machines, operating episodes and time. Regression should be evaluated with measures such as MAE and RMSE; classification requires attention to precision, recall, imbalance and threshold costs; clustering requires meaningful distance measures and engineering interpretation; anomaly detection requires threshold design and investigation of alternative causes. Neural networks provide flexible nonlinear models but introduce data, interpretability and distribution-shift challenges.

Reliable deployment requires separation of training and test information, uncertainty analysis, calibration, monitoring, cybersecurity, documentation and human oversight. The central professional principle is that model performance must be judged in the context of the engineering decision and its consequences.

Key Terms

Artificial intelligence: The field concerned with systems that perform tasks associated with human intelligence.

Machine learning: Methods that infer patterns or decision rules from data.

Feature: An input variable supplied to a model.

Target: The quantity or class that a supervised model predicts.

Regression: Prediction of a continuous numerical quantity.

Classification: Prediction of a discrete class or category.

Clustering: Grouping observations according to similarity without supplied labels.

Anomaly detection: Identification of observations that differ from expected behaviour.

Semi-supervised learning: Learning using both labelled and unlabelled data.

Reinforcement learning: Sequential learning through interaction, actions and rewards.

Overfitting: Learning training-specific noise rather than general relationships.

Regularisation: A method that penalises excessive model complexity.

Feature scaling: Transformation of variables to comparable numerical scales.

Data leakage: Unintended use of information unavailable at prediction time.

Confusion matrix: A table of correct and incorrect classification outcomes.

Precision: The fraction of predicted positives that are true positives.

Recall: The fraction of actual positives that are detected.

Calibration: Agreement between predicted probabilities and observed frequencies.

Epistemic uncertainty: Uncertainty caused by limited knowledge or data.

Distribution shift: Change between the training data distribution and deployment conditions.

Review Questions

Short Answer Questions

1. Distinguish regression from classification and give one mechanical-engineering example of each.
2. Why can accuracy be misleading when equipment failures are rare?
3. State two reasons why random row-level splitting may be unsuitable for sensor time series.
4. What is the purpose of regularisation in an ML model?
5. Distinguish aleatoric uncertainty from epistemic uncertainty.

Long Answer Questions

1. Explain supervised, unsupervised, semi-supervised and reinforcement learning, including an appropriate mechanical-engineering application for each.
2. Describe a complete workflow for developing a predictive-maintenance model, from problem definition through deployment monitoring.
3. Compare MAE, RMSE, precision, recall and F1 score. Explain when each may be especially useful.
4. Explain how a feed-forward neural network produces a prediction and discuss methods for controlling overfitting.
5. Discuss the governance, safety and ethical requirements for deploying ML in an industrial plant.

Analytical / Application Questions

1. A classifier detects 18 of 20 real failures and raises 12 false alarms. Calculate recall and precision, and explain which additional operational information is needed before selecting a decision threshold.
2. Design a validation strategy for data collected from 10 machines, each with multiple operating episodes. Explain how you would prevent information leakage and estimate uncertainty.
3. A vibration anomaly detector produces frequent alarms only at high rotational speed. Propose an investigation plan using physical reasoning, feature analysis, operating-regime modelling and threshold redesign.

Multiple Choice Questions with Answers

Question 1

Which learning paradigm uses labelled input–output examples to learn a prediction function?

A. Supervised learning

B. Unsupervised learning

C. Reinforcement learning

D. Random search

Correct Answer: A. Supervised learning

Question 2

Which metric is especially sensitive to large regression errors because it squares residuals before averaging?

A. MAE

B. RMSE

C. Precision

D. Recall

Correct Answer: B. RMSE

Question 3

A classifier detects 18 of 20 actual failures. What is its recall?

A. 0.10

B. 0.18

C. 0.80

D. 0.90

Correct Answer: D. 0.90

Question 4

What is the main purpose of keeping a final test set separate from model development?

A. To increase the number of training features

B. To provide an approximately unbiased final performance estimate

C. To eliminate measurement noise

D. To guarantee zero overfitting

Correct Answer: B. To provide an approximately unbiased final performance estimate

Question 5

Which method groups observations around centroids by minimising within-cluster squared distances?

A. Logistic regression

B. k-means clustering

C. Naive thresholding

D. Backpropagation

Correct Answer: B. k-means clustering

Question 6

Why is a random row-level split dangerous for adjacent machine-sensor windows?

A. It always removes all labels

B. It can place highly related observations in both training and test sets

C. It prevents feature scaling

D. It makes the target continuous

Correct Answer: B. It can place highly related observations in both training and test sets

Question 7

In a binary classifier, precision is defined as:

A. TP divided by TP plus FN

B. TN divided by TN plus FP

C. TP divided by TP plus FP

D. TP plus TN divided by all observations

Correct Answer: C. TP divided by TP plus FP

Question 8

In a neural-network neuron, what does the activation function do?

A. It converts the training set into a test set

B. It applies a nonlinear or other transformation to the weighted input and bias

C. It guarantees calibrated probabilities

D. It removes all sensor noise

Correct Answer: B. It applies a nonlinear or other transformation to the weighted input and bias

Question 9

Which situation is an example of data leakage?

A. Scaling test data using training-set mean and standard deviation

B. Using a feature recorded after a failure diagnosis to predict that failure

C. Evaluating a model on an independent future period

D. Monitoring missing sensor values

Correct Answer: B. Using a feature recorded after a failure diagnosis to predict that failure

Question 10

Which statement best describes reinforcement learning?

A. It predicts labels from a fixed labelled dataset only

B. It groups data without a target or action

C. It learns sequential decisions through states, actions and rewards

D. It only estimates linear relationships

Correct Answer: C. It learns sequential decisions through states, actions and rewards