Probability, Random Variables, and Engineering Statistics

Introduction

Mechatronic systems operate with incomplete information. Sensor readings contain noise, manufacturing processes vary, components fail at uncertain times, and inspection decisions are based on samples rather than complete populations. Probability and statistics provide the mathematical language for describing this uncertainty and converting measurements into engineering decisions.

Probability models uncertainty before observations are made; statistics uses observations to estimate unknown characteristics and test engineering claims. Together they support sensor fusion, reliability prediction, quality control, fault diagnosis, calibration, and risk-based design.

Learning Objectives

By the end of this chapter, you should be able to:

1. Apply probability axioms, counting methods, conditional probability, independence, and Bayes’ theorem.
2. Distinguish discrete and continuous random variables and use probability distributions.
3. Calculate expectation, variance, covariance, and correlation.
4. Select and apply binomial, Poisson, normal, exponential, and related models.
5. Estimate population parameters and construct confidence intervals.
6. Perform and interpret hypothesis tests for engineering data.
7. Apply statistical methods to noise, reliability, sensors, and quality decisions.
8. Recognize assumptions, limitations, ethical issues, and misuse of statistical evidence.

1. Probability as an Engineering Model

A random experiment has an outcome that cannot be predicted with certainty, even when the experimental procedure is known. The sample space, written as Ω, is the set of all possible outcomes. An event A is a subset of Ω. The probability P(A) satisfies:

for mutually exclusive events A and B. For arbitrary events,

The subtraction prevents the intersection from being counted twice.

The complement of A is the event that A does not occur:

For equally likely outcomes, probability may be calculated as the ratio of favourable outcomes to total outcomes. This model is inappropriate when outcomes are not equally likely or when dependence is ignored.

Conditional Probability and Independence

The conditional probability of A given B, where P(B) > 0, is

Rearranging gives the multiplication rule:

Events A and B are independent when knowledge of B does not change the probability of A:

Equivalently, P(A ∩ B) = P(A)P(B). Independence must be justified physically or statistically; it cannot be assumed merely because events describe different components.

Bayes’ Theorem

If B₁, B₂, …, Bₙ form mutually exclusive and exhaustive events, then

The prior probability P(Bᵢ) represents knowledge before observing A. The likelihood P(A|Bᵢ) describes how probable the observation is under condition Bᵢ. The posterior probability P(Bᵢ|A) is the updated probability.

Worked Example 1: Sensor Alarm and Bayes’ Theorem

Suppose 2% of industrial motors have a developing bearing fault. A vibration alarm detects a fault with probability 0.95, but it also alarms falsely for a healthy motor with probability 0.08. Let F denote a fault and A an alarm.

The numerator is 0.019. The denominator is 0.0974, so

Thus, only about 19.5% of alarms correspond to actual faults under these operating conditions. The result does not mean the sensor is useless. It means that low fault prevalence and false alarms must be considered when designing the inspection response.

2. Random Variables and Distributions

A random variable X assigns a numerical value to each outcome. A discrete random variable takes countable values, such as the number of defective gears. A continuous random variable can take values in an interval, such as temperature or vibration amplitude.

For a discrete variable, the probability mass function is p(x) = P(X = x), with

For a continuous variable, the probability density function f(x) satisfies

and ∫_−∞^∞ f(x) dx = 1. For a continuous variable, P(X = x) = 0; probabilities are assigned to intervals.

The cumulative distribution function is

It is defined for both discrete and continuous variables.

Expected Value and Variance

For a discrete random variable,

For a continuous random variable,

The variance measures spread about the mean:

The standard deviation is σ = √σ² and has the same units as X. For a linear transformation Y = aX + b,

The constant b changes the mean but not the variance; scaling by a changes the standard deviation by |a|.

3. Common Engineering Distributions

Bernoulli and Binomial Distributions

A Bernoulli trial has two outcomes, conventionally success and failure, with success probability p. If X counts successes in n independent trials with constant p, then X has a binomial distribution:

where C(n,k) = n! ÷ [k!(n−k)!]. Its mean and variance are np and np(1 − p), respectively. The assumptions are fixed n, two outcomes per trial, constant p, and independent trials.

Poisson Distribution

The Poisson model describes counts of relatively rare events in a fixed interval when events occur independently at an approximately constant average rate λ:

Both mean and variance equal λ. Examples include defects per metre of cable, controller faults per month, or arrivals at a service station. If the event rate varies substantially with time, a homogeneous Poisson model is unsuitable.

Uniform Distribution

If all values between a and b are equally likely,

The mean is (a + b) ÷ 2 and the variance is (b − a)² ÷ 12. It is often used for bounded manufacturing tolerances when no value within the interval is more likely than another.

Normal Distribution

The normal distribution has density

where μ is the mean and σ > 0 is the standard deviation. It is symmetric and widely useful because sums of many small, approximately independent effects tend toward normality. Standardization gives

The standard normal variable Z has mean 0 and variance 1. The empirical rule states that approximately 68%, 95%, and 99.7% of normal observations lie within 1, 2, and 3 standard deviations of the mean, respectively. These percentages are approximations, not universal laws.

Exponential Distribution

The exponential distribution models a waiting time to the next event in a Poisson process:

Its mean is 1 ÷ λ and variance is 1 ÷ λ². It has the memoryless property:

This property is appropriate for certain random-failure mechanisms, but not for components whose failure risk increases with age.

4. Joint Random Variables and Dependence

For two variables X and Y, covariance is

Positive covariance indicates that large values tend to occur together; negative covariance indicates an inverse tendency. Covariance depends on units, so correlation is often easier to compare:

For σₓ, σᵧ > 0, −1 ≤ ρₓᵧ ≤ 1. Correlation measures linear association, not causation. A correlation near zero does not exclude nonlinear dependence.

For sums,

If X and Y are independent, covariance is zero and the final term disappears. For a weighted sensor fusion estimate Y = aX + bZ,

Ignoring correlated sensor errors can substantially underestimate uncertainty.

5. Reliability and Engineering Noise

Reliability R(t) is the probability that a component performs its required function without failure up to time t:

where T is the failure time. The failure density is f(t), and the hazard or failure rate is

For an exponential lifetime with constant rate λ,

For independent components in series, the system works only if every component works:

For independent components in parallel, the system works if at least one works:

These equations require correctly defined mission success and independent component failures. Common-cause failures violate the simple formulas.

Sensor noise is commonly summarized by mean, variance, root-mean-square value, and spectral density. If independent zero-mean noise sources are added, their variances add. Averaging n independent measurements with common variance σ² produces a sample mean with variance σ² ÷ n and standard deviation σ ÷ √n. Averaging does not remove bias and is less effective when samples are correlated.

6. Descriptive Statistics and Sampling

A population is the complete set of units of interest; a sample is the observed subset. For measurements x₁, …, xₙ, the sample mean is

The unbiased sample variance is

The divisor n − 1 corrects the tendency of the divisor-n expression to underestimate population variance when the mean is estimated from the same data. The standard error of the mean is s ÷ √n when observations are independent and identically distributed.

Sampling must represent the intended population. Convenience samples can produce selection bias; repeated measurements from one machine may not represent variation between machines. Randomization, blocking, replication, and traceable measurement procedures improve validity.

7. Estimation and Confidence Intervals

A point estimator is a statistic used to estimate a population parameter. For a normal population with known σ, a confidence interval for μ is

where 1 − α is the confidence level and z_{α/2} satisfies P(−z_{α/2} ≤ Z ≤ z_{α/2}) = 1 − α.

When σ is unknown and the population is approximately normal, use the t distribution:

The degrees of freedom are n − 1. A 95% confidence interval means that the procedure captures the fixed population mean in 95% of repeated samples. It does not strictly mean there is a 95% probability that the already-fixed mean lies in this particular interval.

For a population proportion p, with sample proportion p̂ = x ÷ n, a large-sample interval is approximately

This approximation requires sufficiently large expected counts; exact or improved intervals are preferable for small samples or proportions near zero or one.

Worked Example 2: Calibration Uncertainty

Eight independent measurements of a reference displacement have mean x̄ = 10.24 mm and sample standard deviation s = 0.16 mm. A 95% interval for the true mean uses t₀.₀₂₅,₇ ≈ 2.365:

The margin is approximately 0.134 mm, giving

Thus the estimated mean reference reading is 10.24 mm, with a 95% confidence interval from approximately 10.106 mm to 10.374 mm. The interval describes uncertainty in the mean, not the range of individual future readings.

8. Hypothesis Testing

A hypothesis test compares a null hypothesis H₀ with an alternative hypothesis H₁. For example, a calibration engineer may test H₀: μ = 10 mm against H₁: μ ≠ 10 mm. The significance level α is the probability of rejecting H₀ when H₀ is true, known as a Type I error.

A Type II error occurs when H₀ is not rejected even though H₁ is true. Test power is 1 − β, where β is the Type II error probability. Power increases with larger samples, larger true effects, lower measurement variation, and larger α.

For a known σ, the one-sample z statistic is

For unknown σ under approximate normality, use

A p-value is the probability, assuming H₀ is true, of obtaining a result at least as inconsistent with H₀ as the observed result. A small p-value is evidence against H₀; it is not the probability that H₀ is true, nor does it measure practical importance.

For two independent means, assumptions include independence between groups, appropriate sampling, and either approximate normality or sufficiently large samples. For paired measurements, analyze the differences within each pair. For proportions and counts, binomial or contingency-table methods may be appropriate. Multiple testing increases the chance of false discoveries and may require correction.

Statistical significance must be distinguished from engineering significance. A difference of 0.01 mm may be statistically detectable in a very large sample but irrelevant to system performance; conversely, a practically important difference may not be detected in a small noisy sample.

9. Quality Decisions and Process Control

A specification interval defines acceptable product performance, such as 49.5 mm ≤ diameter ≤ 50.5 mm. Control limits are statistically calculated limits used to detect unusual process behaviour. Specifications and control limits serve different purposes: a stable process can be consistently off-target, and an unstable process can occasionally produce acceptable products.

For a stable normal process with mean μ and standard deviation σ, the fraction outside two-sided specifications [L, U] is

calculated using the normal distribution. Capability indices summarize centering and spread:

and

where USL and LSL are upper and lower specification limits. These indices assume a stable process and meaningful estimates of μ and σ. They do not prove that the process is stable or that the data are normal.

10. Detailed Applied Case: False Alarms in a Robotic Drive

Background

An automated assembly cell uses a vibration sensor to identify bearing faults in robot joints. Unnecessary maintenance stops production, while missed faults can cause costly downtime.

Problem

Historical evidence estimates that 3% of joints inspected during routine operation have a developing fault. The alarm sensitivity is 0.90 and the false-alarm probability is 0.06. Should every alarm trigger immediate replacement?

Analysis

Let F be a fault and A an alarm:

The numerator is 0.027 and the denominator is 0.0852, giving P(F|A) ≈ 0.317. Approximately 31.7% of alarms are expected to correspond to faults under these conditions.

The probability of a missed fault is P(F ∩ Aᶜ) = 0.03 × 0.10 = 0.003, or 0.3% of all inspected joints. The probability of a false alarm is 0.97 × 0.06 = 0.0582, or 5.82% of all inspected joints.

Approach and Solution

The engineering team adopts a two-stage policy: an alarm initiates a confirmatory measurement and reduced-speed operation; replacement occurs only if the second test and trend analysis support the diagnosis. The policy also records operating temperature and load, because these variables influence vibration and may invalidate a constant false-alarm rate.

Outcome

The policy reduces unnecessary replacements while preserving rapid response to credible faults. Bayesian probabilities are recalculated periodically as fault prevalence, sensor condition, and operating environments change.

Learning Points

1. A reliable detector can have a modest positive predictive value when faults are rare.
2. Decision thresholds should include the costs of false positives and false negatives.
3. Probabilities are conditional on the population, measurement procedure, and operating context.
4. Maintenance data can be used to update priors, but biased records can produce misleading conclusions.

11. Limitations, Ethics, and Contemporary Practice

Statistical models are approximations. Non-independence, drift, non-normal data, missing observations, outliers, selection bias, and measurement-system error can invalidate conclusions. Before modelling, engineers should inspect plots, verify units, document sampling, and understand the physical mechanism.

Measurement uncertainty should not be hidden by excessive decimal places. Engineers must distinguish random error from systematic bias, validate sensors against traceable standards, and preserve calibration records. In safety-critical systems, statistical evidence supports but does not replace hazard analysis, verification, redundancy, and applicable regulatory requirements.

AI and machine learning systems increasingly estimate failure probabilities from high-dimensional sensor streams. Their outputs remain probabilistic predictions, not guaranteed diagnoses. Good practice includes representative training data, separate validation data, calibration of predicted probabilities, monitoring for distribution shift, explainable decision thresholds, cybersecurity protection, and human oversight. Automated decisions should be auditable, especially when they affect safety or maintenance access.

Best-Practice Workflow

1. Define the engineering question and decision consequence.
2. Define the population, sampling unit, response variable, and measurement system.
3. Check calibration, resolution, bias, missingness, and independence.
4. Select a distribution or statistical method justified by physics and data.
5. State assumptions before calculating.
6. Report estimates with uncertainty and units.
7. Separate statistical significance from engineering importance.
8. Validate predictions using new or independent data.
9. Monitor model performance as the process and environment change.
10. Document decisions, thresholds, limitations, and responsible personnel.

Chapter Summary

Probability provides axioms and rules for uncertain events, including conditional probability, independence, and Bayes’ theorem. Random variables represent uncertain engineering quantities through probability mass functions, density functions, and cumulative distributions. Expectation describes central tendency; variance and standard deviation describe dispersion; covariance and correlation describe dependence.

The Bernoulli, binomial, Poisson, uniform, normal, and exponential distributions model common counting, measurement, and lifetime problems. Reliability is expressed through survival probability and hazard rate, while sensor uncertainty can be analysed by combining variances and accounting for correlation.

Sample means, variances, confidence intervals, and hypothesis tests enable engineers to infer population properties from data. Correct interpretation requires attention to sampling, assumptions, Type I and Type II errors, statistical power, practical significance, and measurement quality. In reliability and quality engineering, probability supports maintenance policies, process capability, and risk-based decisions. Ethical practice requires transparent assumptions, traceable measurements, protection against biased data, and cautious use of AI-based predictions.

Key Terms

Probability: Numerical measure of the likelihood of an event.

Sample space: Set of all possible outcomes.

Event: Subset of a sample space.

Conditional probability: Probability of one event given that another has occurred.

Independence: Condition in which knowledge of one event does not alter the probability of another.

Bayes’ theorem: Rule for updating probabilities using new evidence.

Random variable: Numerical function assigning values to outcomes.

Probability mass function: Probability function for a discrete random variable.

Probability density function: Function whose interval integral gives probability for a continuous variable.

Expected value: Probability-weighted mean of a random variable.

Variance: Expected squared deviation from the mean.

Covariance: Measure of joint variation between two variables.

Correlation: Standardized measure of linear association.

Reliability: Probability that a system performs its required function for a specified time.

Hazard rate: Instantaneous failure rate conditional on survival to a specified time.

Estimator: Sample statistic used to estimate a population parameter.

Confidence interval: Interval produced by a procedure designed to capture a parameter at a stated long-run rate.

Null hypothesis: Claim tested as a reference assumption.

p-value: Probability, under the null hypothesis, of data at least as extreme as observed.

Process capability: Comparison of process variation and centering with specification limits.

Review Questions

Short Answer Questions

1. Distinguish a discrete random variable from a continuous random variable and give one mechatronics example of each.
2. State the multiplication rule for two events and explain when it simplifies.
3. What is the engineering meaning of covariance between two sensor errors?
4. Distinguish a confidence interval from a prediction interval.
5. Define Type I error and Type II error.

Long Answer Questions

1. Explain Bayes’ theorem and discuss why a highly sensitive fault detector can still produce many false alarms.
2. Compare the binomial, Poisson, normal, and exponential distributions, including their assumptions and engineering applications.
3. Derive the variance of the mean of n independent measurements with common variance σ².
4. Explain series and parallel reliability models and identify conditions under which their simple formulas fail.
5. Discuss the difference between statistical significance, practical significance, and engineering risk.

Analytical / Application Questions

1. A sensor has sensitivity 0.92, false-alarm probability 0.05, and fault prevalence 0.01. Calculate the probability that an alarm indicates a real fault. Interpret the result for maintenance planning.
2. Sixteen independent calibration readings have x̄ = 25.10 V and s = 0.08 V. Construct a 95% t-based confidence interval for the true mean, using t₀.₀₂₅,₁₅ = 2.131.
3. Two independent components have reliabilities 0.96 and 0.98 over a mission. Calculate the reliability of a series arrangement and a parallel arrangement, and explain which arrangement is preferable when both components are required for operation.

Multiple Choice Questions with Answers

Question 1

If P(A) = 0.2 and P(B|A) = 0.5, what is P(A ∩ B)?

A. 0.10

B. 0.20

C. 0.40

D. 0.70

Correct Answer: A. 0.10

Question 2

A fault has prior probability 0.02. A test has sensitivity 0.95 and false-alarm probability 0.08. Approximately what is P(fault | alarm)?

A. 0.019

B. 0.195

C. 0.800

D. 0.950

Correct Answer: B. 0.195

Question 3

For a binomial random variable with n = 20 and p = 0.3, the mean is:

A. 0.3

B. 6

C. 14

D. 20

Correct Answer: B. 6

Question 4

If X has variance 4, what is the variance of Y = 3X + 2?

A. 6

B. 8

C. 12

D. 36

Correct Answer: D. 36

Question 5

Which distribution is commonly used for the waiting time between events in a constant-rate Poisson process?

A. Binomial

B. Normal

C. Exponential

D. Uniform

Correct Answer: C. Exponential

Question 6

For independent measurements with variance σ², the variance of their mean for n observations is:

A. σ²n

B. σ² ÷ n

C. σ ÷ n

D. n ÷ σ²

Correct Answer: B. σ² ÷ n

Question 7

A 95% confidence interval means that:

A. There is a 95% probability the fixed parameter changes

B. 95% of individual observations equal the sample mean

C. The interval-generating procedure captures the parameter in about 95% of repeated samples

D. The null hypothesis is true with probability 0.95

Correct Answer: C. The interval-generating procedure captures the parameter in about 95% of repeated samples

Question 8

A Type I error occurs when an engineer:

A. Fails to reject a false null hypothesis

B. Rejects a true null hypothesis

C. Uses a large sample

D. Reports a confidence interval

Correct Answer: B. Rejects a true null hypothesis

Question 9

For independent components in series, system reliability is:

A. The sum of component reliabilities

B. The product of component reliabilities

C. One minus the product of reliabilities

D. Always greater than each component reliability

Correct Answer: B. The product of component reliabilities

Question 10

A correlation coefficient near zero indicates:

A. Perfect negative dependence

B. No possible relationship of any kind

C. Little or no linear association, without proving independence

D. Equal means and variances

Correct Answer: C. Little or no linear association, without proving independence