Probability, Random Processes and Uncertainty

Introduction

Mechatronic systems operate with incomplete information. A sensor does not report the exact physical state; it reports a measurement affected by noise, bias, quantisation, drift, environmental influences and modelling error. Manufacturing dimensions vary, component failures occur unpredictably, and control systems must make decisions from finite and uncertain data. Probability and statistics provide the mathematical language for representing this uncertainty and supporting reliable engineering decisions.

This chapter develops the principal tools needed to analyse uncertain measurements and stochastic signals: probability distributions, expectation, covariance, estimation, confidence intervals, random processes and basic reliability models. The emphasis is practical. The methods are applied to sensor calibration, quality control, signal filtering, fault detection and risk assessment.

Learning Objectives

By the end of this chapter, you should be able to:

1. Model engineering uncertainty using probability events and random variables.
2. Select and interpret common discrete and continuous probability distributions.
3. Calculate expectation, variance, covariance and correlation.
4. Estimate unknown means and variances from measurement data.
5. Construct and interpret confidence intervals.
6. Describe stochastic signals using autocorrelation, power spectral density and stationarity.
7. Apply basic reliability and failure-probability models.
8. Distinguish random error, systematic error, uncertainty and risk.
9. Use statistical evidence responsibly in engineering decisions.

1. Probability as an Engineering Model

1.1 Outcomes, events and probability

A random experiment has an outcome that cannot be predicted with certainty before the experiment, although its possible outcomes may be known. The sample space, denoted by Ω, is the set of all possible outcomes. An event A is a subset of Ω. The probability P(A) satisfies

when A and B are mutually exclusive. For general events,

The intersection A ∩ B means that both events occur; the union A ∪ B means that at least one occurs.

Conditional probability describes the probability of A when B is known to have occurred:

Two events are independent if knowledge of one does not alter the probability of the other:

Independence is a modelling assumption, not a consequence of events merely being physically different. For example, two independently manufactured resistors may be approximately independent, whereas two temperature readings taken from the same drifting sensor are usually correlated.

Bayes' theorem updates a prior probability using evidence:

In fault diagnosis, A may represent a failed bearing and B an observed vibration alarm. The theorem is valuable because P(B | A), the probability of an alarm given failure, is not generally equal to P(A | B), the probability of failure given an alarm.

1.2 Random variables

A random variable assigns a numerical value to each outcome. A discrete random variable takes countable values, such as the number of defective products in a batch. A continuous random variable can take values in an interval, such as temperature or shaft diameter.

For a discrete variable X, the probability mass function is p_X(x) = P(X = x). For a continuous variable, the probability density function f_X(x) satisfies

and

A density is not itself a probability at a single point; for a continuous variable, P(X = x) = 0. The cumulative distribution function is

and is valid for both discrete and continuous variables.

2. Important Probability Distributions

The choice of distribution should follow the physical mechanism and data, not convenience alone.

2.1 Bernoulli and binomial distributions

A Bernoulli variable represents one trial with success probability p and failure probability 1 − p. If X is the number of successes in n independent Bernoulli trials, then X has a binomial distribution:

where C(n,k) = n! ÷ [k!(n−k)!]. Its mean and variance are

The binomial model applies when the number of trials is fixed, each trial has two outcomes, p is constant, and trials are independent. A production inspection may use it to model the number of defective units in a sample, although dependence caused by a common machine setting can invalidate the model.

2.2 Poisson distribution

The Poisson distribution models counts of events in a fixed interval when events occur independently at an approximately constant average rate λ:

For a Poisson variable,

Examples include the number of communication errors per minute or failures occurring in a specified operating interval. The equality of mean and variance should be checked against data; substantial overdispersion indicates that a Poisson model may be inadequate.

2.3 Uniform distribution

If every value in [a,b] is equally likely,

with

A uniform model is appropriate when only bounds are known and no value within the interval is favoured. It is often used for quantisation error under the assumption that the signal is not synchronised with the quantiser.

2.4 Normal distribution

The normal distribution is specified by mean μ and variance σ²:

The standardised variable is Z = (X − μ) ÷ σ, which has a standard normal distribution. Approximately 68.3%, 95.4% and 99.7% of observations lie within 1, 2 and 3 standard deviations of μ, respectively, when the normal model is appropriate.

Normal models arise from the combined effect of many small, approximately independent influences, consistent with the central limit theorem. They are not automatically justified by a symmetric-looking histogram. Saturation, outliers, bounded variables and multimodal data require further investigation.

2.5 Exponential distribution

The exponential distribution models a waiting time T between events in a Poisson process:

Its mean is 1 ÷ λ and its variance is 1 ÷ λ². It has the memoryless property P(T > s + t | T > s) = P(T > t), which is useful for constant-hazard components but inappropriate for systems whose failure rate changes with age.

3. Expectation, Variance and Covariance

For a discrete random variable,

For a continuous random variable,

The expectation is a probability-weighted average, not necessarily a value that will actually occur. For a function g(X),

The variance measures spread about the mean:

where μ_X = E[X]. The standard deviation is σ_X = √Var(X), and has the same units as X. Variance has squared units.

For constants a and b,

If X and Y are random variables, covariance is

The correlation coefficient is

Correlation is dimensionless and measures linear association. Zero correlation does not generally imply independence, although independence does imply zero covariance when the relevant moments exist.

For a sum S = X + Y,

If X and Y are independent, the covariance term is zero. Assuming independence when sensor errors share a power supply, temperature, or mechanical source can therefore underestimate uncertainty.

Worked Example 1: Combining sensor uncertainty

Two independent sensors estimate a displacement. Their errors have standard deviations 0.20 mm and 0.15 mm. If the combined estimate is the sum of the two measured contributions, the variance is

Therefore,

This result depends on independence and on treating the stated standard deviations as appropriate representations of random error. A shared bias would not be removed by this calculation.

4. Measurement Uncertainty and Error Propagation

Measurement error is the difference between a measured value and the measurand's true or reference value. Random error varies between repeated observations. Systematic error, or bias, tends to remain consistent under specified conditions. Calibration can estimate and correct bias, but it cannot eliminate all uncertainty.

For a measurement model y = g(x₁,…,x_n), with small uncertainties and approximately linear behaviour, the first-order propagation formula is

Here u_i is the standard uncertainty of input x_i. The covariance terms must be included when inputs are correlated. If inputs are independent, those terms vanish.

For y = ax, u_y = |a|u_x. For y = x + z with independent inputs, u_y = √(u_x² + u_z²). The formula is a local approximation and may be unreliable for strongly nonlinear functions, large uncertainties or non-normal inputs. Monte Carlo propagation can then be preferable.

A useful reporting practice is to state the estimate, uncertainty, coverage convention and units, for example 25.40 ± 0.08 mm, where the meaning of ± must be specified—often a standard uncertainty or an approximately 95% expanded uncertainty.

5. Estimation from Data

Suppose x₁,…,x_n are observations of a common quantity. The sample mean is

It is an unbiased estimator of μ when observations are independent and identically distributed with finite mean. The sample variance is

The denominator n − 1, rather than n, provides an unbiased estimator of the population variance under the usual random-sampling model. The standard error of the mean is

This describes uncertainty in the estimated mean, not the spread of individual observations. Increasing n reduces standard error approximately in proportion to 1 ÷ √n, but does not remove systematic bias.

An estimator should be assessed for bias, variance, consistency and robustness. A precise estimator can still be inaccurate if the measurement system is biased. Before estimation, inspect time order, outliers, missing data, instrument resolution and independence.

Confidence intervals

A confidence interval is a procedure that produces an interval with a stated long-run coverage probability. It is not correct to say that a fixed 95% interval has a 95% probability of containing μ after it has been calculated; under the frequentist interpretation, μ is fixed and the procedure either contains it or does not.

For a normal population with known σ, a 100(1 − α)% confidence interval for μ is

where z_(1−α÷2) is the corresponding standard-normal quantile. When σ is unknown and the sample is reasonably normal, use the Student t quantile:

For a 95% interval, α = 0.05. With n = 16, x̄ = 10.2 V, s = 0.8 V and t_(0.975,15) ≈ 2.131,

Thus the interval is approximately 10.2 ± 0.426 V, or [9.774, 10.626] V. This interval quantifies uncertainty in the mean under the stated assumptions; it does not imply that 95% of individual voltage readings lie in the interval.

For proportions, regression coefficients, variances and non-normal data, alternative intervals or resampling methods may be required. Confidence intervals should be accompanied by sample size, method, assumptions and practical interpretation.

6. Random Processes and Stochastic Signals

A random process is a collection of random variables indexed by time, written X(t) for continuous time or X_k for discrete time. A sensor output may contain a deterministic signal s(t) plus random noise n(t):

A process is strictly stationary if the joint probability distribution is unchanged by a shift in time. Wide-sense stationarity requires only constant mean and an autocovariance that depends on lag τ rather than absolute time:

The autocovariance is C_X(τ) = R_X(τ) − μ_X². A process is ergodic, in a relevant sense, when time averages from one sufficiently long record can estimate ensemble averages. Stationarity and ergodicity are different properties.

The autocorrelation indicates how related a signal is to a delayed version of itself. White noise is an idealisation with zero correlation at nonzero lags and a flat power spectral density. Real sensors have bandwidth limits, so actual noise is usually coloured rather than perfectly white.

The power spectral density (PSD), denoted S_X(f), describes how mean-square signal power is distributed over frequency f. For a wide-sense stationary process, the Wiener–Khinchin relationship connects autocorrelation and PSD through the Fourier transform:

The exact transform convention must be stated because factors of 2π differ between angular-frequency and ordinary-frequency forms. In practice, PSD estimates reveal whether noise is concentrated at low frequency, associated with vibration harmonics, or approximately broadband.

Sampling introduces further considerations. If a continuous signal contains frequencies above half the sampling frequency, aliasing can make them appear as lower frequencies. An anti-aliasing low-pass filter should therefore precede an analogue-to-digital converter. Quantisation error may be modelled as approximately uniform over ±Δ ÷ 2, where Δ is the quantisation step, only when overload is absent and signal/error correlation is negligible.

7. Filtering and Evidence from Noisy Sensors

A moving average of m observations is

If the measurement errors are independent with variance σ², the variance of the average is σ² ÷ m. The filter reduces random noise but introduces delay and can blur rapid changes. For correlated noise, the variance reduction is smaller and must be calculated using covariance terms.

The Kalman filter is a recursive estimator for systems represented by a state-space model. A common linear form is

where x_k is the state, u_k is a known input, z_k is a measurement, A, B and H are model matrices, and w_k and v_k are process and measurement noise. In the standard formulation, these noises are zero-mean, mutually independent and often modelled as Gaussian with known covariance matrices. The filter alternates between prediction from the model and correction using measurement residuals. Its performance depends strongly on model adequacy and covariance tuning; it is not a substitute for diagnosing biased sensors or unmodelled faults.

8. Reliability and Risk

Reliability R(t) is the probability that an item performs its required function for at least time t under stated conditions:

where T is failure time. The cumulative failure distribution is F(t) = P(T ≤ t) = 1 − R(t). The hazard rate is

where f_T(t) is the failure-time density. Hazard is an instantaneous conditional failure rate, not simply the probability of failure over a finite interval.

For a constant hazard λ, the exponential model gives

For independent components in series, the system works only if every component works:

For independent parallel components, the system works if at least one works:

These formulas assume clearly defined component success states and statistical independence. Common-cause failures, maintenance, load sharing and standby behaviour require more detailed models such as fault trees or Markov models.

Risk combines the likelihood of an undesirable event with its consequences. A simple expected-loss model is

Expected loss alone is insufficient for catastrophic low-probability events, where safety constraints, legal requirements and risk tolerability criteria are also essential.

9. Illustrative Case: Robotic Assembly Force Sensor

Background

A robotic assembly cell presses a bearing into a housing. The controller uses a force sensor to detect seating. Excessive force damages the bearing; insufficient force produces an unreliable assembly. The sensor has random noise and a possible calibration bias.

Problem

During a controlled 100 N reference test, 25 readings have sample mean 100.8 N and sample standard deviation 1.5 N. The engineering team must determine whether the sensor's mean output is consistent with the reference and whether a simple noise-reduction method is justified.

Analysis

The standard error is

Using t_(0.975,24) ≈ 2.064, the 95% confidence interval for the mean is

The interval is approximately [100.181, 101.419] N. Since 100 N is below the interval, the data provide evidence of a positive mean bias under the test conditions. This conclusion does not prove that every reading is biased by exactly 0.8 N, nor does it establish performance at other temperatures or loads.

If five independent readings are averaged during operation, the random component of the standard deviation would ideally become

However, averaging does not reduce the estimated calibration bias. It also introduces delay, which may be unsafe if force rises rapidly. A suitable solution is therefore to calibrate the sensor, characterize temperature dependence, use a short validated filter, and verify the complete detection system with false-acceptance and false-rejection tests.

Outcome and learning points

The evidence supports calibration before deployment. Statistical significance alone is not the complete engineering decision: the bias must be compared with the allowable force error, dynamic response requirements and safety consequences. Test records should preserve raw data, environmental conditions, calibration traceability and the decision rule used.

10. Limitations, Governance and Best Practice

Statistical conclusions can fail through selection bias, pseudoreplication, hidden dependence, data leakage, inappropriate distributions, uncontrolled confounding and unreported exclusions. A large dataset cannot compensate for a biased measurement process. Engineers should plot data against time, inspect residuals, check units and document assumptions.

Measurement systems should be calibrated against traceable references, with calibration intervals based on stability and risk. Uncertainty budgets should distinguish repeatability, resolution, calibration, environmental effects and model assumptions. Safety-related decisions should use conservative acceptance criteria and independent review where required by applicable standards.

AI and machine-learning systems also depend on probabilistic evidence. Training and test data must be separated, uncertainty and out-of-distribution behaviour should be monitored, and a high-confidence prediction should not be confused with a safe physical decision. Automated anomaly detection requires validation under realistic operating conditions, including sensor faults and changing environments.

Best practice is to: define the measurand and decision before collecting data; select a model connected to the physical process; preserve raw observations; quantify uncertainty; test independence and stationarity where relevant; report confidence intervals rather than unsupported certainty; distinguish statistical from practical significance; and validate the complete mechatronic system rather than only its software.

Chapter Summary

Probability models uncertain outcomes and provides rules for combining evidence. Random variables may be discrete or continuous and are described by mass functions, density functions and cumulative distribution functions. Bernoulli, binomial, Poisson, uniform, normal and exponential distributions are useful engineering models, but each has assumptions.

Expectation describes a probability-weighted average, variance describes spread, and covariance describes joint variation. Propagation of uncertainty must include covariance when inputs are related. Sample means and variances estimate population properties, while standard errors quantify uncertainty in estimated means. Confidence intervals describe the long-run performance of an interval-producing procedure and must not be confused with the spread of individual observations.

Random processes extend probability into time. Stationarity, autocorrelation and power spectral density help engineers understand stochastic sensor signals, sampling and filtering. Reliability uses survival probability and hazard rate to analyse failures; series and parallel formulas apply only under appropriate independence and system-structure assumptions. In professional practice, statistical evidence must be combined with calibration, physical knowledge, safety requirements, traceability and sound engineering judgement.

Key Terms

Random experiment: A process with an uncertain outcome.

Sample space: The set of all possible outcomes.

Event: A subset of the sample space.

Conditional probability: Probability of an event given specified information.

Bayes' theorem: A rule for updating probabilities using evidence.

Random variable: A numerical function of a random outcome.

Probability mass function: Probabilities assigned to values of a discrete variable.

Probability density function: A function whose integral over an interval gives continuous probability.

Expectation: The probability-weighted mean of a random variable.

Variance: The expected squared deviation from the mean.

Covariance: A measure of joint variation between two variables.

Correlation: A dimensionless measure of linear association.

Estimator: A rule for calculating an unknown parameter from data.

Standard error: The standard deviation of an estimator's sampling distribution.

Confidence interval: An interval produced by a procedure with specified long-run coverage.

Random process: A time-indexed collection of random variables.

Autocorrelation: Similarity between a process and a time-shifted version of itself.

Power spectral density: Distribution of signal power over frequency.

Reliability: Probability that an item performs its required function for a specified time.

Hazard rate: Conditional instantaneous rate of failure.

Review Questions

Short Answer Questions

1. Distinguish a random error from a systematic error in a sensor measurement.
2. State the conditions under which the binomial distribution is appropriate.
3. Why does covariance matter when combining measurement uncertainties?
4. Explain the difference between standard deviation and standard error.
5. What does reliability R(t) represent?

Long Answer Questions

1. Explain the normal, Poisson and exponential distributions, including one mechatronics application and one limitation for each.
2. Derive the variance of the sum of two random variables and explain the special case of independent variables.
3. Describe how a 95% confidence interval for an unknown mean is constructed when the population standard deviation is unknown.
4. Explain stationarity, autocorrelation and power spectral density in the analysis of a noisy sensor signal.
5. Compare series and parallel reliability structures and state the assumptions behind their standard formulas.

Analytical / Application Questions

1. A temperature sensor produces 20 readings with mean 24.6 °C and standard deviation 0.9 °C. Explain how you would construct a 95% confidence interval for the mean, identify the required quantile, and distinguish this interval from the range of individual readings.
2. Two sensor errors have standard deviations 0.4 mm and 0.3 mm and covariance 0.06 mm². Calculate the variance and standard deviation of their sum, and explain how the result would change if covariance were incorrectly ignored.
3. A safety controller uses two independent protection channels, each with reliability 0.98 over one year. Calculate the reliability of a series arrangement and a parallel arrangement, then discuss why common-cause failures could make both calculations optimistic.

Multiple Choice Questions with Answers

Question 1

Which expression defines conditional probability for P(B) > 0?

A. P(A | B) = P(A)P(B)

B. P(A | B) = P(A ∩ B) ÷ P(B)

C. P(A | B) = P(A ∪ B) ÷ P(A)

D. P(A | B) = 1 − P(B)

Correct Answer: B. P(A | B) = P(A ∩ B) ÷ P(B)

Question 2

A binomial random variable has n = 10 and p = 0.2. What is its expected value?

A. 0.02

B. 2

C. 5

D. 10.2

Correct Answer: B. 2

Question 3

If X and Y are independent, which statement is correct?

A. Cov(X,Y) = 1

B. Var(X + Y) = Var(X) − Var(Y)

C. Cov(X,Y) = 0

D. ρ_XY = −1

Correct Answer: C. Cov(X,Y) = 0

Question 4

Twenty-five independent readings have sample standard deviation 1.5 N. What is the standard error of their mean?

A. 0.06 N

B. 0.30 N

C. 1.50 N

D. 7.50 N

Correct Answer: B. 0.30 N

Question 5

Which statement about a 95% frequentist confidence interval is correct?

A. There is a 95% probability that the fixed parameter is inside this already calculated interval.

B. 95% of individual observations must lie inside the interval.

C. The interval-producing procedure covers the true parameter in 95% of repeated samples under its assumptions.

D. The interval eliminates systematic measurement bias.

Correct Answer: C. The interval-producing procedure covers the true parameter in 95% of repeated samples under its assumptions.

Question 6

For a constant hazard rate λ, which reliability model is appropriate?

A. R(t) = 1 + λt

B. R(t) = λt

C. R(t) = e^(−λt)

D. R(t) = e^(λt)

Correct Answer: C. R(t) = e^(−λt)

Question 7

What does a power spectral density primarily describe?

A. The probability of a single discrete outcome

B. How signal power is distributed over frequency

C. The calibration bias of a sensor

D. The number of components in a system

Correct Answer: B. How signal power is distributed over frequency

Question 8

Two independent components each have reliability 0.98. What is the reliability of a series system?

A. 0.0196

B. 0.0396

C. 0.9604

D. 0.9996

Correct Answer: C. 0.9604

Question 9

Which uncertainty source is not reduced simply by averaging repeated independent readings?

A. Random noise with zero mean

B. Independent quantisation variation

C. Constant calibration bias

D. Independent electrical interference

Correct Answer: C. Constant calibration bias

Question 10

A moving average of m independent measurements, each with variance σ², has what variance?

A. mσ²

B. σ²

C. σ² ÷ m

D. σ² ÷ m²

Correct Answer: C. σ² ÷ m