Stochastic Systems, Estimation and Diagnostics

Introduction

Real control systems operate in the presence of uncertainty. Sensor readings are corrupted by noise, actuators exhibit variability, plant parameters drift, and unmeasured disturbances affect system evolution. A deterministic model may describe nominal behavior, but reliable monitoring and control require a probabilistic description of what is known, what is uncertain, and how evidence should be combined.

Stochastic systems theory provides this description. State-space models are extended with random process and measurement disturbances; covariance matrices quantify uncertainty; estimators infer hidden states from imperfect observations; and diagnostic algorithms identify abnormal behavior through residual analysis. The Kalman filter is the central linear-Gaussian estimator, while its extensions support nonlinear and non-Gaussian applications. In engineering practice, estimation and diagnostics are closely connected: an estimator predicts expected behavior, and deviations between prediction and measurement form the basis of fault detection.

This chapter develops these ideas for postgraduate study, with emphasis on modeling assumptions, covariance propagation, Kalman filtering, sensor fusion, residual-based diagnosis, and industrial condition monitoring.

Learning Objectives

After completing this chapter, the reader should be able to:

1. Model process noise, measurement noise, and uncertainty using stochastic state-space equations.

2. Interpret covariance matrices, correlation, observability, and uncertainty propagation.

3. Derive and implement the discrete-time Kalman filter.

4. Explain the extended and unscented Kalman filters and their limitations.

5. Design residual-based fault-detection schemes using statistical thresholds.

6. Combine heterogeneous sensors through model-based and probabilistic fusion.

7. Evaluate estimator consistency, robustness, cybersecurity, and practical deployment issues.

1. Deterministic Models and the Role of Uncertainty

A discrete-time linear state-space model is commonly written as

\[

x_{k+1}=A_kx_k+B_ku_k+G_kw_k,

\]

\[

y_k=C_kx_k+D_ku_k+v_k.

\]

Here, \(x_k\in\mathbb{R}^n\) is the state, \(u_k\in\mathbb{R}^m\) is the known input, and \(y_k\in\mathbb{R}^p\) is the measurement. Matrices \(A_k\), \(B_k\), \(C_k\), and \(D_k\) describe system dynamics and measurement mapping. The random vector \(w_k\) represents process disturbances, while \(v_k\) represents measurement noise. Matrices \(G_k\) and, implicitly, the measurement-noise covariance determine how these disturbances enter the model.

A standard assumption for the Kalman filter is

\[

E[w_k]=0,\qquad E[v_k]=0,

\]

\[

E[w_kw_j^T]=Q_k\delta_{kj},\qquad E[v_kv_j^T]=R_k\delta_{kj},

\]

and

\[

E[w_kv_j^T]=0.

\]

The Kronecker delta \(\delta_{kj}\) is one when \(k=j\) and zero otherwise. Thus, process and measurement noise are assumed white and mutually uncorrelated. Usually, \(Q_k\succeq0\) and \(R_k\succ0\), where positive semidefiniteness and positive definiteness are required for valid covariance matrices and an invertible innovation covariance.

These assumptions are convenient rather than universal. Real disturbances may be colored, biased, non-Gaussian, time-varying, or correlated with measurements. A strong engineering design therefore treats the stochastic model as an approximation that must be validated from data and revised when evidence contradicts it.

2. Probability, Covariance and Uncertainty Representation

For a random vector \(x\) with mean \(\mu\), covariance is defined by

\[

P=E[(x-\mu)(x-\mu)^T].

\]

The diagonal entries of \(P\) are component variances, and the off-diagonal entries describe cross-covariances. A covariance matrix must be symmetric and positive semidefinite. Its units are not necessarily identical: if one state is position in metres and another is velocity in metres per second, their covariance has units of metre-squared per second.

A Gaussian state model represents uncertainty as

\[

x\sim\mathcal{N}(\mu,P).

\]

For a scalar Gaussian variable, approximately 68.3% of values lie within one standard deviation of the mean and 95.4% within two standard deviations. For multidimensional states, however, componentwise intervals are not identical to probability-containment ellipsoids. The covariance determines the orientation and scale of the ellipsoid through

\[

(x-\mu)^TP^{-1}(x-\mu)=c.

\]

The value of \(c\) must be selected from the appropriate chi-square distribution if a specified joint probability is required.

A linear transformation \(z=Hx+d\), with deterministic \(H\) and \(d\), has

\[

E[z]=H\mu+d,

\qquad

\operatorname{Cov}(z)=HPH^T.

\]

This relation is fundamental to sensor modeling and uncertainty propagation. It also shows why ignoring state correlations can produce incorrect confidence estimates.

3. Stochastic State-Space Models

The state is not necessarily a directly measurable physical variable. It may include position, velocity, temperature, bias, actuator state, or latent degradation variables. The model should be selected so that the state is sufficiently informative for prediction and estimation.

For a continuous-time stochastic model,

\[

\dot{x}(t)=A x(t)+B u(t)+Lw(t),

\]

where \(w(t)\) may be idealized as white noise. In continuous time, white noise is a generalized process, so its interpretation requires care. A common engineering practice is to define a continuous-time spectral-density matrix and compute its discrete-time equivalent over a sampling interval \(\Delta t\). The resulting discrete process covariance is not generally equal to the continuous spectral density; it depends on the dynamics and noise-input matrix.

For time-invariant discrete models, stability of \(A\) affects how uncertainty propagates. Even when the physical plant is stable, poor modeling of \(Q\) or \(R\) can produce an estimator that is sluggish, excessively noisy, or inconsistent. Observability is also essential. The pair \((A,C)\) is observable if the initial state can, in principle, be reconstructed from a finite sequence of noiseless measurements. For an \(n\)-state time-invariant system, the observability matrix is

\[

\mathcal{O}=\begin{bmatrix}C\\CA\\CA^2\\\cdots\\CA^{n-1}\end{bmatrix}.

\]

Full column rank is the standard test. In stochastic estimation, weak observability may not make estimation impossible, but it can produce large uncertainty and slow convergence.

4. The Kalman Filter

The Kalman filter recursively computes the conditional mean and covariance of the state for a linear model with Gaussian disturbances. Under the stated assumptions, the estimate is the minimum mean-square error estimate and is also the conditional maximum-likelihood estimate.

Let \(\hat{x}_{k|k}\) denote the estimate using measurements through time \(k\), and let \(P_{k|k}\) be its error covariance.

4.1 Prediction

The time-update equations are

\[

\hat{x}_{k|k-1}=A_{k-1}\hat{x}_{k-1|k-1}+B_{k-1}u_{k-1},

\]

\[

P_{k|k-1}=A_{k-1}P_{k-1|k-1}A_{k-1}^T+G_{k-1}Q_{k-1}G_{k-1}^T.

\]

The first equation predicts the state mean. The second propagates uncertainty and adds process-noise uncertainty. Omitting the process-noise term incorrectly suggests that uncertainty can decrease indefinitely during prediction.

4.2 Measurement Update

The innovation, or measurement residual, is

\[

\nu_k=y_k-C_k\hat{x}_{k|k-1}-D_ku_k.

\]

Its covariance is

\[

S_k=C_kP_{k|k-1}C_k^T+R_k.

\]

The Kalman gain is

\[

K_k=P_{k|k-1}C_k^TS_k^{-1}.

\]

The state and covariance updates are

\[

\hat{x}_{k|k}=\hat{x}_{k|k-1}+K_k\nu_k,

\]

\[

P_{k|k}=(I-K_kC_k)P_{k|k-1}.

\]

For numerical robustness, the Joseph form is often preferred:

\[

P_{k|k}=(I-K_kC_k)P_{k|k-1}(I-K_kC_k)^T+K_kR_kK_k^T.

\]

The Joseph form better preserves symmetry and positive semidefiniteness under finite-precision arithmetic. In implementation, one should avoid explicitly computing \(S_k^{-1}\); a Cholesky factorization or a linear solver is safer and more efficient.

4.3 Interpretation of the Gain

The gain balances model prediction and measurement information. If \(R_k\) is large relative to predicted uncertainty, the gain is small and the estimate relies mainly on the model. If \(P_{k|k-1}\) is large and the measurement is precise, the gain is larger. This interpretation is valid only when covariance values are correctly scaled and expressed in compatible units.

Worked Example: Scalar Sensor Estimation

Suppose a constant temperature is estimated using a scalar model. Before a measurement, the predicted temperature is 20 °C with variance 4 °C². The sensor reports 23 °C and has noise variance 1 °C². Thus \(C=1\), \(P^-=4\), and \(R=1\).

The innovation is

\[

\nu=23-20=3\ ^\circ\mathrm{C},

\]

and its variance is

\[

S=P^-+R=4+1=5\ ^\circ\mathrm{C}^2.

\]

The gain is \(K=4/5=0.8\). Therefore,

\[

\hat{x}^+=20+0.8(3)=22.4\ ^\circ\mathrm{C},

\]

and

\[

P^+=(1-0.8)4=0.8\ ^\circ\mathrm{C}^2.

\]

The estimate lies closer to the measurement because the sensor is more precise than the prior prediction, but it does not equal the measurement because the prediction also contains useful information.

5. Nonlinear and Advanced Estimation

Many control systems are nonlinear:

\[

x_{k+1}=f(x_k,u_k)+w_k,

\qquad y_k=h(x_k)+v_k.

\]

The extended Kalman filter (EKF) linearizes \(f\) and \(h\) about the current estimate using Jacobian matrices. It is computationally efficient but may be inaccurate or unstable when nonlinearities are strong, the initial estimate is poor, or the posterior distribution is substantially non-Gaussian.

The unscented Kalman filter (UKF) propagates a carefully selected set of sigma points through the nonlinear functions. It avoids explicit Jacobian calculation and often captures mean and covariance transformations more accurately for moderate nonlinearities. It is not universally superior: computational cost, tuning parameters, numerical conditioning, and non-Gaussian distributions remain concerns.

Particle filters represent the posterior using weighted samples and can handle nonlinear, non-Gaussian problems. Their disadvantages include high computational cost, sample impoverishment, and sensitivity to proposal distributions. Choice of estimator should follow model structure, computational constraints, safety requirements, and validation evidence rather than fashion.

6. Covariance Tuning and Filter Consistency

The matrices \(Q\) and \(R\) are often imperfectly known. Increasing \(Q\) makes the filter less confident in the model and generally more responsive to measurements. Increasing \(R\) makes it less confident in the sensor and generally smoother. These effects should not be used as arbitrary knobs without physical justification.

Noise covariances can be estimated from calibration data, repeated experiments, known sensor specifications, or system-identification procedures. Biases should not be hidden inside large white-noise covariances when they can be modeled as additional states. For example, a slowly varying sensor bias may be represented by

\[

b_{k+1}=b_k+\eta_k,

\]

where \(\eta_k\) is a small random walk.

A useful consistency diagnostic is the normalized innovation squared (NIS):

\[

\epsilon_k=\nu_k^TS_k^{-1}\nu_k.

\]

Under correct modeling, Gaussian assumptions, and a measurement dimension of \(p\), \(\epsilon_k\) approximately follows a chi-square distribution with \(p\) degrees of freedom. This statement applies to an individual innovation under appropriate conditions; serial correlation, adaptive tuning, nonlinearity, and model errors complicate interpretation. The normalized estimation error squared (NEES),

\[

\zeta_k=e_k^TP_{k|k}^{-1}e_k,

\qquad e_k=x_k-\hat{x}_{k|k},

\]

requires the true state and is therefore mainly available in simulation or instrumented experiments.

7. Residual-Based Fault Detection and Diagnosis

A fault is an abnormal condition affecting a component, sensor, actuator, communication link, or plant parameter. Detection asks whether behavior is abnormal; isolation identifies the source; identification estimates fault magnitude or type.

The innovation \(\nu_k\) is a natural residual. Under healthy operation, it should have approximately zero mean and covariance \(S_k\). A simple scalar detector uses

\[

|\nu_k|>\gamma,

\]

where \(\gamma\) is selected from a desired false-alarm probability and the expected residual variance. For vector measurements, the NIS is more appropriate:

\[

\epsilon_k>\chi^2_{p,1-\alpha},

\]

where \(\chi^2_{p,1-\alpha}\) is the \((1-\alpha)\)-quantile of the chi-square distribution. The threshold is meaningful only when the residual covariance is credible and the residual is sufficiently close to the assumed distribution.

Single-sample tests may miss small persistent faults. CUSUM, moving-window averages, exponentially weighted statistics, and generalized likelihood-ratio tests accumulate evidence. For example, a windowed statistic can be formed as

\[

J_k=\sum_{i=k-N+1}^{k}\nu_i^TS_i^{-1}\nu_i.

\]

The threshold must account for window length and serial dependence; simply applying a single-sample chi-square threshold to \(J_k\) is generally incorrect.

Residual design can also support isolation. A bank of observers may be configured so that each residual is sensitive to a particular fault and relatively insensitive to others. Structured residuals, parity relations, unknown-input observers, and model-based classifiers are alternatives. Isolation becomes difficult when faults produce similar signatures or when disturbances are indistinguishable from faults.

8. Sensor Fusion

Sensor fusion combines information from sensors with different noise characteristics, sampling rates, coordinate frames, and failure modes. A Kalman filter is a sequential fusion mechanism: each measurement updates a common state estimate according to its uncertainty and correlation with the state.

For independent scalar measurements of the same constant quantity, the posterior precision is additive:

\[

\frac{1}{P^+}=\frac{1}{P^-}+\frac{1}{R_1}+\frac{1}{R_2}+\cdots.

\]

This relation is a special case. It assumes independent measurement errors, a common scalar quantity, and a Gaussian linear model. Correlated sensors require a joint covariance model; treating correlated measurements as independent causes overconfidence.

Practical fusion requires time synchronization, calibration, frame transformation, outlier handling, and management of missing data. A high-rate inertial sensor may provide short-term propagation, while lower-rate positioning or vision sensors correct drift. Sensor confidence should be based on validated uncertainty, not merely nominal data-sheet precision.

9. Condition Monitoring and Predictive Maintenance

Condition monitoring uses measurements to assess equipment health and anticipate failure. Typical signals include vibration, current, temperature, acoustic emission, pressure, and performance residuals. A model-based pipeline may include:

1. acquisition and synchronization;

2. filtering and quality checks;

3. state estimation;

4. residual or feature generation;

5. health-index construction;

6. thresholding or probabilistic classification;

7. maintenance decision and feedback.

Features may be statistical, spectral, time-frequency, or learned. Examples include root-mean-square vibration, kurtosis, spectral-band energy, temperature trends, and innovation statistics. Machine-learning models can classify faults or estimate remaining useful life, but they require representative labeled or weakly labeled data and careful treatment of distribution shift.

AI should complement, not automatically replace, physical reasoning. Physics-informed models, hybrid observers, uncertainty-aware neural networks, and anomaly detectors are promising because they can exploit both known dynamics and data. Safety-critical decisions require traceability, validation, monitoring of model drift, and human accountability.

10. Illustrative Case: Fault Detection in a Motor Drive

Background

Consider an illustrative permanent-magnet motor drive equipped with a speed sensor, phase-current sensors, and a temperature sensor. The controller uses a discrete state model containing rotor speed, load-torque disturbance, and a slowly varying speed-sensor bias.

Problem

A small intermittent speed-sensor fault must be detected without confusing it with legitimate load changes. A fixed threshold on raw speed error produces nuisance alarms because load disturbances cause transient speed deviations.

Analysis

The estimator predicts speed using motor inputs and current measurements. The speed measurement is then incorporated through a Kalman update. The innovation is normalized using its predicted covariance. During healthy operation, a validation data set is used to check the empirical distribution of the NIS and its temporal behavior. The sensor bias is included as a state rather than absorbed into large measurement noise.

Approach

A two-stage detector is implemented. First, each NIS value is compared with a statistical limit to identify abrupt outliers. Second, a moving accumulation of normalized residuals detects persistent moderate deviations. Current and temperature channels provide corroborating evidence. If speed residuals are abnormal while current-based torque estimates remain plausible, the speed sensor is assigned a high fault likelihood. If all channels deviate coherently, a plant or actuator fault is considered instead.

Missing or implausible measurements are gated before the update. When the speed sensor is rejected, the estimator propagates using the motor model and remaining sensors, while a degraded-mode flag is sent to the supervisory controller.

Outcome

The approach distinguishes short load transients from persistent sensor degradation more effectively than a raw-error threshold. Its performance depends on covariance calibration, observability during degraded operation, and timely detection of model changes. It does not prove that a particular component has failed; it produces evidence that must be interpreted within system context.

Learning Points

The case illustrates that diagnosis is an integrated process involving modeling, estimation, statistical testing, redundancy, and operational decision rules. Residual thresholds alone cannot solve ambiguity caused by unknown disturbances or common-mode failures.

11. Challenges, Governance and Best Practices

Important limitations include incorrect noise models, unmodeled biases, nonlinear behavior, time-varying dynamics, numerical instability, weak observability, and environmental changes. Fault detectors face a fundamental trade-off between false alarms and missed detections. Thresholds should be selected with explicit costs, safety requirements, and operating-envelope considerations.

Cybersecurity is increasingly relevant. Spoofed sensor data can be statistically plausible and may evade conventional residual tests. Secure time sources, authentication, redundant sensing, anomaly detection, and attack-aware estimation are therefore important. Diagnostic data can also reveal production conditions or operator behavior, creating privacy and governance obligations.

Best practices include:

• define states and units explicitly;
• derive discrete noise covariances consistently with the sampling interval;
• validate observability and numerical conditioning;
• model biases and correlations rather than hiding them in arbitrary noise;
• use innovation and covariance diagnostics during commissioning;
• prefer factorization-based numerical methods to explicit matrix inversion;
• validate thresholds on representative healthy and faulty data;
• test missing data, saturation, outliers, delays, and mode changes;
• separate anomaly evidence from maintenance authorization;
• document assumptions, uncertainty, model versions, and human override procedures.

Contemporary Developments

Current research includes distributed and federated estimation, event-triggered communication, robust and risk-sensitive filtering, set-membership estimation, graph-based sensor fusion, digital twins, and learning-enabled observers. Distributed methods reduce dependence on a central node but must address communication delays, inconsistent local estimates, and adversarial agents. Robust filters seek acceptable performance under heavy-tailed or bounded disturbances rather than exact Gaussian assumptions. Digital twins connect real-time estimation with simulation and lifecycle maintenance, but their value depends on calibration and data quality.

Chapter Summary

Stochastic control analysis represents process and measurement uncertainty explicitly through random variables and covariance matrices. The linear stochastic state-space model provides a foundation for prediction, estimation, and diagnosis. The Kalman filter alternates covariance-aware prediction with measurement correction; its gain balances model and sensor information. Correct specification of \(Q\), \(R\), units, correlations, and observability is essential.

Nonlinear systems require methods such as the EKF, UKF, or particle filter, each with assumptions and limitations. Innovations and normalized residuals provide statistically grounded tools for fault detection, while accumulated tests improve sensitivity to persistent faults. Sensor fusion combines complementary information but must account for correlation, calibration, timing, and missing data. Condition monitoring extends these principles to health indices, machine learning, and predictive maintenance. Safe deployment requires validation, cybersecurity, explainability, governance, and explicit treatment of uncertainty.

Key Terms

Stochastic system: A system whose variables or disturbances include probabilistic uncertainty.

Process noise: Random disturbance entering the system dynamics.

Measurement noise: Random error corrupting a sensor observation.

Covariance: A matrix describing variances and pairwise uncertainty relationships.

Gaussian distribution: A probability distribution characterized by a mean and covariance for vector variables.

State-space model: A representation using internal states, inputs, and measured outputs.

Observability: The ability to infer internal states from available measurements and inputs.

Kalman filter: A recursive minimum-mean-square estimator for linear Gaussian models.

Innovation: The difference between an actual measurement and its predicted value.

Innovation covariance: The predicted covariance of the measurement innovation.

Kalman gain: The matrix determining how strongly an innovation changes the state estimate.

EKF: An extended Kalman filter using local linearization of nonlinear models.

UKF: An unscented Kalman filter using sigma points to propagate uncertainty.

NIS: Normalized innovation squared, a statistical consistency and detection measure.

NEES: Normalized estimation error squared, requiring knowledge of the true state.

Residual: A discrepancy between observed and model-predicted behavior.

Fault detection: Determining whether abnormal behavior is present.

Fault isolation: Determining which component or fault class is responsible.

Sensor fusion: Combining information from multiple sensors or models.

Condition monitoring: Assessing equipment health from measurements and inferred indicators.

Review Questions

Short Answer Questions

1. Distinguish process noise from measurement noise.

2. What physical meaning is carried by the diagonal and off-diagonal entries of a covariance matrix?

3. Why must the process-noise covariance be added during Kalman prediction?

4. Define the innovation and innovation covariance.

5. State one limitation of the extended Kalman filter.

Long Answer Questions

1. Derive and explain the prediction and measurement-update equations of the discrete Kalman filter.

2. Discuss how observability, process-noise covariance, and measurement-noise covariance affect estimator performance.

3. Compare the EKF, UKF, and particle filter for nonlinear estimation.

4. Explain the design of a residual-based fault-detection system, including threshold selection and persistence logic.

5. Discuss the practical requirements for deploying sensor fusion and condition monitoring in an industrial system.

Analytical / Application Questions

1. A scalar estimator has predicted mean 10, predicted variance 9, and a measurement of 13 with measurement variance 3. Calculate the Kalman gain, updated estimate, and updated variance.

2. Design a residual-based diagnostic strategy for a temperature sensor that develops a slowly increasing bias while the plant is subject to random load disturbances.

3. Two sensors measure the same variable but have correlated errors. Explain why independent-sensor fusion is overconfident and propose a suitable modeling or testing approach.

Multiple Choice Questions with Answers

Question 1

In a linear Kalman filter, what does the process-noise covariance Q represent?

A. Uncertainty introduced by system disturbances between samples

B. Uncertainty in the measured output only

C. The inverse of the measurement matrix

D. The covariance of the initial control input

Correct Answer: A. Uncertainty introduced by system disturbances between samples

Question 2

Which condition is required for a valid measurement-update innovation covariance in the standard Kalman filter?

A. R must be negative definite

B. R is typically positive definite

C. A must be diagonal

D. C must be square and invertible

Correct Answer: B. R is typically positive definite

Question 3

A scalar prediction has variance 4 and the measurement variance is 1. What is the Kalman gain when C=1?

A. 0.20

B. 0.25

C. 0.80

D. 4.00

Correct Answer: C. 0.80

Question 4

What is the innovation in a Kalman filter?

A. The difference between the predicted and actual control input

B. The difference between the actual measurement and its predicted value

C. The covariance of the state estimate

D. The process-noise realization

Correct Answer: B. The difference between the actual measurement and its predicted value

Question 5

The normalized innovation squared is compared most directly with which distribution under the standard Gaussian assumptions?

A. Uniform distribution

B. Poisson distribution

C. Chi-square distribution

D. Exponential distribution

Correct Answer: C. Chi-square distribution

Question 6

Which estimator uses Jacobian matrices to locally linearize a nonlinear model?

A. Extended Kalman filter

B. Particle filter

C. Moving-average filter

D. Parity-only observer

Correct Answer: A. Extended Kalman filter

Question 7

Why can treating correlated sensors as independent cause a problem?

A. It always increases measurement noise

B. It can produce an unjustifiably small covariance and overconfidence

C. It makes the state unobservable in every case

D. It eliminates the need for calibration

Correct Answer: B. It can produce an unjustifiably small covariance and overconfidence

Question 8

Which task identifies the component responsible for an abnormal residual?

A. Fault isolation

B. State propagation

C. Sampling quantization

D. Covariance symmetrization

Correct Answer: A. Fault isolation

Question 9

In the scalar worked temperature example, the updated estimate was closest to the measurement because:

A. The process model was ignored

B. The sensor variance was smaller than the predicted variance

C. The innovation was zero

D. The measurement matrix was singular

Correct Answer: B. The sensor variance was smaller than the predicted variance

Question 10

Which practice improves numerical robustness in covariance updates?

A. Explicitly computing every matrix inverse

B. Discarding covariance off-diagonal terms

C. Using the Joseph covariance form or matrix factorization

D. Setting all process noise to zero

Correct Answer: C. Using the Joseph covariance form or matrix factorization