10. Stochastic Systems, Estimation and Sensor Fusion

Introduction and Context

Classical control analysis often begins with a deterministic model: given an input and an initial condition, the future state and output are assumed to be exactly calculable. Physical systems, however, rarely behave in this ideal manner. Electrical measurements contain sensor noise, actuators introduce disturbances, model parameters vary with temperature and ageing, and unmeasured inputs influence system evolution. Consequently, a controller must often act on incomplete and uncertain information.

Stochastic systems theory provides the mathematical language for this situation. It represents disturbances and measurement errors as random processes and replaces a single predicted state by a probability distribution or, more commonly, by its mean and covariance. Estimation then becomes the task of inferring hidden system states or parameters from noisy measurements. Sensor fusion extends this task by combining information from multiple sensors whose accuracy, sampling rate, and failure characteristics differ.

These ideas are central to modern control systems. They support navigation of aircraft and autonomous vehicles, condition monitoring of electrical machines, robotics, process control, power-system state estimation, medical instrumentation, and industrial automation. The Kalman filter and its nonlinear extensions are particularly important because they provide recursive estimates suitable for real-time implementation.

Learning Objectives

After studying this chapter, the reader should be able to:

1. Formulate linear and nonlinear systems with process and measurement uncertainty.

2. Explain covariance, correlation, whiteness, stationarity, and Gaussian assumptions.

3. Derive and interpret the Wiener and Kalman filtering approaches.

4. Apply covariance propagation, innovation analysis, and Kalman gain calculation.

5. Distinguish the extended Kalman filter (EKF) from the unscented Kalman filter (UKF).

6. Combine measurements from multiple sensors using statistically meaningful methods.

7. Diagnose estimator divergence, poor tuning, bias, and sensor inconsistency.

8. Evaluate professional, ethical, cybersecurity, and implementation issues in estimation systems.

1. Modelling Uncertainty in Dynamic Systems

A discrete-time stochastic state-space model is commonly written as

x_k = f(x_{k-1}, u_{k-1}) + w_{k-1}

y_k = h(x_k) + v_k,

where x_k is the state, u_k is the known input, y_k is the measurement, w_k is process noise, and v_k is measurement noise. The functions f and h describe state evolution and measurement generation. In the linear case,

x_k = A_{k-1}x_{k-1}+B_{k-1}u_{k-1}+w_{k-1},

y_k=C_kx_k+D_ku_k+v_k.

Process noise represents uncertainty in the plant model, external disturbances, neglected dynamics, and parameter variation. Measurement noise represents sensor quantisation, electronic noise, environmental interference, calibration error, and finite resolution. These two noises have different physical meanings and should not be interchanged merely to improve numerical results.

The random variables w_k and v_k are often modelled as zero-mean white Gaussian sequences with covariance matrices

E[w_k w_j^T] = Q_k delta_kj,

E[v_k v_j^T] = R_k delta_kj.

Here Q is the process-noise covariance and R is the measurement-noise covariance. A cross-covariance matrix may also be required when process and measurement noise are correlated. The Gaussian assumption is convenient because a Gaussian distribution is completely specified by its mean and covariance, but it is an approximation rather than a universal law.

The state estimate is represented by a conditional mean,

xhat_k|k = E[x_k | y_1,...,y_k],

and its uncertainty by the conditional covariance

P_k|k = E[(x_k-xhat_k|k)(x_k-xhat_k|k)^T | y_1,...,y_k].

A covariance matrix is symmetric and positive semidefinite. Its diagonal elements are variances; its off-diagonal elements describe statistical dependence between state errors. A large covariance indicates uncertainty, not necessarily a large state value.

Essential stochastic concepts

A random process is a collection of random variables indexed by time. A process is stationary in a strict sense when its statistical properties are invariant under time shifts. Wide-sense stationarity requires constant mean and autocorrelation depending only on time difference. White noise has zero correlation at distinct time instants, although ideal white noise is a mathematical model and cannot have finite energy over infinite bandwidth.

The innovation, or residual, is the difference between an actual measurement and its predicted value. For a linear filter,

r_k = y_k-C_kxhat_k|k-1.

Innovation analysis is a powerful diagnostic: under a correctly specified filter, innovations should be approximately zero mean, mutually uncorrelated, and consistent with their predicted covariance.

2. Estimation as an Inference Problem

A system may possess states that cannot be measured directly. For example, rotor flux, battery state of charge, aircraft attitude, and internal temperature may be inferred from available measurements. Estimation is therefore an inverse problem: infer hidden quantities from a model and observations.

Two properties are especially important. Observability concerns whether the internal state can be reconstructed from input-output data. A system can be mathematically observable yet practically difficult to estimate if the measurements are weakly sensitive to some state or if the signal-to-noise ratio is poor. Identifiability concerns whether unknown parameters can be uniquely inferred. An estimator cannot compensate for a fundamentally unobservable or unidentifiable system.

A Bayesian interpretation is useful. The prediction step forms a prior distribution from the previous estimate and the process model. The measurement update combines this prior with the likelihood of the new measurement. The result is a posterior distribution. The Kalman filter is the exact recursive Bayesian estimator for linear systems with Gaussian noise.

3. Wiener Filtering and the Frequency-Domain View

The Wiener filter is an optimum linear estimator for stationary random processes when the objective is to minimize mean-square error. Suppose a desired signal d(t) is observed through a noisy signal z(t). A linear filter with impulse response g(t) produces an estimate of d(t). The Wiener-Hopf equation determines the filter that minimizes

J = E[(d(t)-dhat(t))^2].

In the frequency domain, the noncausal Wiener filter is commonly expressed as

G_opt(jw) = S_dz(jw) / S_zz(jw),

where S_dz is the cross-power spectral density and S_zz is the observation power spectral density. For causal real-time systems, spectral factorisation is needed, and the solution is more involved.

The Wiener approach is valuable when statistical properties are stationary and known, and when a frequency-domain interpretation is desirable. Its limitations include dependence on accurate spectra, difficulty with time-varying systems, and less direct treatment of changing operating conditions. The Kalman filter can be understood as a recursive, state-space counterpart that is especially suitable for nonstationary and real-time problems.

4. The Linear Kalman Filter

The Kalman filter alternates between prediction and correction. For the linear model, the time update is

xhat_k|k-1 = A_k xhat_k-1|k-1 + B_k u_k,

P_k|k-1 = A_k P_k-1|k-1 A_k^T + Q_k.

The predicted covariance grows when process uncertainty is added. The measurement update begins with the innovation covariance,

S_k = C_k P_k|k-1 C_k^T + R_k,

and the Kalman gain,

K_k = P_k|k-1 C_k^T S_k^-1.

The estimate is corrected according to

xhat_k|k = xhat_k|k-1 + K_k(y_k-C_k xhat_k|k-1).

The covariance may be written as

P_k|k = (I-K_kC_k)P_k|k-1.

For numerical robustness, the Joseph form is often preferred:

P_k|k = (I-K_kC_k)P_k|k-1(I-K_kC_k)^T + K_kR_kK_k^T.

The gain expresses a balance between model confidence and measurement confidence. If R is large, the measurement is noisy and the gain generally decreases. If Q is large, the model is less trusted and the filter tends to respond more strongly to measurements. This interpretation is more useful than treating Q and R as arbitrary tuning knobs.

The filter is not simply a low-pass filter. Its effective behaviour depends on system dynamics, sampling time, covariance assumptions, sensor geometry, and the relationship between states and measurements. In steady conditions, covariance may converge to a solution of the discrete algebraic Riccati equation, provided suitable detectability and stabilisability conditions hold.

Covariance analysis and consistency

The predicted covariance quantifies uncertainty before seeing the current measurement; the posterior covariance quantifies uncertainty afterwards. A measurement normally reduces uncertainty in observable directions, but it need not reduce every state variance. Correlations allow information about one state to improve estimates of another.

Consistency compares the actual estimation error with the reported covariance. If the filter is overconfident, its covariance is too small and innovations may be larger than expected. If it is excessively conservative, the covariance is too large and the estimate may respond unnecessarily slowly. The normalized innovation squared is

NIS_k = r_k^T S_k^-1 r_k.

For appropriately dimensioned Gaussian innovations, NIS can be compared with a chi-square distribution to identify abnormal measurements or covariance mismatch.

5. Nonlinear State Estimation

Many engineering systems have nonlinear dynamics or sensors. Examples include robot range-bearing measurements, aircraft attitude, chemical processes, and electromechanical systems with saturation or trigonometric relationships. A nonlinear model is written as

x_k=f(x_k-1,u_k-1)+w_k-1,

y_k=h(x_k)+v_k.

Extended Kalman filter

The EKF linearises the model about the current estimate. The Jacobian matrices are

F_k = partial f / partial x,

H_k = partial h / partial x.

These matrices replace A and C in the covariance and gain calculations. The state itself is propagated through the nonlinear functions, while uncertainty is approximated by first-order Taylor expansion.

The EKF is computationally efficient and widely used. However, linearisation may be poor when uncertainty is large, the model is strongly nonlinear, or the operating point changes rapidly. Incorrect Jacobians can cause instability or silent performance degradation. Angle wrapping, coordinate singularities, and inconsistent treatment of nonlinear variables are common implementation problems.

Unscented Kalman filter

The UKF avoids explicit Jacobian calculation. It selects a set of sigma points whose weighted mean and covariance match the current estimate. Each point is propagated through the nonlinear model, and the transformed points are recombined to estimate the new mean and covariance. The measurement transformation is performed in the same way.

For suitable smooth nonlinearities, the unscented transformation captures mean and covariance effects more accurately than first-order linearisation. It is particularly attractive when derivatives are unavailable or difficult to compute. Its disadvantages include greater computational cost, sensitivity to scaling parameters, and the fact that it still represents uncertainty primarily through mean and covariance. Neither EKF nor UKF is automatically reliable for highly non-Gaussian or multimodal distributions; particle filters may then be more appropriate.

6. Sensor Fusion Architectures

Sensor fusion combines measurements to obtain a more accurate, complete, or reliable estimate than any individual sensor can provide. Fusion is not merely averaging. It requires knowledge of measurement models, uncertainty, timing, coordinate frames, and dependence between errors.

In centralized fusion, all raw measurements are sent to one estimator. This can provide statistically efficient results but may require high bandwidth and a common computational platform. In decentralized fusion, local estimators process sensor data separately and exchange estimates or information. This improves modularity and fault containment but creates a difficult problem: estimates may share common information. If that correlation is ignored, the fused covariance becomes artificially small.

For independent estimates xhat_1 and xhat_2 with covariances P_1 and P_2, an information-form fusion rule is

P_f^-1 = P_1^-1 + P_2^-1,

xhat_f = P_f(P_1^-1xhat_1 + P_2^-1xhat_2).

This formula must not be used blindly when the estimates are correlated. Cross-covariance, common process models, or shared measurements can result in double counting. Covariance intersection provides a conservative alternative when cross-correlation is unknown:

P_f^-1 = omega P_1^-1 +(1-omega)P_2^-1,

with the weight omega selected according to a chosen criterion. The resulting covariance is generally less optimistic but remains safe under unknown correlation assumptions.

Practical fusion also requires time synchronisation, spatial calibration, data association, outlier rejection, and management of missing or delayed measurements. A camera, inertial measurement unit, encoder, and GPS receiver may each use different sampling rates and coordinate conventions. A technically correct filter can still fail if these interfaces are mishandled.

7. Applied Scenario: Sensor Fusion for an Autonomous Mobile Platform

Background

Consider an illustrative warehouse vehicle required to estimate planar position, velocity, and heading while moving among shelves. It has wheel encoders, an inertial measurement unit (IMU), and intermittent radio-based position measurements. Wheel slip is possible, the IMU has bias, and radio measurements may be noisy or temporarily unavailable.

Situation and problem

Dead reckoning from encoders and gyroscopes provides rapid updates but accumulates error. Radio positioning limits long-term drift but is too noisy for smooth control. The control system requires a state estimate at every sampling instant, together with uncertainty information for collision avoidance and path planning.

Analysis and approach

A state vector may include position, velocity, heading, gyro bias, and wheel-scale error. The motion model propagates this state at the IMU rate. Process covariance is increased when wheel slip or unmodelled acceleration is likely. Radio position measurements are incorporated through a measurement update whenever they arrive. The nonlinear relationship between heading and velocity motivates an EKF or UKF.

Innovation tests are used to reject radio measurements inconsistent with the predicted state. The radio sensor is not simply discarded when its noise increases; instead, its measurement covariance is adapted using a validated quality indicator. If the radio and local estimator share a common prior, their estimates are not fused as independent quantities. A centralized filter avoids this double-counting issue.

Outcome

The fused estimate provides high-rate motion information from the IMU and encoders while using radio observations to limit long-term drift. During a radio outage, covariance grows, signalling reduced confidence to the planner. When measurements return, the filter corrects the accumulated error gradually rather than making an implausibly abrupt jump.

Learning points

The case demonstrates that estimator design is a systems problem, not only an equation-selection problem. Sensor physics, timing, calibration, fault detection, covariance tuning, and downstream decision rules all affect reliability. Uncertainty should be communicated to the controller rather than hidden behind a single point estimate.

8. Implementation Challenges, Risks, and Professional Practice

Poor tuning of Q and R is a frequent cause of unsatisfactory performance. Setting R too small makes the filter chase noise and become overconfident; setting Q too small makes it resist genuine changes. Biases should normally be modelled as states rather than disguised as white noise. Initial covariance should reflect actual knowledge: an incorrect but very small initial covariance can delay recovery from a poor initial estimate.

Numerical issues also matter. Covariance symmetry can be lost through round-off, and direct matrix inversion may be unstable. Cholesky-factor or square-root filters improve numerical conditioning. State units must be consistent, and covariance matrices must use corresponding squared units. Angles require careful wrapping, while quaternions require normalisation and attention to manifold geometry.

Outliers caused by spoofing, multipath, packet corruption, or sensor faults can severely damage an estimator. Innovation gating, robust loss functions, redundant sensing, and fault isolation are useful safeguards, but rejection logic must not conceal a persistent modelling error. A system should record estimator health indicators, rejected observations, covariance growth, and mode changes for maintenance and audit.

In safety-critical applications, estimation software requires traceability, verification, validation, controlled configuration, and defined fail-safe behaviour. Decisions based on estimated states can affect people, equipment, and infrastructure. Ethical practice therefore requires honest representation of uncertainty and avoidance of unjustified claims of accuracy. Cybersecurity is also relevant: malicious data injection can resemble a sensor fault and may manipulate a filter unless authentication, plausibility checking, and independent monitoring are provided.

9. Digital Tools and Emerging Directions

Modern estimation is supported by simulation environments, automatic differentiation, symbolic Jacobian generation, embedded linear-algebra libraries, and data-driven covariance adaptation. Digital twins can compare measured and predicted behaviour, while machine-learning models may augment imperfect physical models. However, a learned model should be monitored for distribution shift and should not be assumed to preserve physical constraints.

Current research and industrial development include distributed estimation over networked systems, event-triggered communication, simultaneous localisation and mapping, robust and set-membership filtering, moving-horizon estimation, and learning-assisted observers. In networked control, communication delay, packet loss, and limited bandwidth become part of the estimation problem. Hybrid systems combine continuous dynamics with discrete operating modes and may require interacting multiple-model filters.

Best practice is to begin with a physically meaningful model, identify observability and identifiability limitations, characterise sensor errors experimentally, and validate the complete estimator under nominal, disturbed, and failure conditions. Monte Carlo testing, innovation statistics, parameter sensitivity studies, and hardware-in-the-loop experiments should complement theoretical analysis.

Chapter Summary

Stochastic control models represent process and measurement uncertainty explicitly. Means and covariance matrices provide a practical description of uncertain states, while innovation statistics support performance monitoring. Wiener filtering gives an optimum linear stationary solution in the frequency domain; the Kalman filter provides recursive state estimation for linear Gaussian systems. The EKF extends the method through local linearisation, whereas the UKF propagates carefully selected sigma points through nonlinear models. Sensor fusion combines complementary information but must account for timing, calibration, outliers, and correlations between estimates. Reliable implementation depends on correct covariance modelling, numerical care, observability analysis, fault detection, cybersecurity, and honest communication of uncertainty.

Key Terms

Covariance: A matrix describing variances and cross-correlation among random variables.

Process noise: Random uncertainty entering the system dynamics.

Measurement noise: Random error affecting sensor observations.

White noise: A process with zero correlation at distinct time instants in the idealised model.

Gaussian distribution: A probability distribution completely specified by its mean and covariance.

State estimate: An inferred value of an internal system state.

Prior estimate: The prediction formed before incorporating the current measurement.

Posterior estimate: The estimate obtained after incorporating the current measurement.

Innovation: The difference between an actual measurement and its predicted value.

Innovation covariance: The predicted covariance of the innovation.

Kalman gain: The matrix determining how strongly the measurement changes the prediction.

Wiener filter: A linear minimum-mean-square-error filter based on stationary signal statistics.

Observability: The ability to reconstruct internal states from inputs and outputs.

Identifiability: The ability to determine unknown parameters uniquely from data.

Extended Kalman filter: A nonlinear filter based on first-order model linearisation.

Unscented Kalman filter: A nonlinear filter using weighted sigma points to propagate mean and covariance.

Sensor fusion: Combination of information from multiple sensors or estimators.

Covariance intersection: Conservative fusion used when cross-correlation is unknown.

Innovation gating: Statistical testing used to reject implausible measurements.

Estimator consistency: Agreement between actual estimation errors and reported uncertainty.

Review Questions

Short Answer Questions

1. Distinguish process noise from measurement noise.

2. What information is represented by the off-diagonal elements of a covariance matrix?

3. Define the innovation in a Kalman filter.

4. Why is observability important for state estimation?

5. State one fundamental difference between the EKF and UKF.

Long Answer Questions

1. Derive the prediction and measurement-update equations of the discrete-time linear Kalman filter and interpret each covariance term.

2. Explain the Wiener filtering problem and compare its assumptions with those of the Kalman filter.

3. Discuss how Q and R influence estimator behaviour and describe practical procedures for selecting them.

4. Explain centralized and decentralized sensor fusion, including the problem of correlated estimates.

5. Describe the major numerical, modelling, and safety issues involved in implementing an estimator in an industrial control system.

Analytical / Application Questions

1. A position sensor becomes unreliable while an inertial sensor remains available. Explain how the measurement covariance, innovation statistics, and covariance growth should change in a robust estimator.

2. Two local filters provide state estimates but use a common motion model and the same historical measurement. Determine why direct information-form fusion may be overconfident and propose a safer alternative.

3. Design an estimation strategy for a motor-drive system in which rotor speed is measured, load torque is unmeasured, and electrical parameters vary with temperature. Specify candidate states, disturbances, measurements, and validation tests.

22. Multiple Choice Questions with Answers

Question 1

In a linear Kalman filter, increasing the measurement-noise covariance R generally causes the filter to:

A. Trust the measurement more strongly

B. Trust the model prediction relatively more strongly

C. Increase the process noise automatically

D. Make the system observable

Correct Answer: B. Trust the model prediction relatively more strongly

Question 2

The innovation is best defined as:

A. The difference between the true state and its estimate

B. The predicted state covariance

C. The difference between an actual and predicted measurement

D. The process-noise covariance

Correct Answer: C. The difference between an actual and predicted measurement

Question 3

Which statement correctly characterizes the standard Kalman filter?

A. It is exact for linear systems with Gaussian noise

B. It requires all sensors to have identical sampling rates

C. It estimates only directly measured states

D. It assumes measurement noise is always zero

Correct Answer: A. It is exact for linear systems with Gaussian noise

Question 4

A principal limitation of the extended Kalman filter is that it:

A. Cannot use a state-space model

B. Requires stationary signals only

C. Uses local linearisation that may be poor for strong nonlinearities

D. Cannot estimate covariance

Correct Answer: C. Uses local linearisation that may be poor for strong nonlinearities

Question 5

The unscented Kalman filter differs from the EKF primarily because it:

A. Uses sigma points instead of explicitly calculating Jacobians

B. Ignores measurement noise

C. Can estimate only scalar states

D. Requires no model of system dynamics

Correct Answer: A. Uses sigma points instead of explicitly calculating Jacobians

Question 6

If two estimates share common historical measurements, direct fusion treating them as independent will most likely:

A. Produce an unnecessarily large covariance

B. Double-count information and produce an overconfident covariance

C. Remove all measurement noise

D. Make the estimates unbiased automatically

Correct Answer: B. Double-count information and produce an overconfident covariance

Question 7

For a correctly tuned Gaussian filter, the innovations should approximately be:

A. Large and strongly correlated

B. Zero mean and consistent with their predicted covariance

C. Identical to the process noise

D. Independent of the measurement model

Correct Answer: B. Zero mean and consistent with their predicted covariance

Question 8

Which modelling choice is generally appropriate when a slowly varying sensor bias is important?

A. Represent the bias as an additional state

B. Set measurement noise covariance to zero

C. Remove the sensor from the model

D. Assume the bias is white noise with no state representation

Correct Answer: A. Represent the bias as an additional state

Question 9

Covariance intersection is useful mainly when:

A. The system is completely deterministic

B. Cross-correlation between estimates is unknown

C. There is only one measurement

D. The measurement matrix is square and diagonal

Correct Answer: B. Cross-correlation between estimates is unknown

Question 10

In the mobile-platform scenario, covariance growth during a radio outage indicates that:

A. The state estimate is necessarily diverging

B. The estimator has no process model

C. Uncertainty is increasing because corrective measurements are unavailable

D. The IMU measurements have become exact

Correct Answer: C. Uncertainty is increasing because corrective measurements are unavailable