Correlation, Random Signals, and Power Spectra

Introduction

Signals encountered in communication, radar, biomedical instrumentation, control, and sensor networks are rarely completely deterministic. Thermal noise, interference, fading, quantization effects, and unpredictable source behavior require a probabilistic description. This chapter develops the principal tools for analyzing such signals: random processes, correlation functions, covariance, stationarity, ergodicity, power spectral density, cross-spectral density, and linear-system response.

A deterministic signal is specified completely as a function of time. A random signal, or random process, is instead a family of possible waveforms together with probabilities assigned to their occurrence. The objective is not usually to predict one exact waveform, but to determine statistically meaningful quantities such as average power, correlation, bandwidth, and noise-induced output variance.

Learning Objectives

After studying this chapter, the reader should be able to:

• distinguish random variables, random processes, sample functions, and ensembles;
• calculate means, autocorrelation functions, covariance, and mean-square values;
• explain strict-sense and wide-sense stationarity and ergodicity;
• derive and interpret power spectral density using the Wiener–Khinchin theorem;
• analyze the effect of linear time-invariant systems on random signals;
• use cross-correlation and cross-spectral density in estimation and detection;
• evaluate practical measurement limitations and noise-model assumptions.

1. Random Signals and Random Processes

A random variable X assigns a numerical value to each outcome of a probabilistic experiment. A random process X(t) assigns a random variable to every time t. A single realization x(t) is called a sample function. The collection of all possible sample functions is the ensemble.

For a complex-valued process, the ensemble mean is

where E[·] denotes mathematical expectation. If X(t) has probability density p_X(x;t), then

For a discrete-valued process, the corresponding integral is replaced by a summation. The mean has the same units as X(t). The mean-square value is

and represents average squared amplitude. For a voltage across resistance R, average power is E[|X(t)|²] ÷ R, assuming the voltage is measured across a purely resistive load.

The variance is

For a real process, this becomes E[(X(t) − m_X(t))²]. Variance has squared signal units and measures fluctuations around the mean.

Autocorrelation and covariance

The autocorrelation function is

where * denotes complex conjugation. For a real process, conjugation is unnecessary. Autocorrelation measures statistical similarity between values at two times. At equal times,

which is the mean-square value.

The autocovariance is

and therefore

Autocorrelation includes the contribution of the mean; covariance describes only random departures from the mean.

For every physically valid process, the autocorrelation is positive semidefinite. Thus, for arbitrary complex constants a₁,…,a_n and times t₁,…,t_n,

This condition is important because an arbitrarily selected function cannot necessarily be a valid autocorrelation function.

2. Stationarity and Ergodicity

Strict-sense stationarity

A process is strict-sense stationary (SSS) if all finite-dimensional probability distributions remain unchanged under a common time shift. For every n, times t₁,…,t_n, and shift τ,

SSS is a strong requirement and can be difficult to verify experimentally.

Wide-sense stationarity

A process is wide-sense stationary (WSS) if its first two moments satisfy:

1. m_X(t) = m_X, a constant;
2. R_X(t₁,t₂) depends only on the time difference τ = t₁ − t₂.

Consequently,

and

WSS is sufficient for most power and spectral calculations. A stationary process has constant average power, but stationarity does not imply that successive samples are statistically independent.

For a WSS process, important properties include

and, for a real-valued process, R_X(τ) is an even function. Also, |R_X(τ)| ≤ R_X(0). The value R_X(0) is the total mean-square value.

Ergodicity

A process is ergodic in the mean if a time average equals the ensemble mean with probability one:

It is ergodic in correlation if time-averaged products reproduce the autocorrelation. Ergodicity permits one sufficiently long observed waveform to estimate ensemble statistics. Stationarity alone does not guarantee ergodicity. For example, X(t)=A, where A is a random constant selected once, is stationary, but a single realization cannot reveal the ensemble distribution of A.

3. Important Models of Random Signals

Gaussian processes

A process is Gaussian if every finite set of samples has a jointly Gaussian distribution. Gaussian processes are especially important because their mean and autocorrelation completely determine all finite-dimensional distributions. If a Gaussian process is WSS, it is also strictly stationary under the usual consistency conditions.

A zero-mean Gaussian random variable with variance σ² has density

for real X. Gaussian noise is a highly effective model for aggregate thermal and electronic disturbances, although it is not universally valid.

White noise

Ideal white noise has constant power spectral density over all frequencies. Its ideal autocorrelation is proportional to a Dirac delta:

for a real, two-sided noise PSD equal to N₀⁄2. Since an ideal white-noise process has infinite total power over infinite bandwidth, it is a mathematical model, not a directly realizable signal. Physical noise is band-limited or has a finite correlation time.

Colored noise

Noise with nonconstant PSD is called colored noise. Low-frequency-dominant noise is often called pink or flicker noise when its PSD approximately varies as 1⁄|f| over a specified range. Such labels are model approximations and must be accompanied by frequency limits.

4. Power Spectral Density

For a WSS process, the power spectral density (PSD) is the Fourier transform of its autocorrelation:

The inverse relation is

Here f is frequency in hertz, τ is time in seconds, and S_X(f) has units of signal-squared per hertz. The pair is the Wiener–Khinchin theorem, applicable to WSS processes under appropriate integrability or generalized-function conditions.

For a real process, S_X(f) is real, nonnegative, and even. The total average power is

If the process has nonzero mean m_X, its PSD contains a spectral line at zero frequency:

where X_c(t)=X(t)−m_X is the zero-mean component.

For angular frequency ω, the transform pair is

with inverse factor 1⁄(2π). Mixing hertz and radians per second without the appropriate 2π factor is a common source of error.

Worked Example 1: Exponentially correlated process

Suppose a zero-mean WSS process has

where σ² is the mean-square value and a has units s⁻¹. Using the Fourier transform of e^(−a|τ|),

At f=0, S_X(0)=2σ²/a, and integrating the PSD over all frequencies gives σ², as required. The correlation time is of order 1/a: larger a means faster decorrelation and a wider spectrum.

5. Linear Time-Invariant Systems and Random Inputs

Let a stable LTI system have impulse response h(t) and frequency response H(f). If the input X(t) is WSS and the required integrals exist, the output Y(t) is WSS with

This result is central to noise analysis. The system does not alter the input PSD arbitrarily; it weights each frequency by the squared magnitude response.

The output mean is

for a system whose DC response is defined. The output mean-square value is

For a real resistor and voltage noise, output noise power is obtained by integrating the voltage PSD over the passband and dividing by resistance when appropriate.

If the input and output are jointly WSS, their cross-spectrum is

and the reverse cross-spectrum is related by conjugation under standard conventions.

Equivalent noise bandwidth

For a filter with peak power gain |H(f₀)|², its equivalent noise bandwidth is

This definition assumes a two-sided frequency integral. For flat two-sided input PSD S₀, output power is S₀∫|H(f)|²df. One-sided conventions require corresponding changes in the PSD and integration limits.

Worked Example 2: White-noise filtering

A zero-mean voltage noise has two-sided PSD S_N(f)=10⁻¹² V²/Hz. An ideal low-pass filter passes frequencies |f|≤5 kHz with unit gain and rejects all others. The output mean-square voltage is

Thus the rms output voltage is √(10⁻⁸)=10⁻⁴ V = 100 μV. The calculation uses a two-sided PSD and a bandwidth of 10 kHz.

6. Cross-Correlation, Coherence, and Estimation

For two jointly WSS processes X(t) and Y(t), the cross-correlation is

The cross-power spectral density is its Fourier transform:

Cross-correlation is useful for delay estimation. If Y(t)=X(t−τ₀)+N(t), where N is uncorrelated with X, the cross-correlation generally exhibits a maximum near τ₀, subject to waveform ambiguity and noise.

The magnitude-squared coherence is

where denominators are positive. Under standard conditions, 0≤γ²_XY(f)≤1. A value near one indicates a strong linear frequency-domain relationship, not necessarily causality. Coherence is widely used in vibration diagnosis, system identification, biomedical signal analysis, and audio engineering.

The cross-correlation coefficient at zero lag is

for real variables with nonzero standard deviations σ_X and σ_Y. It measures linear association, not general statistical dependence.

7. Noise in Communication and Detection

In an additive noise channel,

where S(t) is the transmitted signal and N(t) is noise. If signal and noise are uncorrelated,

The signal-to-noise ratio is

and in decibels,

when P_S and P_N are powers measured in the same bandwidth and at the same reference point. For amplitude ratios that correspond to equal impedances, 20 log₁₀(amplitude ratio) gives the same result.

A matched filter maximizes the output peak signal-to-noise ratio in additive white Gaussian noise under specified finite-energy signal and sampling assumptions. Its impulse response is proportional to a time-reversed, conjugated version of the known signal. The result depends on the assumed noise spectrum; in colored noise, a whitening operation or generalized matched filter is required.

8. Illustrative Case Study: Vibration-Based Bearing Monitoring

Background

An industrial motor is instrumented with an accelerometer. Bearing defects generate periodic impacts, while motor rotation and structural resonances shape the measured spectrum.

Problem

A single time waveform appears noisy and does not clearly reveal the defect. The engineering team must distinguish a defect-related periodic component from broadband mechanical and electronic noise.

Analysis

The team first removes sensor bias and verifies sampling adequacy. A long record is divided into overlapping segments. For each segment, a windowed periodogram is computed and averaged, producing a Welch PSD estimate. Averaging reduces estimator variance but decreases frequency resolution when shorter segments are used.

An accelerometer mounted near a healthy reference bearing supplies a second process. Cross-spectral density and coherence are calculated. A strong narrowband peak with high coherence between the defective-bearing sensor and a known shaft-speed reference supports a mechanical origin. Broadband energy with low coherence to the reference is more consistent with unrelated noise, although this conclusion is not definitive.

Approach and outcome

A band-pass filter isolates the resonance band, and the envelope spectrum is examined for repetition at the bearing defect frequency. The maintenance decision is based on repeated measurements, calibration records, operating load, and comparison with historical baselines rather than on one spectral peak.

Learning points

1. PSD describes distribution of average power, not phase information.
2. Cross-correlation can reveal timing relationships hidden in a noisy waveform.
3. Coherence helps assess whether two channels share a linear frequency-dependent component.
4. Windowing, record length, leakage, and sensor placement affect conclusions.
5. Statistical evidence should be combined with physical knowledge and uncertainty estimates.

9. Practical Spectral Estimation

A finite record cannot provide the exact PSD. The periodogram is obtained by taking the squared magnitude of the finite-record Fourier transform and applying a normalization. It is simple but has substantial variance and spectral leakage. Window functions reduce leakage at the cost of wider main lobes and altered amplitude calibration.

Welch’s method averages modified periodograms from overlapping, windowed segments. Increasing segment length improves frequency resolution; increasing the number of segments generally reduces variance. These objectives conflict for a fixed record length.

Good practice includes:

• state whether PSD is one-sided or two-sided;
• report sampling rate, record length, window, overlap, and normalization;
• remove mean or trend when appropriate;
• avoid interpreting leakage sidelobes as physical components;
• verify anti-alias filtering before sampling;
• use confidence intervals or repeated records when decisions are consequential.

Aliasing occurs when spectral content above the Nyquist frequency is sampled without adequate suppression. It cannot generally be removed after sampling because different continuous-time frequencies become indistinguishable in the samples.

10. Limitations, Governance, and Contemporary Directions

Random-process models are abstractions. Gaussianity may fail for impulsive interference, communication outliers, rotating machinery faults, and biological signals. WSS assumptions may fail during startup, switching, fading, or changing traffic conditions. A PSD also discards phase and may conceal transient behavior.

In safety-critical systems, measurement uncertainty, calibration, traceability, and reproducibility are essential. Sensor data may contain identifiable information, especially in biomedical or industrial environments; access control, secure storage, and documented retention policies are therefore relevant. Automated or AI-based anomaly detectors should report uncertainty, maintain audit trails, and be validated against changing operating conditions. A model trained on one machine or environment may not generalize to another.

Contemporary research combines classical spectral analysis with time-frequency methods, cyclostationary analysis, robust statistics, sparse representations, and learned representations. These methods are valuable when signals are nonstationary, but they do not eliminate the need for physical modeling, appropriate sampling, and statistically defensible validation.

Chapter Summary

Random processes describe uncertain signals through ensembles of possible waveforms. Their principal second-order descriptors are mean, mean-square value, variance, autocorrelation, and covariance. Strict stationarity concerns all probability distributions, whereas wide-sense stationarity requires only a constant mean and an autocorrelation dependent on lag. Ergodicity explains when time averages from one record can estimate ensemble quantities.

For WSS processes, the Wiener–Khinchin theorem connects autocorrelation and power spectral density. The PSD shows how average power is distributed with frequency, and its integral equals mean-square value. An LTI system transforms PSD according to S_Y(f)=|H(f)|²S_X(f). Cross-correlation and coherence support delay estimation, system identification, and fault diagnosis. Practical spectral estimates require careful treatment of finite records, windows, leakage, variance, sampling, and one-sided versus two-sided conventions.

Key Terms

Random process: A time-indexed family of random variables.

Sample function: One possible waveform generated by a random process.

Ensemble: The collection of possible sample functions.

Mean: Expected value of a process at a specified time.

Variance: Mean squared deviation from the mean.

Autocorrelation: Expected product of process values at two times.

Covariance: Correlation of deviations from the mean.

Strict-sense stationarity: Invariance of all finite-dimensional distributions under time shift.

Wide-sense stationarity: Constant mean and lag-dependent autocorrelation.

Ergodicity: Equality of appropriate time and ensemble averages.

Gaussian process: A process whose finite sample vectors are jointly Gaussian.

White noise: Ideal noise with constant PSD over frequency.

Colored noise: Noise with frequency-dependent PSD.

Power spectral density: Average power distribution per unit frequency.

Cross-correlation: Statistical relationship between two processes at relative delay.

Cross-spectral density: Fourier transform of cross-correlation.

Coherence: Normalized frequency-domain measure of linear association.

Equivalent noise bandwidth: Noise bandwidth of a filter normalized to its peak gain.

Aliasing: Irreversible spectral overlap caused by inadequate sampling.

Periodogram: Finite-record estimate of a power spectrum.

Review Questions

Short Answer Questions

1. Distinguish a random process from a sample function.
2. State the two conditions for wide-sense stationarity.
3. What does R_X(0) represent for a WSS process?
4. Why is ideal white noise not physically realizable over infinite bandwidth?
5. Define magnitude-squared coherence.

Long Answer Questions

1. Derive the relationship between autocorrelation and covariance for a complex random process.
2. Explain the Wiener–Khinchin theorem and discuss its engineering significance.
3. Derive the output PSD of a stable LTI system driven by a WSS input.
4. Compare strict stationarity, wide-sense stationarity, and ergodicity.
5. Explain the principal sources of error in practical PSD estimation.

Analytical / Application Questions

1. A zero-mean process has R_X(τ)=4e^(−2|τ|). Derive its two-sided PSD and verify that the integrated PSD equals 4.
2. A unit-gain ideal filter passes |f|≤10 kHz and is driven by white noise with two-sided PSD 3×10⁻¹³ V²/Hz. Determine output mean-square and rms voltage.
3. Design an experimental procedure using autocorrelation, cross-correlation, and coherence to distinguish a periodic machine fault from uncorrelated sensor noise. State assumptions and limitations.

Multiple Choice Questions with Answers

Question 1

For a WSS process, which condition must its autocorrelation satisfy?

A. It depends on t₁ and t₂ independently

B. It depends only on the lag τ

C. It must be constant for every τ

D. It must always be zero at τ=0

Correct Answer: B. It depends only on the lag τ

Question 2

The value R_X(0) for a WSS process equals its:

A. Mean

B. Variance only

C. Mean-square value

D. Bandwidth

Correct Answer: C. Mean-square value

Question 3

According to the Wiener–Khinchin theorem, the PSD is the Fourier transform of the:

A. Mean

B. Autocorrelation function

C. Impulse response

D. Probability density

Correct Answer: B. Autocorrelation function

Question 4

If a stable LTI system has frequency response H(f), the output PSD is:

A. H(f)S_X(f)

B. |H(f)|S_X(f)

C. |H(f)|²S_X(f)

D. S_X(f)⁄H(f)

Correct Answer: C. |H(f)|²S_X(f)

Question 5

Ideal white noise is physically unrealizable over infinite bandwidth because it has:

A. Zero total power

B. Infinite total power

C. No autocorrelation

D. A deterministic waveform

Correct Answer: B. Infinite total power

Question 6

For a real WSS process, the PSD is generally:

A. Complex and odd

B. Real, nonnegative, and even

C. Negative at all frequencies

D. Independent of autocorrelation

Correct Answer: B. Real, nonnegative, and even

Question 7

A two-sided white-noise PSD is 10⁻¹² V²/Hz over |f|≤5 kHz. What is the mean-square voltage after an ideal unit-gain filter?

A. 5×10⁻⁹ V²

B. 1×10⁻⁸ V²

C. 2×10⁻⁸ V²

D. 1×10⁻¹² V²

Correct Answer: B. 1×10⁻⁸ V²

Question 8

The magnitude-squared coherence is bounded by:

A. −∞ and 0

B. 0 and 1

C. 1 and ∞

D. −1 and 1 only for complex signals

Correct Answer: B. 0 and 1

Question 9

Ergodicity permits an engineer to estimate ensemble statistics from:

A. A single sufficiently long realization

B. Only the system impulse response

C. A frequency response without measurements

D. A deterministic sinusoid exclusively

Correct Answer: A. A single sufficiently long realization

Question 10

Aliasing is primarily caused by:

A. Using a sampling rate insufficient for the signal bandwidth

B. Using a stable filter

C. Subtracting the signal mean

D. Increasing record length

Correct Answer: A. Using a sampling rate insufficient for the signal bandwidth