Probability, Random Processes and Noise

Introduction

Communication receivers operate in uncertainty. Information symbols may be random, channels fluctuate, electronic components generate noise, and interfering transmissions cannot usually be predicted exactly. Probability and random-process theory provide the mathematical framework for describing these effects and predicting communication performance.

A deterministic waveform is fully specified by a formula or measured record. A random process instead describes an ensemble of possible waveforms. Its statistical properties—mean, correlation, probability distribution and power spectral density—allow engineers to estimate bandwidth, output noise power, signal-to-noise ratio (SNR), error probability and receiver sensitivity.

Learning Objectives

After studying this chapter, students should be able to:

• apply probability laws, conditional probability and Bayes’ theorem;
• calculate moments and probabilities for common random variables;
• distinguish stationarity, wide-sense stationarity and ergodicity;
• calculate autocorrelation, cross-correlation and power spectral density;
• characterize real and complex Gaussian processes correctly;
• analyze thermal, shot, white and band-limited noise;
• calculate SNR, noise factor, noise figure and equivalent noise temperature;
• evaluate cascaded receiver noise and communication performance.

1. Probability Foundations

1.1 Experiments, events and axioms

A random experiment has an uncertain outcome. The sample space Ω contains all possible outcomes, while an event is a subset of Ω. A probability measure P satisfies:

1. P(A) ≥ 0 for every event A;
2. P(Ω)=1;
3. for mutually exclusive events A_i,

The complement rule is P(A^c)=1-P(A). For any two events,

1.2 Conditional probability and independence

The probability of A given that B has occurred is

Therefore,

Events A and B are independent if P(A∩B)=P(A)P(B). Independence is stronger than mutual exclusivity: two nonempty mutually exclusive events cannot be independent.

If mutually exclusive events B₁,…,B_n partition Ω, the total-probability theorem gives

Bayes’ theorem reverses the conditioning:

Worked example—bit-source identification. Source 1 generates 60% of received bits and has error probability 0.01. Source 2 generates 40% and has error probability 0.03. If E denotes an error,

The probability that an erroneous bit came from source 2 is

Although source 2 supplies fewer bits, it causes approximately two-thirds of the errors.

2. Random Variables

A random variable maps each outcome to a numerical value. A discrete random variable has a probability mass function (PMF) p_X(x)=P(X=x). A continuous random variable has a probability density function (PDF) f_X(x), for which

The cumulative distribution function (CDF), applicable to both types, is

For a continuous variable, f_X(x)=dF_X(x)/dx wherever the derivative exists.

2.1 Expectation and moments

For a discrete variable,

and for a continuous variable,

The mean is μ_X=E[X]. Variance measures dispersion:

Standard deviation σ_X has the same units as X. Variance has squared units.

For two random variables, covariance is

The correlation coefficient is

provided both standard deviations are nonzero. The Cauchy–Schwarz inequality ensures -1≤ρ_XY≤1. Independence implies zero covariance when the required moments exist, but zero covariance does not generally imply independence. Jointly Gaussian variables are an important exception: for them, zero covariance implies independence.

2.2 Common communication distributions

For a Poisson process with arrival rate ν events per second, the number N(T) observed during T seconds has

and both its mean and variance equal νT. If the symbol λ is instead defined as the expected count within the specified interval, then the mean and variance are λ. A rate and an interval count must not be confused.

The Gaussian PDF is

The standardized tail probability is represented by

3. Random Processes

A random process X(t,ω) assigns a waveform to every outcome ω. Fixing ω produces one sample function; fixing t produces a random variable.

Figure: An ensemble of possible sample functions generated by a random process.

3.1 Mean and correlation functions

The mean function is

For a possibly complex process, autocorrelation is defined as

where * denotes complex conjugation. Autocovariance is

For two processes X and Y, cross-correlation is R_XY(t₁,t₂)=E[X(t₁)Y^*(t₂)]. Correlation indicates statistical similarity, not necessarily causation.

3.2 Stationarity and ergodicity

A process is strict-sense stationary if every finite-dimensional joint distribution is invariant under a common time shift. This is a strong condition.

A process is wide-sense stationary (WSS) if:

1. its mean is constant, m_X(t)=m_X;
2. its autocorrelation depends only on time difference.

Using lag τ=t₁-t₂,

For a real WSS process, R_X(-τ)=R_X(τ). For a complex WSS process, R_X(-τ)=R_X^*(τ). In both cases, R_X(0)=E[|X(t)|²] is the average power and |R_X(τ)|≤R_X(0).

Ergodicity permits ensemble statistics to be inferred from one sufficiently long record. For example, mean ergodicity requires, under appropriate convergence conditions,

Stationarity does not automatically imply ergodicity. This distinction matters because engineers commonly possess only one measured waveform.

4. Power Spectral Density

For a WSS process with a suitably integrable autocorrelation, the Wiener–Khinchin theorem gives the two-sided power spectral density (PSD):

The inverse relation is

Thus,

For voltage or current processes, PSD units are respectively V²/Hz or A²/Hz. For a real process, the two-sided PSD is real, nonnegative and even. Periodic or nonzero-mean components can produce spectral impulses, so an ordinary integrable function is not always sufficient; generalized spectra may be required.

4.1 Worked PSD transformation

Suppose a zero-mean WSS voltage has

Using the transform pair e^-a|τ| ↔ 2a/(a²+(2πf)²) for a>0,

At f=1000 Hz,

As a consistency check,

4.2 Passage through an LTI system

If a WSS input X passes through a stable linear time-invariant system with frequency response H(f), the output PSD is

The output power is therefore

This relationship is fundamental to receiver noise calculations. It assumes that the output is the steady-state response and that the integral is finite.

For white input noise with constant two-sided PSD N₀/2,

For a low-pass filter, the equivalent noise bandwidth is commonly defined as

provided H(0)≠0. This is a one-sided bandwidth definition. If the filter has unity DC gain, output noise from one-sided white PSD N₀ is N₀B_n.

5. Gaussian Processes

A real random process is Gaussian if every finite collection X(t₁),…,X(t_n) has a joint Gaussian distribution. A real Gaussian process is completely specified by its mean function and covariance function. If it is WSS, uncorrelated samples are also independent.

Complex Gaussian processes require greater care. Let Z(t) have mean m_Z(t). Its covariance and complementary covariance are

In general, the mean and ordinary covariance alone do not completely specify a complex Gaussian process; the complementary covariance is also required. A complex process is proper when P_Z(t,s)=0 for all t and s. Consequently, a proper complex Gaussian process is specified by its mean and covariance. For a zero-mean complex Gaussian vector, properness also yields circular rotational symmetry.

Gaussian models are prevalent because thermal noise results from many microscopic contributions and because sums of many weak independent disturbances often approach a Gaussian distribution through central-limit behavior. Nevertheless, impulsive interference and nonlinear distortion can have heavy-tailed, non-Gaussian statistics.

6. Physical and Engineering Noise Models

6.1 Thermal noise

A resistor R at absolute temperature T produces approximately white noise over ordinary radio and electronic frequencies where hf≪kT. Here k is Boltzmann’s constant, k=1.380649×10^-23 J/K.

For the open-circuit resistor voltage:

• one-sided PSD for f≥0:

• corresponding two-sided PSD:

The convention must always be stated; integrating a one-sided formula over both positive and negative frequencies would double-count the noise.

A matched source at temperature T delivers available noise power

where B is the receiver’s equivalent noise bandwidth in hertz. At 290 K, kT≈4.00×10^-21 W/Hz, equivalent to approximately −174 dBm/Hz.

6.2 Shot and quantization noise

Shot noise is caused by discrete charge flow. Under the ideal independent-arrival model, a DC current I has one-sided current-noise PSD

where q=1.602176634×10^-19 C. Its two-sided value is qI A²/Hz. Device correlations or multiplication processes may require an excess-noise factor.

For an ideal uniform quantizer with step size Δ, the quantization error is often approximated as independent and uniform over [-Δ/2,Δ/2]. Under these assumptions,

The model becomes inaccurate when the input is small, periodic, overloaded or strongly correlated with the quantization error. Dither can improve the model by decorrelating the error.

6.3 White, colored and impulsive noise

White noise has constant PSD over the frequency range of interest. Ideal white noise has infinite total power if integrated over unlimited frequency and is therefore a mathematical abstraction. Physical systems always impose finite bandwidth.

Colored noise has frequency-dependent PSD, such as flicker noise, whose low-frequency behavior is approximately proportional to 1/f. Impulsive noise contains occasional high-amplitude events and is poorly summarized by variance alone.

7. SNR, Noise Figure and Receiver Performance

7.1 Signal-to-noise ratio

If signal and noise powers are measured at the same point and over the same bandwidth,

In decibels,

SNR is dimensionless. Voltage ratios may be converted using 20log₁₀(V_S/V_N) only when the voltages refer to the same impedance and represent compatible RMS quantities.

For ideal coherent BPSK in additive white Gaussian noise (AWGN), the bit-error probability is

where E_b is received energy per bit and N₀/2 is the two-sided noise PSD. This formula assumes coherent detection, equally likely bits, ideal timing and no coding or fading.

If noise power is N₀B in receiver bandwidth B and the bit rate is R_b, then

Bandwidth, rate and SNR must all refer to consistent points in the receiver.

7.2 Noise factor and noise figure

A device’s linear noise factor is

for a specified source reference temperature, conventionally T₀=290 K. Noise figure is

The equivalent input noise temperature is

For cascaded stages with linear available power gains G₁,G₂,… and noise factors F₁,F₂,…, Friis’ formula is

All gains and noise figures must be converted from decibels to linear ratios before substitution. The formula assumes compatible reference impedances or appropriately defined available/transducer gains and noise parameters.

Worked example—two-stage receiver. A low-noise amplifier has gain 10 dB and noise figure 2 dB. A following stage has noise figure 6 dB. Thus,

Therefore,

The first stage dominates, explaining why high gain and low noise are particularly valuable near the antenna.

Figure: Receiver cascade illustrating why first-stage gain and noise factor dominate total noise performance.

8. Applied Case: Rural Telemetry Receiver

Background

This illustrative case concerns a 200 kHz telemetry receiver operating at 290 K. The received signal is −100 dBm, receiver noise figure is 5 dB, and the uncoded bit rate is 100 kbit/s. The modem requires E_b/N₀=10 dB for its target error rate.

Problem

Determine the receiver noise power, SNR, energy-per-bit ratio and implementation margin. Assume the specified 200 kHz is the equivalent noise bandwidth and external interference is negligible.

Analysis

The thermal noise density at 290 K is approximately −174 dBm/Hz. Hence the input-referred receiver noise is

The resulting SNR is

Using E_b/N₀=SNR·B/R_b,

Approach and outcome

The link exceeds the modem requirement by approximately

This margin could accommodate propagation fading, cable loss, imperfect synchronization and component variation. It should not automatically be converted into higher transmit power or a higher data rate: spectral masks, modulation bandwidth, battery life and fading statistics must also be considered.

Learning points

• Noise bandwidth, not merely nominal channel spacing, determines integrated noise.
• Receiver noise figure raises the effective noise floor.
• SNR and E_b/N₀ are related through bandwidth and bit rate.
• A robust design allocates margin to quantified impairments rather than treating all uncertainty as Gaussian noise.

9. Measurement, Limitations and Professional Practice

Noise measurements require calibrated instruments, known bandwidths and consistent one-sided or two-sided PSD conventions. Spectrum-analyzer readings are affected by resolution bandwidth, detector type, averaging and instrument noise floor. Noise figure analyzers also require loss and mismatch corrections.

Models have limits. Gaussian AWGN does not represent intentional jamming, adjacent-channel leakage, oscillator phase noise or impulsive industrial interference. Stationarity may fail in mobile channels and time-varying networks. Estimated correlations can be biased when records are short or nonrepresentative.

Good engineering practice includes:

• recording temperature, bandwidth, impedance and PSD conventions;
• separating thermal noise, interference and distortion where possible;
• validating Gaussian and stationarity assumptions using measured data;
• propagating calibration uncertainty through reported SNR and noise figure;
• protecting captured communication data and removing personal identifiers;
• complying with spectrum allocations, emission limits and electromagnetic-compatibility regulations.

Contemporary software-defined radios estimate PSDs in real time using averaged periodograms. Machine-learning systems can classify interference or detect anomalous noise, but their outputs depend on representative training data and calibrated front ends. They should complement, not replace, physical noise models and reproducible measurements.

Chapter Summary

Probability quantifies uncertain communication events, while random variables describe numerical outcomes through distributions and moments. Conditional probability and Bayes’ theorem support detection and diagnosis. Random processes extend these ideas to time-varying waveforms; stationarity determines whether their statistics are time invariant, and ergodicity determines whether ensemble properties can be estimated from one record.

Autocorrelation measures temporal dependence, and the Wiener–Khinchin theorem relates it to power spectral density. An LTI system shapes input noise according to S_Y(f)=|H(f)|²S_X(f). Real Gaussian processes are specified by mean and covariance, whereas general complex Gaussian processes also require complementary covariance unless they are proper.

Thermal, shot, quantization and colored-noise models have different physical origins and assumptions. One-sided and two-sided PSD conventions must not be mixed. SNR, noise factor, noise figure and equivalent noise temperature quantify receiver quality. Friis’ formula demonstrates the dominant influence of the first receiver stage, while E_b/N₀ connects physical receiver performance to digital error probability.

Key Terms

Probability density function: Function whose integral over an interval gives the probability of a continuous random variable lying in that interval.

Cumulative distribution function: Probability that a random variable does not exceed a specified value.

Expected value: Probability-weighted average of a random variable.

Variance: Mean squared deviation from the mean.

Covariance: Measure of joint linear variation between two random variables.

Poisson process: Counting process with independent arrivals occurring at a constant average rate.

Random process: Ensemble of possible time-dependent sample functions.

Wide-sense stationarity: Constant mean and autocorrelation dependent only on time lag.

Ergodicity: Property allowing ensemble statistics to be obtained from long time averages.

Autocorrelation: Expected similarity between a process and a time-shifted version of itself.

Power spectral density: Distribution of average signal or noise power over frequency.

Gaussian process: Process whose every finite collection of samples is jointly Gaussian.

Proper complex process: Complex process with zero complementary covariance.

White noise: Idealized noise having constant PSD over frequency.

Thermal noise: Noise produced by random thermal motion of charge carriers.

Equivalent noise bandwidth: Rectangular bandwidth that passes the same white-noise power as an actual filter.

Noise factor: Ratio of input SNR to output SNR in linear units.

Noise figure: Noise factor expressed in decibels.

Review Questions

Short Answer Questions

1. Distinguish mutually exclusive events from independent events.
2. What is the difference between a PMF, PDF and CDF?
3. State the two conditions for wide-sense stationarity.
4. Why must a resistor noise PSD be identified as one-sided or two-sided?
5. Define noise factor and noise figure.

Long Answer Questions

1. Derive Bayes’ theorem from conditional probability and illustrate its use in receiver fault diagnosis.
2. Explain mean, variance, covariance and correlation coefficient, including their engineering interpretation.
3. Develop the relationship between autocorrelation and power spectral density and state the required assumptions.
4. Compare thermal, shot, quantization, white, colored and impulsive noise models.
5. Explain Gaussian processes, including why complementary covariance is required for a general complex Gaussian process.

Analytical / Application Questions

1. A WSS process has R_X(τ)=9e^-2000|τ| V². Derive its PSD, calculate its value at 500 Hz, and verify its total average power.
2. A three-stage receiver has gains of 12 dB and 15 dB for its first two stages, with noise figures of 1.5 dB, 5 dB and 8 dB. Calculate the total noise figure using Friis’ formula.
3. A receiver at 290 K has bandwidth 1 MHz, noise figure 4 dB and received signal power −95 dBm. Determine noise power, SNR and E_b/N₀ for a bit rate of 500 kbit/s.

Multiple Choice Questions with Answers

Question 1

A Poisson process has an arrival rate of 4 events/s. What are the mean and variance of the count in 0.5 s?

A. Mean 2, variance 2

B. Mean 4, variance 4

C. Mean 2, variance 4

D. Mean 8, variance 8

Correct Answer: A. Mean 2, variance 2

Question 2

Which statement about two random variables with finite variances is generally correct?

A. Zero covariance always implies independence

B. Independence implies zero covariance

C. Mutual exclusivity implies independence

D. Equal means imply zero covariance

Correct Answer: B. Independence implies zero covariance

Question 3

Which pair of conditions defines a wide-sense stationary process?

A. Zero mean and finite duration

B. Gaussian PDF and constant variance

C. Constant mean and autocorrelation dependent only on lag

D. Independent samples and constant PSD

Correct Answer: C. Constant mean and autocorrelation dependent only on lag

Question 4

For R_X(τ)=4e^-1000|τ| V², what is S_X(1000)?

A. 1.976×10^-4 V²/Hz

B. 1.689×10^-3 V²/Hz

C. 4.000×10^-3 V²/Hz

D. 8.000×10^-6 V²/Hz

Correct Answer: A. 1.976×10^-4 V²/Hz

Question 5

Which resistor voltage-noise PSD convention is correct in the classical thermal-noise region?

A. 4kTR V²/Hz two-sided and 2kTR V²/Hz one-sided

B. 4kTR V²/Hz one-sided and 2kTR V²/Hz two-sided

C. kTR V²/Hz for both conventions

D. 4kT/R V²/Hz one-sided

Correct Answer: B. 4kTR V²/Hz one-sided and 2kTR V²/Hz two-sided

Question 6

A WSS process with PSD S_X(f) passes through an LTI system with response H(f). What is the output PSD?

A. H(f)S_X(f)

B. |H(f)|S_X(f)²

C. |H(f)|²S_X(f)

D. S_X(f)/|H(f)|²

Correct Answer: C. |H(f)|²S_X(f)

Question 7

An amplifier has 10 dB gain and 2 dB noise figure. It is followed by a stage with 6 dB noise figure. What is the approximate total noise figure?

A. 2.00 dB

B. 2.75 dB

C. 6.00 dB

D. 8.00 dB

Correct Answer: B. 2.75 dB

Question 8

For coherent BPSK in AWGN under ideal detection assumptions, the bit-error probability is:

A. Q(√(E_b/N₀))

B. Q(E_b/N₀)

C. Q(√(2E_b/N₀))

D. exp(-E_b/N₀)

Correct Answer: C. Q(√(2E_b/N₀))

Question 9

Which information completely specifies a proper complex Gaussian process?

A. Its variance alone

B. Its PSD alone in every case

C. Its mean and ordinary covariance; properness makes complementary covariance zero

D. Its mean and complementary covariance only

Correct Answer: C. Its mean and ordinary covariance; properness makes complementary covariance zero

Question 10

At 290 K, a receiver has 1 MHz equivalent noise bandwidth and 3 dB noise figure. Using −174 dBm/Hz thermal-noise density, what is its input-referred noise power?

A. −174 dBm

B. −114 dBm

C. −111 dBm

D. −108 dBm

Correct Answer: C. −111 dBm