Multivariate Probability and Stochastic Processes

Introduction

Data science rarely concerns one uncertain quantity in isolation. A medical model may use age, blood pressure, glucose level, and medication status simultaneously; a financial model may represent returns across many assets; and a recommender system may describe sequences of user actions. Multivariate probability provides the language for dependence among several random variables, while stochastic-process theory describes random systems evolving over time.

This chapter develops joint, marginal, and conditional distributions; transformations of random vectors; multivariate Gaussian models; laws of large numbers; the central limit theorem; concentration inequalities; and Markov chains. The emphasis is on assumptions, interpretation, computation, and the limitations of each model.

Learning objectives

By the end of this chapter, you should be able to:

1. Define and manipulate joint, marginal, and conditional distributions.
2. Distinguish independence from merely having zero correlation.
3. Transform multivariate random variables using Jacobians.
4. Interpret covariance matrices and multivariate Gaussian distributions.
5. Explain the laws of large numbers, the central limit theorem, and concentration bounds.
6. Construct and analyze finite-state Markov chains.
7. Select suitable probabilistic models for data-science applications and recognize their limitations.

1. Random vectors and joint distributions

A random vector is a collection of random variables written as X = (X₁, …, X_d)^T. Its joint cumulative distribution function is

For a discrete random vector, the joint probability mass function is p_X(x) = P(X = x). It must satisfy p_X(x) ≥ 0 and ∑_x p_X(x) = 1. For a continuous vector, the joint density f_X(x) satisfies

where A is a measurable region in ℝ^d, and ∫_ℝ^d f_X(x) dx = 1. A density value is not itself a probability; probabilities are obtained by integration over regions.

Marginal distributions

A marginal distribution describes a subset of variables after the remaining variables have been ignored. For a discrete pair (X,Y),

For a continuous pair,

The same principle extends to higher dimensions by summing or integrating over unwanted coordinates.

Conditional distributions

For discrete variables, provided p_Y(y) > 0,

For continuous variables, provided f_Y(y) > 0,

The product rule follows directly:

Bayes’ rule reverses conditioning:

where

Bayesian inference uses this identity to update a prior distribution using observed evidence.

Independence and dependence

Two random variables are independent when

for all relevant values, with the analogous equation for probability mass functions. Independence implies zero covariance when the moments exist, but zero covariance does not generally imply independence. An important exception is the jointly Gaussian case: jointly Gaussian variables are independent if and only if their covariance is zero.

A useful measure of linear dependence is covariance:

where μ_X = E[X] and μ_Y = E[Y]. The correlation coefficient is

provided both standard deviations are positive and finite. Correlation is dimensionless and lies between −1 and 1.

2. Expectation and covariance of random vectors

The mean vector of X is

The covariance matrix is

Its entry in row i and column j is Σ_ij = Cov(X_i,X_j). A covariance matrix is symmetric and positive semidefinite: for every vector a,

This follows because a^TΣa = Var(a^T X). If Σ is positive definite, it is invertible and has positive determinant.

For a constant matrix A and vector b,

and

The additive constant changes the mean but not the covariance. Linear combinations are therefore central to dimensionality reduction, regression, portfolio theory, and uncertainty propagation.

3. Transformations of random vectors

Suppose X has density f_X(x), and Y = g(X). If g is one-to-one, differentiable, and has a differentiable inverse, then

where J_g⁻¹(y) is the Jacobian matrix of the inverse transformation. Equivalently, if y = g(x),

with x = g⁻¹(y). The absolute determinant measures local volume expansion; omitting it produces an incorrectly scaled density.

If a transformation is not one-to-one, the contributions from all inverse points must be summed:

when the inverse set is discrete and regular.

Worked example: sum and ratio coordinates

Let X and Y be independent exponential random variables with rate λ > 0, so f_X,Y(x,y)=λ²e^−λ(x+y) for x,y>0. Define U=X+Y and V=X/(X+Y). The inverse is X=UV and Y=U(1−V), with u>0 and 0<v<1. The Jacobian determinant is

Therefore,

on the stated support. This factors as

because the uniform density on (0,1) is 1. Thus V is uniform on (0,1) and independent of U, while U has a Gamma distribution with shape 2 and rate λ. The result illustrates how an appropriate transformation can reveal independence that is not obvious in the original coordinates.

4. The multivariate Gaussian model

A random vector X ∈ ℝ^d is multivariate normal with mean μ and covariance Σ, written X ∼ 𝒩_d(μ,Σ), when every linear combination a^T X is univariate normal. If Σ is positive definite, its density is

Here |Σ| is the determinant. The quadratic term is squared Mahalanobis distance. Unlike Euclidean distance, it accounts for scale and correlation.

Important properties include:

• Every marginal distribution is Gaussian.
• Every conditional distribution is Gaussian.
• Linear transformations remain Gaussian.
• If two subvectors are uncorrelated, they are independent.

Partition the vector and parameters as

Assuming Σ₁₁ is invertible, the conditional distribution is

The conditional mean is affine in the observed value. The conditional covariance is the Schur complement and does not depend on x₁. Conditioning cannot increase uncertainty in the positive-semidefinite ordering.

Worked example: conditional Gaussian prediction

Suppose (X₁,X₂) has mean (10,20), standard deviations 2 and 3, and correlation 0.6. Then

For X₁=12, the conditional mean of X₂ is

The conditional variance is

so the conditional standard deviation is 2.4. The observation raises the predicted value because the correlation is positive, while uncertainty is reduced from variance 9 to 5.76.

The Gaussian model is computationally attractive, but its assumptions can be inappropriate for heavy-tailed data, asymmetric variables, multimodal populations, or nonlinear dependence. In high dimensions, estimating Σ reliably requires substantial data; regularization or structured covariance models may be necessary.

5. Limit theorems and concentration

Laws of large numbers

Let X₁,X₂,… be independent and identically distributed with finite mean μ. The weak law of large numbers states that for every ε>0,

where X̄_n=(1/n)∑_i=1ⁿ X_i. Thus the sample mean converges in probability to the population mean. The strong law states the stronger almost-sure convergence X̄_n→μ under standard conditions, including finite expected absolute value for i.i.d. variables. Convergence is not the same as equality at finite n; sampling error remains.

If the variables are independent with common finite variance σ²,

This explains the familiar 1/√n decline in standard error, but it does not imply that every error probability declines exponentially.

Central limit theorem

For i.i.d. variables with mean μ and finite, positive variance σ², the central limit theorem states

where ⇒ denotes convergence in distribution. Consequently, for sufficiently large n,

The theorem concerns the distribution of normalized sums, not the claim that the original observations are Gaussian. Accuracy depends on sample size, skewness, tail behavior, and dependence. Berry–Esseen results provide finite-sample error bounds under a finite third absolute moment, but the bound can be loose.

Concentration inequalities

Concentration bounds provide non-asymptotic probability guarantees.

Markov’s inequality. If Z≥0 and a>0,

Chebyshev’s inequality. If Var(Z)<∞,

For independent variables bounded by a_i≤X_i≤b_i, Hoeffding’s inequality gives

For identically bounded variables with range length L, this becomes 2exp(−2nt²/L²). Boundedness and independence are essential conditions. Bernstein’s inequality can be sharper when the variance is small, while sub-Gaussian assumptions generalize exponential concentration beyond strictly bounded variables.

In machine learning, concentration supports generalization statements: an empirical average can approximate its expected value uniformly or for a fixed hypothesis. However, dependence among observations, adaptive data analysis, distribution shift, and multiple testing can invalidate naïve guarantees.

6. Markov chains

A discrete-time stochastic process {X_t:t=0,1,…} is a Markov chain if

The future is conditionally independent of the past given the present. For a finite state space, transition probabilities form a matrix P, where P_ij=P(X_t+1=j|X_t=i). Each entry is nonnegative and each row sums to 1.

If π_t is a row vector of state probabilities, then

and therefore π_t=π₀P^t. The (i,j) entry of P^t is the probability of moving from state i to state j in t steps.

A stationary distribution π satisfies

It describes an invariant distribution, not necessarily the distribution at every time. A finite irreducible chain has a unique stationary distribution. If it is also aperiodic, the distribution from any initial state converges to that stationary distribution. Irreducibility means every state can eventually reach every other state; aperiodicity excludes deterministic cycles.

A chain is reversible with respect to π if detailed balance holds:

for every pair of states. Reversibility is sufficient, but not necessary, for stationarity and often simplifies analysis.

Worked example: customer-state model

Consider a subscription service with states Basic (B), Premium (P), and Churned (C). Let

Churn is absorbing because the final row leaves the customer in C. Thus the only stationary distribution is π=(0,0,1). A short-run distribution can still contain active customers, but the long-run model predicts eventual churn with probability one. This conclusion may reflect the model’s design rather than reality: if reactivation is possible but omitted, the absorbing-state conclusion is misleading.

For a nonabsorbing chain, stationarity is found by solving the linear equations π=πP together with normalization. Transition matrices can be estimated from observed transitions, but sparse data create unstable estimates. Smoothing, Bayesian estimation, or low-dimensional parameterization may be appropriate.

7. Broader stochastic processes

A stochastic process is a family of random variables indexed by time or another parameter. Discrete-time processes use t=0,1,…, while continuous-time processes use t≥0. A process may be classified by its state space and time index: finite or continuous state, discrete or continuous time.

A Poisson process {N(t)} with rate λ>0 counts events over time and satisfies:

1. N(0)=0.
2. Independent increments over disjoint time intervals.
3. The number of events in an interval of length t is Poisson(λt).
4. Interarrival times are independent exponential random variables with rate λ.

For a Poisson count,

and both mean and variance equal λt. The process is useful for arrivals, failures, and transactions when a constant-rate, independent-arrival approximation is defensible.

A stochastic process can also be specified through conditional expectations, covariance functions, or differential equations. In time-series data, dependence, trend, seasonality, and changing variance must be addressed before applying independent-sample limit theorems.

8. Applied case: anomaly detection in industrial sensors

Background. An illustrative manufacturing system records temperature, vibration, and pressure every minute. Engineers want to identify abnormal operating states early without excessive false alarms.

Problem. Sensors are correlated, measurements are noisy, and operating conditions change between normal production and maintenance. Treating each sensor independently creates too many alerts.

Analysis. Let X_t∈ℝ³ be the sensor vector. During verified normal operation, estimate mean μ and covariance Σ. A Gaussian baseline assigns the squared Mahalanobis score

If the Gaussian model is appropriate and Σ is estimated accurately, D_t² is approximately chi-square with 3 degrees of freedom under normal operation. A threshold can therefore be selected to control an approximate false-alarm rate. In small samples or high dimensions, covariance shrinkage is preferable to an unstable inverse of the sample covariance.

Operating mode can be represented by a Markov chain with states Normal, Degraded, and Fault. Transition probabilities encode persistence: a degraded state is more likely to remain degraded than to return immediately to normal. Sensor evidence can update state probabilities through a hidden Markov model, in which the operating state is unobserved and sensor readings are emissions.

Approach and outcome. The organization should validate the Gaussian baseline separately by operating mode, estimate transition parameters from time-ordered data, and evaluate detection delay, false alarms, and missed faults. The resulting system combines multivariate dependence with temporal persistence rather than treating alerts as independent observations.

Learning points. A covariance-aware score detects joint anomalies; a Markov model captures temporal structure; and validation must respect time ordering. A mathematically elegant model is not automatically operationally reliable.

9. Limitations, governance, and best practice

Multivariate models can encode historical bias. If sensor coverage differs across factories or customer groups, estimated distributions may produce unequal error rates. Analysts should document data provenance, missingness, measurement error, and the population represented by the training sample.

Privacy risks arise because joint distributions can reveal sensitive combinations even when individual variables appear harmless. Access controls, aggregation, differential privacy where suitable, and retention limits should be considered. In regulated settings, threshold decisions and automated state classifications require auditability and human review.

Recommended practice includes:

• Check support, units, missingness, and outliers before fitting a distribution.
• Visualize marginal and pairwise structure, but do not infer independence from a plot alone.
• Use covariance regularization in high dimensions.
• Validate assumptions with held-out, time-aware data.
• Distinguish parameter uncertainty from random variation.
• Report uncertainty intervals and sensitivity to model choice.
• Monitor drift; stationarity and transition probabilities may change.
• Avoid interpreting correlation as causation.

Chapter Summary

Joint distributions describe several variables simultaneously, while marginalization removes variables by summing or integrating and conditioning incorporates observed information. Independence requires factorization of the joint distribution; zero covariance is weaker except in the jointly Gaussian setting. Random-vector means and covariance matrices summarize first- and second-order structure, and Jacobians correctly transform continuous densities.

The multivariate Gaussian model offers tractable marginals, conditionals, and linear transformations, but its assumptions must be tested. The laws of large numbers explain consistency of averages, the central limit theorem gives an asymptotic Gaussian approximation, and concentration inequalities provide finite-sample probability bounds under explicit conditions. Markov chains model state evolution using transition matrices; stationary behavior depends on irreducibility and aperiodicity. More general stochastic processes, including Poisson processes, describe event counts and temporal dependence. Effective data-science practice combines these tools with validation, regularization, governance, and awareness of distribution shift.

Key Terms

Random vector: A vector whose components are random variables.

Joint distribution: The probability law of two or more variables considered together.

Marginal distribution: The distribution of a subset obtained by summing or integrating out other variables.

Conditional distribution: The distribution of one variable given the value of another.

Independence: A factorization of a joint distribution into marginal distributions.

Covariance: A measure of joint linear variation between two variables.

Correlation: Standardized covariance, bounded between −1 and 1.

Covariance matrix: Matrix containing all pairwise covariances of a random vector.

Jacobian: Matrix of first derivatives used to measure local transformation volume.

Mahalanobis distance: Covariance-adjusted distance from a mean vector.

Multivariate Gaussian: A distribution whose every linear combination is Gaussian.

Weak law of large numbers: Convergence in probability of an average to its expected value.

Strong law of large numbers: Almost-sure convergence of an average to its expected value.

Central limit theorem: A limit result giving a Gaussian law for normalized sums.

Concentration inequality: A finite-sample bound on deviation from an expected value.

Markov chain: A process whose next state depends on the past only through the current state.

Transition matrix: Matrix of one-step Markov-chain probabilities.

Stationary distribution: A distribution unchanged by application of the transition matrix.

Poisson process: A counting process with independent stationary increments and constant rate.

Hidden Markov model: A model with an unobserved Markov state and observed emissions.

Review Questions

Short Answer Questions

1. What is the difference between a joint distribution and a marginal distribution?
2. State the condition for independence of two continuous random variables.
3. Why is the Jacobian determinant required in a change-of-variables calculation?
4. What does a stationary distribution represent in a Markov chain?
5. Distinguish convergence in probability from convergence in distribution.

Long Answer Questions

1. Derive the conditional mean and covariance of a partitioned multivariate Gaussian distribution and interpret both results.
2. Explain the covariance matrix’s mathematical properties and its role in multivariate modeling.
3. Compare the weak law of large numbers, the central limit theorem, and Hoeffding’s inequality.
4. Explain irreducibility, periodicity, stationarity, and long-run convergence for finite Markov chains.
5. Discuss the assumptions and limitations of using a multivariate Gaussian model for anomaly detection.

Analytical / Application Questions

1. A data scientist observes two correlated sensor measurements. Design a covariance-aware anomaly score and explain how you would validate its threshold.
2. Construct a three-state Markov model for customer retention, identify whether churn is absorbing, and explain the managerial implications of that assumption.
3. A sample contains dependent, time-ordered observations. Determine which limit-theorem or concentration assumptions fail and propose a suitable validation strategy.

Multiple Choice Questions with Answers

Question 1

Which operation produces the marginal density f_X(x) from a continuous joint density f_{X,Y}(x,y)?

A. Differentiate with respect to y

B. Integrate over all possible y values

C. Multiply by f_Y(y)

D. Set y equal to its mean

Correct Answer: B. Integrate over all possible y values

Question 2

If two jointly Gaussian variables have covariance zero, what follows?

A. They are independent

B. They must have equal means

C. They must be uniformly distributed

D. Their variances are zero

Correct Answer: A. They are independent

Question 3

For Y = g(X), what factor is required when transforming a continuous multivariate density?

A. The trace of the covariance matrix

B. The expectation of Y

C. The absolute determinant of the relevant Jacobian

D. The correlation coefficient

Correct Answer: C. The absolute determinant of the relevant Jacobian

Question 4

If X has covariance matrix Σ and Y = AX + b, what is Cov(Y)?

A. AΣAᵀ

B. AΣ + b

C. Σ + AAᵀ

D. bΣbᵀ

Correct Answer: A. AΣAᵀ

Question 5

For i.i.d. observations with finite variance σ², what is Var(X̄_n)?

A. σ²n

B. σ²/n

C. σ/n²

D. n/σ²

Correct Answer: B. σ²/n

Question 6

Which statement best describes the central limit theorem?

A. Every observation becomes Gaussian as n increases

B. The normalized sample mean approaches a Gaussian distribution under stated conditions

C. The sample mean equals the population mean for every sufficiently large sample

D. It applies only when the observations are uniformly distributed

Correct Answer: B. The normalized sample mean approaches a Gaussian distribution under stated conditions

Question 7

What is required of every row of a finite-state Markov transition matrix?

A. Its entries must sum to zero

B. Its entries must sum to one and be nonnegative

C. Its diagonal entries must all equal one

D. It must be symmetric

Correct Answer: B. Its entries must sum to one and be nonnegative

Question 8

A distribution π satisfying π = πP is called what?

A. A marginal distribution

B. A conditional density

C. A stationary distribution

D. A Jacobian distribution

Correct Answer: C. A stationary distribution

Question 9

In a Poisson process with rate λ, what is the distribution of the number of events in time t?

A. Normal with mean t

B. Binomial with parameters λ and t

C. Poisson with mean λt

D. Exponential with mean λt

Correct Answer: C. Poisson with mean λt

Question 10

Which inequality requires a nonnegative random variable and a positive threshold?

A. Markov’s inequality

B. The central limit theorem

C. The Gaussian conditioning formula

D. The covariance identity

Correct Answer: A. Markov’s inequality