Joint Distributions and Dependence Modelling

Introduction

Many quality-management decisions involve several random quantities observed on the same unit, process, batch, or time period. Examples include the strength and thickness of a component, cycle time and defect count, or supplier lead time and incoming quality. Analysing each variable separately can conceal important relationships. Joint distributions provide the mathematical framework for describing such variables simultaneously, while dependence models and copulas allow analysts to separate marginal behaviour from the way variables interact.

This chapter develops the theory of random vectors, joint, marginal, and conditional distributions; covariance and correlation; important multivariate distributions; transformations; dependence structures; and copulas. The emphasis is on interpretation, assumptions, calculation, and application in advanced quality and operational analysis.

Learning Objectives

By the end of this chapter, the reader should be able to:

1. Define random vectors and distinguish joint, marginal, and conditional distributions.
2. Calculate expectations, variances, covariance, and correlation from joint models.
3. Explain independence and distinguish it from zero correlation.
4. Work with multivariate normal and other important multivariate distributions.
5. Derive distributions of transformed random variables using Jacobians or distribution functions.
6. Explain dependence structures and construct models using copulas.
7. Apply joint-distribution methods to quality, reliability, and risk problems.
8. Evaluate assumptions, limitations, and governance issues in dependence modelling.

1. Random Vectors and Joint Distributions

A random vector is an ordered collection of random variables. For two variables, write

X = (X₁, X₂)

where X₁ and X₂ may represent measurements on the same unit. For example, X₁ may be tensile strength and X₂ may be thickness.

The joint cumulative distribution function (CDF) is

It gives the probability that both variables fall below specified limits. A valid joint CDF must be nondecreasing in each argument, right-continuous, and satisfy the appropriate boundary conditions.

Discrete joint distributions

For discrete variables, the joint probability mass function (PMF) is

The probabilities must satisfy

Continuous joint distributions

For continuous variables, the joint probability density function (PDF), denoted f_X1,X2(x₁,x₂), satisfies

for any suitable region A. The density must be nonnegative and integrate to one over the entire support. A density value is not itself a probability; probabilities are obtained by integration over regions.

For a continuous pair,

The joint density is recovered, where differentiability holds, by

2. Marginal and Conditional Distributions

A marginal distribution describes one component without specifying the value of the other component.

For discrete variables,

For continuous variables,

The corresponding formula for f_X2(x₂) integrates over x₁.

A conditional distribution describes one variable given information about another. For discrete variables,

provided p_X1(x₁)>0. For continuous variables,

provided f_X1(x₁)>0. Bayes' rule follows directly:

Conditional expectation is central to prediction and quality analytics. The law of total expectation states

The law of total variance is

The first term represents average residual variation; the second represents variation in conditional means between groups or operating conditions.

Worked Example 1: A discrete joint distribution

Suppose the joint PMF of defect count D and inspection status S is given below. Here S=1 means that an inspection alarm is triggered.

The probability of at least one defect and an alarm is

P(D≥1,S=1)=0.15+0.10=0.25.

The conditional probability of an alarm given at least one defect is

P(S=1|D≥1)=0.25/(0.25+0.15)=0.625.

Thus, the alarm probability is 62.5% among units with at least one defect. This conditional result is more informative for diagnostic evaluation than the unconditional alarm probability of 35%.

3. Independence and Dependence

Two random variables are independent if knowledge of one does not change the distribution of the other. Formally,

for all values, or, in the continuous case,

For discrete variables the analogous factorisation applies to PMFs. Independence implies zero covariance when the relevant moments exist, but zero covariance does not generally imply independence. The distinction is crucial in nonlinear processes.

For example, let X be symmetrically distributed about zero and let Y=X². Then X and Y may have zero covariance because positive and negative values of X cancel, but Y is completely determined by X; they are not independent.

4. Moments, Covariance, and Correlation

For a random vector X=(X₁,...,X_p)ᵀ, the mean vector is

The covariance matrix is

Its element in row i and column j is

The diagonal elements are variances. A covariance matrix is symmetric and positive semidefinite. For any vector a, aᵀΣa≥0, because it is the variance of aᵀX.

The correlation coefficient is

where σ_i=√Var(X_i) and σ_j=√Var(X_j). It is dimensionless and lies between −1 and 1. Correlation measures linear association, not all forms of dependence.

For a linear combination Y=aᵀX,

These formulas support measurement-system analysis, portfolio-style risk aggregation, and propagation of uncertainty.

Sample covariance and correlation

For observations (x_i1,...,x_ip), the sample covariance between variables j and k is

where n is sample size and x̄_j is the sample mean. The divisor n−1 gives the usual unbiased estimator of covariance under independent identically distributed sampling, though dependence, missingness, and clustered data require additional care.

5. Important Multivariate Models

Multivariate normal distribution

A p-dimensional vector X has a multivariate normal distribution with mean vector μ and covariance matrix Σ, written X∼N_p(μ,Σ), if its density is

when Σ is positive definite. If Σ is singular, a density with respect to ordinary p-dimensional volume does not exist, although a degenerate normal distribution can still be defined.

Important properties include:

• Every marginal distribution is normal.
• Every linear combination of components is normal.
• For a jointly normal vector, zero covariance is equivalent to independence.
• Conditional distributions are normal.

Partition X and μ as X=(X₁,X₂), with covariance blocks Σ₁₁,Σ₁₂,Σ₂₁,Σ₂₂. Then

The conditional covariance is the Schur complement and is no larger, in the positive-semidefinite ordering, than the unconditional covariance. Observing related variables therefore reduces uncertainty under the correctly specified model.

Multinomial distribution

If n independent trials fall into k mutually exclusive categories with probabilities p₁,...,p_k satisfying ∑p_j=1, the count vector N=(N₁,...,N_k) has PMF

where ∑n_j=n. Multinomial counts are dependent because their total is fixed. In fact, for i≠j, Cov(N_i,N_j)=−np_i p_j.

Multivariate t distribution

The multivariate t distribution is useful when heavier tails than the normal distribution are needed. It can be represented as

where Z is multivariate normal with covariance matrix Σ, W is chi-square with ν degrees of freedom, and they are independent. Small ν produces heavier tails. This feature is important in extreme-risk and reliability applications, but parameter interpretation and existence of moments must be checked; for example, the mean requires ν>1, and finite covariance requires ν>2.

6. Transformations of Random Vectors

Transformations are used to obtain distributions of derived quality indicators, such as total cost, average dimension, stress-to-strength ratio, or principal components.

If Y=g(X) is a one-to-one differentiable transformation from x to y, with inverse x=h(y), then the transformed density is

where J_h(y)=∂(x₁,...,x_p)/∂(y₁,...,y_p) is the Jacobian determinant. The absolute value is essential because densities cannot be negative.

For non-one-to-one transformations, partition the domain into one-to-one regions and sum the contributions. Alternatively, derive the CDF directly.

Worked Example 2: Sum and difference

Let X and Y be independent normal variables with means μ_X,μ_Y and variances σ_X²,σ_Y². Define

U=X+Y and V=X−Y.

Because linear transformations of jointly normal variables remain normal,

Their covariance is

Therefore, U and V are independent if and only if σ_X²=σ_Y², because they are jointly normal. This conclusion depends on joint normality; uncorrelated transformed variables are not generally independent outside that setting.

7. Dependence Modelling and Copulas

A dependence model should distinguish two questions:

1. What is the distribution of each variable individually?
2. How are the variables associated, especially in the tails?

A copula answers the second question. Sklar's theorem states that for any joint CDF H with marginal CDFs F₁,...,F_p, there exists a copula C such that

If all margins are continuous, the copula is unique. A copula maps marginal probabilities to the unit interval and combines them into a joint distribution. This permits, for example, a lognormal production-time margin to be combined with a discrete or heavy-tailed defect-risk margin.

Common copula families

A Gaussian copula is constructed by transforming uniform variables through the standard normal quantile function and imposing a multivariate normal dependence matrix. It should not be confused with a multivariate normal distribution: the margins can be nonnormal.

Rank-based dependence

Pearson correlation measures linear association and is sensitive to scale and outliers. Spearman's rho is the Pearson correlation of ranks. Kendall's tau is based on concordant and discordant pairs. Rank measures are invariant under strictly increasing transformations, making them useful when marginal scales are arbitrary or nonnormal.

Tail dependence is especially important in quality and operational risk. For lower tails, the coefficient is conceptually

where U₁ and U₂ are uniform marginal variables. Upper-tail dependence is defined similarly as u→1− for joint exceedance probabilities. These limits may be zero even when variables are substantially correlated in the centre.

Estimation workflow

A defensible copula analysis typically involves:

1. Define the unit of analysis and dependence-relevant time horizon.
2. Fit and diagnose each marginal distribution.
3. Transform observations to approximately uniform values using fitted CDFs or rank methods.
4. Estimate candidate copula parameters.
5. Compare candidates using likelihood, information criteria, graphical diagnostics, and tail fit.
6. Validate through out-of-sample or stress-testing analysis.
7. Propagate parameter uncertainty into reported probabilities and decisions.

Ignoring serial, spatial, batch, supplier, or common-cause dependence can seriously understate joint-failure probabilities.

8. Applied Case: Correlated Quality Characteristics

Background

The following is an illustrative case. A manufacturer measures coating thickness X₁ in micrometres and adhesion strength X₂ in megapascals on finished units. A unit is acceptable only if

40≤X₁≤60 and X₂≥8.

Historical data suggest a bivariate normal model with

μ₁=50, σ₁=4, μ₂=10, σ₂=1.2, and ρ=0.60.

Problem

Management has been using separate marginal acceptance rates and multiplying them, implicitly assuming independence. Is that justified?

Analysis

The standardized limits for thickness are −2.5 and 2.5, while the adhesion limit is

(8−10)/1.2=−1.667.

The marginal probability of acceptable thickness is approximately

Φ(2.5)−Φ(−2.5)≈0.9876,

and the marginal probability of acceptable adhesion is approximately

1−Φ(−1.667)≈0.9522.

Under independence, the estimated joint acceptance would be approximately

0.9876×0.9522≈0.9404.

However, the positive correlation means that high thickness tends to occur with high adhesion, while low thickness tends to occur with low adhesion. Because the adhesion failure threshold is in the lower tail, the correct joint probability must be evaluated using the bivariate normal CDF rather than the product of marginal probabilities:

where (Z₁,Z₂) is standard bivariate normal with correlation 0.60. Numerical software should calculate this probability using a documented bivariate-normal routine. The key managerial conclusion is not to substitute marginal multiplication for a joint calculation. The direction and magnitude of the difference must be quantified before setting capability targets or release rules.

Approach and outcome

The quality team fits the bivariate model, checks scatterplots and marginal normality, computes the joint acceptance probability, and performs sensitivity analysis for correlations from 0.3 to 0.8. It also evaluates a copula with empirical margins because the adhesion data show heavier tails than a normal distribution.

The resulting release policy uses the joint probability and reports uncertainty from estimated parameters. Process engineers then investigate the common curing condition that generates the positive association. The analysis changes the improvement programme from two independent inspections to control of a shared process cause.

Learning points

• Marginal capability indices do not determine joint conformity.
• Correlation can arise from common causes, not merely measurement coincidence.
• Tail behaviour may matter more than central correlation.
• Model validation and sensitivity analysis are essential before operational decisions.

9. Limitations, Governance, and Contemporary Practice

Joint models are conditional representations of reality. Common limitations include nonstationarity, hidden subpopulations, censoring, measurement error, small samples, outliers, and dependence that changes over time. A single correlation estimate may conceal regime changes or asymmetric tail dependence.

In quality governance, analysts should document data lineage, sampling design, variable definitions, missing-data treatment, model selection, and approval thresholds. Models used for product release or safety decisions require traceability and independent validation. Automated and AI-assisted tools can help discover nonlinear associations, estimate flexible copulas, and monitor drift, but they do not remove the need for distributional assumptions, uncertainty quantification, explainability, and human accountability.

Best practice is to compare several plausible models, assess calibration on relevant regions of the support, stress-test dependence parameters, and communicate both model results and limitations. Dependence assumptions should never be hidden inside software defaults.

Chapter Summary

Joint distributions describe multiple random quantities simultaneously. Marginal distributions are obtained by summing or integrating out other variables, while conditional distributions quantify updated uncertainty given information. Independence requires factorisation of the full joint distribution and is stronger than zero correlation. Covariance matrices summarise second-order dependence and determine the variance of linear combinations. The multivariate normal model is analytically convenient, particularly because its conditional distributions are normal and zero covariance implies independence within the model. Transformations use linear algebra, distribution functions, or Jacobians. Copulas separate marginal distributions from dependence and are especially useful for nonnormal margins and tail-risk analysis. In professional quality management, joint modelling supports more realistic conformity, reliability, supplier-risk, and common-cause decisions, provided assumptions are validated and uncertainty is governed.

Key Terms

Random vector: An ordered collection of random variables.

Joint CDF: The probability that all components of a random vector are below specified values.

Joint PMF: Probability assigned to each combination of values for discrete variables.

Joint PDF: Density describing a continuous random vector.

Marginal distribution: The distribution of one component after integrating or summing over the others.

Conditional distribution: The distribution of one variable given a value or event involving another.

Independence: A factorisation property in which knowledge of one variable does not alter the distribution of another.

Covariance: A measure of joint linear variation between two variables.

Correlation: Standardised covariance ranging from −1 to 1.

Covariance matrix: Matrix containing all variances and covariances of a random vector.

Multivariate normal distribution: A multivariate model in which every linear combination is normally distributed.

Jacobian: Determinant describing local volume change under a differentiable transformation.

Transformation: A function used to construct new random variables from existing ones.

Copula: A multivariate distribution on the unit cube that links marginal distributions.

Tail dependence: Association between unusually low or high values.

Gaussian copula: A copula whose dependence structure is generated by a multivariate normal distribution.

Student t copula: A copula capable of representing symmetric tail dependence.

Spearman's rho: Correlation calculated from ranks.

Kendall's tau: Dependence measure based on concordant and discordant pairs.

Schur complement: Matrix expression giving conditional covariance in a multivariate normal model.

Review Questions

Short Answer Questions

1. What is the difference between a joint distribution and a marginal distribution?
2. State the factorisation condition for independence.
3. Why does zero covariance not generally imply independence?
4. What does a covariance matrix contain?
5. What purpose does a copula serve in dependence modelling?

Long Answer Questions

1. Derive the marginal and conditional density formulas for two continuous random variables and explain the required positivity condition.
2. Explain the multivariate normal distribution and derive the mean and variance of a linear combination.
3. Compare Pearson correlation, Spearman's rho, and Kendall's tau in terms of interpretation and robustness.
4. Explain the Jacobian method for transforming a multivariate density, including the role of one-to-one mappings.
5. Discuss how dependence assumptions can affect quality-release and reliability decisions.

Analytical / Application Questions

1. A process has two correlated quality characteristics. Design a statistical workflow for estimating the probability that both meet specification, including model validation and sensitivity analysis.
2. Explain why a Gaussian copula may be inadequate for joint extreme failures and propose a suitable alternative modelling strategy.
3. A supplier's delivery time and defect rate show changing dependence across production regimes. Develop a dependence-modelling approach that addresses regime changes, tail behaviour, and governance requirements.

Multiple Choice Questions with Answers

Question 1

For continuous variables, how is the marginal density f_{X_1}(x_1) obtained from the joint density?

A. By differentiating with respect to x_2

B. By integrating over all possible x_2 values

C. By multiplying the joint density by x_1

D. By setting x_2 equal to its mean

Correct Answer: B. By integrating over all possible x_2 values

Question 2

Which statement is correct about independence?

A. Zero covariance always implies independence

B. Independence implies zero covariance when covariance exists

C. Independence is possible only for normal variables

D. Correlation must equal 1 under independence

Correct Answer: B. Independence implies zero covariance when covariance exists

Question 3

What is the range of a correlation coefficient?

A. 0 to 1

B. −∞ to ∞

C. −1 to 1

D. −2 to 2

Correct Answer: C. −1 to 1

Question 4

For Y=aᵀX with covariance matrix Σ, which formula gives Var(Y)?

A. aᵀΣa

B. aΣ

C. Σᵀa

D. aᵀΣ⁻¹a

Correct Answer: A. aᵀΣa

Question 5

In a multivariate normal distribution, what does zero covariance between two components imply?

A. They have identical means

B. They are independent

C. They have equal variances

D. They are uniformly distributed

Correct Answer: B. They are independent

Question 6

What is the principal role of a Jacobian in a multivariate transformation?

A. It changes a discrete variable into a continuous one

B. It accounts for local volume change

C. It guarantees independence

D. It removes covariance

Correct Answer: B. It accounts for local volume change

Question 7

What does Sklar's theorem connect?

A. A joint distribution with its marginal distributions through a copula

B. A mean with its sample size

C. A covariance with a regression intercept

D. A PMF with a measurement scale

Correct Answer: A. A joint distribution with its marginal distributions through a copula

Question 8

Which copula is specifically associated with stronger upper-tail dependence?

A. Gumbel

B. Clayton

C. Independence copula only

D. Uniform marginal model

Correct Answer: A. Gumbel

Question 9

If X and Y are independent with variances 4 and 9, respectively, what is Var(X+Y)?

A. 5

B. 13

C. 36

D. 1

Correct Answer: B. 13

Question 10

Why can marginal acceptance probabilities not generally be multiplied to obtain joint acceptance probability?

A. Marginal probabilities are always biased

B. The variables may be dependent

C. Joint probabilities do not exist

D. Normal distributions cannot be used for quality data

Correct Answer: B. The variables may be dependent