Conditional Expectation, Convergence and Limit Theorems

Introduction and Context

Advanced quality management depends on decisions made under uncertainty. A process engineer may need to estimate defect risk after observing a supplier, determine whether a control chart statistic stabilizes, or approximate the probability that a large production lot exceeds a defect threshold. Conditional expectation, convergence concepts, laws of large numbers, and central limit theorems provide the mathematical foundation for these tasks.

The central idea is that probability models become more informative when they incorporate available information. Conditional expectation converts partial information into an updated prediction. Convergence describes how estimators, process measurements, or simulation outputs behave as the sample size increases. Limit theorems explain why averages often become stable and approximately normal, even when individual observations are not normally distributed.

These results are especially valuable in Total Quality Management (TQM), where decisions concern process capability, supplier performance, reliability, inspection, sampling, and continuous improvement.

Learning Objectives

By the end of this chapter, the reader should be able to:

1. Define and interpret conditional expectation and conditional variance.
2. Apply the law of total expectation and the law of total variance.
3. Distinguish almost sure, in-probability, in-distribution, and mean-square convergence.
4. Use the weak and strong laws of large numbers in quality applications.
5. Apply the central limit theorem (CLT) and understand its assumptions and limitations.
6. Use normal, Poisson, and continuity-corrected approximations appropriately.
7. Evaluate uncertainty, sampling risk, and managerial implications in complex quality problems.

1. Conditional Expectation as an Information-Based Prediction

Let X be a random variable and let Y represent observed information. The conditional expectation E[X | Y] is the best prediction of X based on the value of Y, under squared-error loss. In a discrete model,

where the sum is over possible values of X and P(X = x | Y = y) is defined when P(Y = y) > 0. For continuous variables, the corresponding expression is

where f_{X|Y} is the conditional density. The units of E[X | Y] are the same as those of X.

More generally, E[X | Y] is itself a random variable: after observing Y, it takes the value E[X | Y = y]. It is therefore useful to write E[X | Y], rather than only E[X | Y = y].

Example 1: Supplier-related defect risk

Suppose a lot comes from either a certified supplier C or a non-certified supplier N. The prior probabilities are P(C) = 0.6 and P(N) = 0.4. The defect probability is 0.01 for C and 0.04 for N. Let D be the indicator that a randomly selected unit is defective. Then

Thus the unconditional defect probability is 2.2%. If the supplier category is known, the conditional prediction is either 1% or 4%, which is more useful for supplier-specific action.

Conditional expectation as a projection

For any square-integrable random variable X, E[X | Y] is the minimum mean-square predictor among all functions g(Y):

The inequality holds for every measurable g(Y) for which the expectations exist. This projection interpretation explains why conditional expectation is central to forecasting, Bayesian updating, regression, filtering, and statistical process monitoring.

Fundamental properties

Assuming the stated expectations exist:

• Linearity: E[aX + bZ | Y] = aE[X | Y] + bE[Z | Y].
• If X ≥ 0, then E[X | Y] ≥ 0.
• If X ≤ Z, then E[X | Y] ≤ E[Z | Y].
• If X is determined by Y, then E[X | Y] = X.
• If X and Y are independent, then E[X | Y] = E[X].

The last property is important: information cannot improve prediction when it is statistically irrelevant.

2. Total Expectation and Total Variance

The law of total expectation states

For a partition into groups A₁, …, Aₖ with positive probabilities,

This is a weighted average of subgroup means. It is valid when X is integrable, meaning E[|X|] < ∞.

The law of total variance decomposes overall variation into within-group and between-group components:

The first term is expected variation remaining within groups; the second is variation in group means. This decomposition is valuable in quality improvement because it distinguishes process noise from variation caused by shifts among machines, operators, suppliers, or time periods.

Example 2: Decomposing defect variation

Assume a production line uses Machine 1 with probability 0.7 and Machine 2 with probability 0.3. The defect probabilities are p₁ = 0.01 and p₂ = 0.05. For a Bernoulli defect indicator D,

Given the machine, Var(D | M = i) = pᵢ(1 − pᵢ). Therefore,

The between-machine component is

Their sum is approximately 0.0210504, which equals Var(D) for an overall Bernoulli probability of 0.022: 0.022(0.978) = 0.021516? The apparent discrepancy signals an arithmetic issue in the within-component calculation. Recomputing:

0.7 × 0.0099 = 0.00693 and 0.3 × 0.0475 = 0.01425, so E[Var(D|M)] = 0.02118. The between-machine component is 0.0004704. The total is 0.0216504, still not equal to 0.021516 because the between component calculation must use the exact weighted variance: 0.7(−0.012)² + 0.3(0.028)² = 0.0001008 + 0.0002352 = 0.000336. Hence total variance = 0.02118 + 0.000336 = 0.021516. The example illustrates the importance of checking both arithmetic and the identity.

3. Conditional Variance and Conditional Independence

Conditional variance is defined by

Equivalently,

Conditional independence means that, after conditioning on relevant information, additional variables provide no further dependence. For example, two inspection results may be conditionally independent given the true defect state, even though they are marginally dependent because both reflect that state.

In hierarchical quality models, conditional independence can simplify analysis. If units are independent given a batch-specific defect rate P, then observations may be correlated marginally because they share P. Ignoring this hierarchy can understate uncertainty and produce overconfident control limits.

4. Modes of Convergence

Let Xₙ be a sequence of random variables and X a target random variable. Different modes of convergence describe different meanings of “approaches.”

Almost sure convergence

Xₙ converges almost surely to X, written Xₙ → X almost surely, if

This means that, except on an event of probability zero, the realized sequence eventually converges pointwise. Almost sure convergence is strong and closely related to long-run operational behavior.

Convergence in probability

Xₙ converges in probability to X, written Xₙ →ₚ X, if for every ε > 0,

This says that the probability of a practically meaningful error tends to zero. It is often the natural criterion for estimator consistency.

Convergence in distribution

Xₙ converges in distribution to X, written Xₙ →ᵈ X, if the distribution functions satisfy

at every continuity point x of F, where Fₙ and F are the distribution functions of Xₙ and X. This mode concerns distributions rather than individual sample paths. It is the mode used most frequently in the CLT.

Mean-square convergence

Xₙ converges in mean square to X if

It requires finite second moments and is useful when squared prediction error matters.

Relationships among modes

Under standard conditions,

Also, mean-square convergence implies convergence in probability. The converses generally do not hold. Consequently, stating only that convergence occurs is incomplete; the mode must be identified.

Slutsky’s theorem and the continuous mapping principle

If Xₙ →ᵈ X and Yₙ →ₚ c, where c is a constant, then

and, where division is valid,

The continuous mapping principle states that if Xₙ →ᵈ X and g is continuous, then g(Xₙ) →ᵈ g(X). These results justify replacing unknown standard errors with consistent estimates in asymptotic statistics.

5. Laws of Large Numbers

Let X₁, X₂, … be observations with common mean μ and define the sample mean

Weak law of large numbers

If X₁, …, Xₙ are independent and identically distributed (i.i.d.) with E[|X₁|] < ∞, then for every ε > 0,

Thus X̄ₙ →ₚ μ. The theorem supports statements such as: with sufficiently many independently sampled units, the observed defect fraction is likely to be close to the true defect probability.

Strong law of large numbers

Under i.i.d. sampling with E[|X₁|] < ∞,

The strong law gives a pathwise long-run guarantee, not a probability statement for one fixed sample size.

Practical qualifications

The laws do not automatically apply when observations are dependent, the process is nonstationary, or the mean does not exist. Autocorrelation, drift, batch effects, and adaptive inspection must be investigated. In quality systems, increasing sample size cannot repair systematic sampling bias.

6. Central Limit Theorem

The classical i.i.d. CLT states that if X₁, X₂, … are i.i.d. with finite mean μ and finite, positive variance σ², then

Equivalently, for large n,

where N(μ, σ²) denotes a normal distribution with mean μ and variance σ². The standard deviation of the mean is σ/√n, called the standard error when σ is known.

If σ is unknown, it is commonly replaced by the sample standard deviation Sₙ, provided Sₙ is a consistent estimator of σ:

For small samples, especially from skewed or heavy-tailed populations, the approximation may be poor. The sample size required depends on skewness, tail behavior, dependence, and the accuracy demanded.

CLT for proportions

If Xᵢ are Bernoulli(p), then the sample proportion P̂ = X̄ₙ has mean p and variance p(1 − p)/n. For sufficiently large n,

A common rule for a normal approximation is np ≥ 10 and n(1 − p) ≥ 10, although this is a practical guideline rather than a universal theorem.

CLT for sums and quality costs

For a sum Sₙ = ∑Xᵢ,

If Xᵢ is a unit-level cost in monetary units, nμ is the expected lot cost and √(nσ²) is its standard deviation in monetary units. Unit consistency is essential: the mean of a sum has the same units as an individual cost, while variance has squared units.

7. Distributional Approximations

Binomial-to-normal approximation

If K ~ Binomial(n, p), then E[K] = np and Var(K) = np(1 − p). For sufficiently large n,

For an integer event, continuity correction improves accuracy. For example,

where Y ~ N(np, np(1 − p)). Standardization uses

Poisson approximation

If K ~ Binomial(n,p), n is large, p is small, and λ = np is moderate, then

The Poisson model has P(K = k) = e^(−λ)λᵏ/k! for k = 0,1,2,… and mean and variance both equal λ. It is useful for rare defects, failures, or complaints, but the approximation should be checked when p is not small or when events are dependent.

Delta method

If √n(Tₙ − θ) →ᵈ N(0, τ²) and g is differentiable at θ, then

This is useful when a quality metric is a nonlinear function of an estimated parameter, such as a logarithm of a failure rate or a ratio of two estimated means.

8. Worked Quality Example: Sampling a Defect Rate

An organization audits n = 400 independently selected units. The true defect rate is p = 0.02. Let K be the number of defective units and P̂ = K/400.

The expected number of defects is np = 8, and the variance is np(1 − p) = 400(0.02)(0.98) = 7.84. Hence the standard deviation is √7.84 ≈ 2.80.

Using the normal approximation with continuity correction, the probability of observing at most 10 defects is

The standardized value is approximately 0.893. Therefore P(K ≤ 10) is approximately Φ(0.893) ≈ 0.814. The continuity correction matters because K is discrete while the normal approximation is continuous.

The approximation is defensible because np = 8 is slightly below the common guideline of 10; consequently, an exact binomial calculation or simulation would be preferable for a high-stakes decision. This illustrates an important professional principle: an approximation is a model choice, not a substitute for validation.

9. Illustrative Case: Stabilizing a Multi-Line Inspection System

Background

A manufacturer operates three filling lines. Historical data suggest that line-specific defect probabilities are 0.008, 0.015, and 0.030. Production shares are 0.50, 0.30, and 0.20, respectively. The quality manager wants to estimate the overall defect rate and determine whether a large audit can reliably detect a rate above 2%.

Problem

The lines differ in both operating conditions and defect risk. Treating all observations as if they came from one homogeneous process may conceal between-line variation.

Analysis

The overall rate is

Thus the expected overall defect rate is 1.45%. For an audit of n = 1,000 units, if the production mix remains stable and units are approximately independent, the expected defect count is 14.5 and its variance is 1,000(0.0145)(0.9855) ≈ 14.29.

However, if the audit samples entire shifts from individual lines, observations may be clustered. A line-level random effect can create overdispersion relative to a binomial model. The manager should therefore compare the observed variance with the binomial variance and assess serial or cluster dependence.

Approach and solution

1. Estimate line-specific rates rather than reporting only the pooled rate.
2. Preserve the production mix when drawing the audit sample, or use appropriate weighting.
3. Use the law of total variance to separate within-line and between-line variation.
4. Use a binomial or normal approximation only after checking independence and stability.
5. If clustering is substantial, use a hierarchical model, cluster-robust uncertainty, or a designed stratified sample.
6. Establish a decision rule before inspecting the data to reduce bias.

Outcome

The pooled estimate provides an operational baseline, while stratified estimates identify line 3 as the priority for corrective action. A large sample alone would not solve the problem if it overrepresented the lower-risk lines or ignored dependence.

Learning points

Conditional expectation supports risk aggregation across heterogeneous lines. The laws of large numbers justify stabilization only under suitable sampling conditions. The CLT may approximate pooled statistics, but governance requires checking whether its assumptions are credible.

10. Quality, Governance, and Technology

Modern quality systems use statistical software, simulation, automated inspection, and machine-learning models. These technologies can estimate conditional risks and approximate sampling distributions rapidly, but they do not remove the need for sound probability assumptions.

AI-based inspection systems may estimate P(defect | image, machine, temperature). Such conditional probabilities must be calibrated, validated across operating regimes, and monitored for drift. A model that performs well on average may fail for a particular supplier or product family. Conditional performance measures should therefore be reported by relevant strata.

Governance issues include traceability of data, protection of personal or supplier information, reproducibility of calculations, and avoidance of selective reporting. An approximation should be labelled as such, and uncertainty should not be hidden by excessive decimal precision. Quality decisions affecting safety or customers require documented validation and human accountability.

11. Limitations and Best Practices

Important limitations include:

• Non-independence caused by batches, shifts, machines, or serial correlation.
• Nonstationarity caused by process drift or improvement interventions.
• Heavy-tailed measurements for which finite variance may fail.
• Small samples and inaccurate normal approximations.
• Selection bias and changing inspection rules.
• Confusion between statistical significance and practical quality significance.

Best practices are to define the estimand, identify the information set, inspect the sampling design, test stability, quantify both within-group and between-group variation, choose convergence and approximation results appropriate to the problem, and validate critical calculations using exact methods or simulation. Reporting should state assumptions, sample size, uncertainty, and the operational consequence of error.

Chapter Summary

Conditional expectation is an information-based prediction and the minimum mean-square predictor among functions of the available information. The law of total expectation aggregates subgroup predictions, while the law of total variance separates expected within-group variation from variation among conditional means.

Convergence has several distinct meanings. Almost sure convergence is pathwise, convergence in probability concerns the chance of a substantive error, convergence in distribution concerns limiting distributions, and mean-square convergence concerns expected squared error. Stronger modes imply weaker modes under standard conditions, but the reverse implications generally fail.

The weak and strong laws of large numbers explain why sample averages stabilize under appropriate sampling assumptions. The CLT explains why standardized averages are often approximately normal, enabling confidence intervals, acceptance decisions, and risk calculations. Normal, Poisson, continuity-corrected, and delta-method approximations are useful but must be checked against sample size, dependence, skewness, and decision risk.

For TQM, these ideas support stratified quality analysis, process monitoring, supplier evaluation, audit design, reliability assessment, and responsible use of statistical technology. Large datasets do not guarantee valid conclusions: information quality, sampling design, model assumptions, and governance remain decisive.

Key Terms

Conditional expectation: The expected value of a random variable given specified information.

Conditional variance: Variation remaining in a random variable after conditioning on information.

Law of total expectation: Identity E[X] = E[E[X | Y]].

Law of total variance: Identity Var(X) = E[Var(X | Y)] + Var(E[X | Y]).

Almost sure convergence: Convergence except on an event of probability zero.

Convergence in probability: The probability of an error exceeding any fixed tolerance tends to zero.

Convergence in distribution: Convergence of distribution functions at continuity points.

Mean-square convergence: Convergence of expected squared error to zero.

Weak law of large numbers: Convergence of a sample mean to its expectation in probability.

Strong law of large numbers: Almost sure convergence of a sample mean to its expectation.

Central limit theorem: Approximate normality of standardized sums or means under suitable conditions.

Standard error: The standard deviation of an estimator or, commonly, of a sample mean.

Slutsky’s theorem: A result permitting combination of distributional and probability limits.

Continuous mapping principle: Preservation of convergence under suitable continuous transformations.

Continuity correction: Adjustment used when approximating a discrete distribution by a continuous one.

Poisson approximation: Approximation of a rare-event binomial count by a Poisson variable.

Delta method: Approximation of the distribution of a smooth function of an estimator.

Stratification: Sampling or analysis by meaningful subgroups.

Overdispersion: Variability greater than that predicted by a reference model, often due to dependence or heterogeneity.

Review Questions

5 Short Answer Questions

1. What does E[X | Y] represent in a quality prediction problem?
2. State the law of total variance.
3. Distinguish convergence in probability from convergence in distribution.
4. What assumptions are required by the classical i.i.d. central limit theorem?
5. Why is continuity correction used in a binomial-to-normal approximation?

5 Long Answer Questions

1. Explain conditional expectation, its projection interpretation, and its role in supplier-risk evaluation.
2. Derive and interpret the law of total variance in the context of multiple production lines.
3. Compare almost sure, in-probability, in-distribution, and mean-square convergence, including their logical relationships.
4. Explain how the weak and strong laws of large numbers differ and discuss their relevance to quality auditing.
5. Critically evaluate the assumptions and limitations of the central limit theorem in process monitoring.

3 Analytical / Application Questions

1. A plant uses two machines with production shares 0.6 and 0.4 and defect probabilities 0.01 and 0.04. Calculate the overall defect probability and decompose the variance of the defect indicator into within-machine and between-machine components.
2. An audit samples 500 independent units from a process with estimated defect rate 0.03. Explain whether a normal approximation is appropriate, calculate the approximate standard error of the sample proportion, and identify what additional checks are needed before making a release decision.
3. A quality analyst observes that sample averages stabilize as the sample size increases, but measurements are strongly correlated within batches. Explain which limit-theorem conclusions may be invalid and propose a sampling or modelling strategy that addresses the dependence.

Multiple Choice Questions with Answers

Question 1

Which expression states the law of total expectation?

A. E[X | Y] = E[X]

B. E[X] = E[E[X | Y]]

C. E[X²] = E[X]²

D. E[X + Y] = E[X]E[Y]

Correct Answer: B. E[X] = E[E[X | Y]]

Question 2

The law of total variance decomposes Var(X) into which two components?

A. Bias and standard error

B. Mean and median variation

C. Expected conditional variance and variance of conditional means

D. Within-sample range and interquartile range

Correct Answer: C. Expected conditional variance and variance of conditional means

Question 3

If Xₙ converges in probability to X, what happens to P(|Xₙ − X| > ε) for every ε > 0?

A. It converges to 1

B. It converges to 0

C. It remains equal to ε

D. It becomes undefined

Correct Answer: B. It converges to 0

Question 4

Under the classical i.i.d. central limit theorem, the limiting distribution of √n(X̄ₙ − μ)/σ is:

A. Uniform(0,1)

B. N(μ, σ²)

C. N(0,1)

D. Poisson(σ)

Correct Answer: C. N(0,1)

Question 5

For a Bernoulli sample with p = 0.02 and n = 400, what is the variance of the defect count K?

A. 0.0196

B. 7.84

C. 8.00

D. 400.02

Correct Answer: B. 7.84

Question 6

Which condition most directly threatens a binomial model for audit counts?

A. Units are independently sampled

B. Each unit has the same defect probability

C. Defect outcomes are strongly correlated within batches

D. The sample size is recorded

Correct Answer: C. Defect outcomes are strongly correlated within batches

Question 7

If Xₙ converges almost surely to X, which conclusion normally follows?

A. Xₙ converges in probability to X

B. Xₙ has exactly the same distribution as X for every n

C. Xₙ is independent of X

D. Xₙ has zero variance for every n

Correct Answer: A. Xₙ converges in probability to X

Question 8

A Poisson approximation to a binomial count is most appropriate when:

A. p is near 0.5 and n is small

B. p is small, n is large, and np is moderate

C. n and p are both exactly 1

D. events are strongly dependent

Correct Answer: B. p is small, n is large, and np is moderate

Question 9

For K ~ Binomial(n,p), a continuity-corrected approximation to P(K ≤ k) uses:

A. P(Y ≤ k − 0.5)

B. P(Y ≤ k)

C. P(Y ≤ k + 0.5)

D. P(Y ≥ k + 1)

Correct Answer: C. P(Y ≤ k + 0.5)

Question 10

What does the between-group component Var(E[X | Y]) measure?

A. Variation of observations within each group

B. Variation among conditional group means

C. The sampling-unit measurement scale

D. The probability of a rare event

Correct Answer: B. Variation among conditional group means