Expectations, Transforms and Probability Inequalities

Introduction

Uncertainty analysis is central to Total Quality Management (TQM). Quality engineers and managers rarely know the exact number of defects, the precise lifetime of a component, or the exact future demand. Probability models provide a disciplined way to describe such uncertainty. Expectations summarize long-run performance, variance measures dispersion, transforms encode distributions and facilitate calculation, and probability inequalities provide guaranteed bounds when complete distributional information is unavailable.

These tools support process capability analysis, reliability assessment, inspection planning, supplier evaluation, inventory decisions and risk-based quality improvement. They are especially valuable when data are limited or when assumptions about a probability distribution must be challenged.

Learning Objectives

By the end of this chapter, the reader should be able to:

1. Define and interpret expectation, variance, covariance and conditional expectation.
2. Use laws of total expectation and total variance.
3. Apply probability-generating, moment-generating and characteristic functions.
4. Explain and apply Markov, Chebyshev, Jensen and Chernoff inequalities.
5. Distinguish exact distributional calculations from distribution-free bounds.
6. Select suitable tools for uncertainty analysis in quality-management applications.
7. Evaluate assumptions, limitations and governance implications of probabilistic results.

1. Expectation as a Measure of Long-Run Performance

Let X be a random variable defined on a probability space. If X is discrete with possible values xᵢ and probabilities pᵢ, its expectation is

provided the series is absolutely convergent when necessary. Here E[X] is the probability-weighted average of X. For a continuous random variable with density fₓ(x),

provided the integral exists. The unit of E[X] is the same as the unit of X: if X is measured in millimetres, E[X] is measured in millimetres; if X is a cost, E[X] is a monetary amount.

More generally, for a measurable function g,

for a discrete variable, or

for a continuous variable. This is the law of the unconscious statistician: it permits calculation of a function's expectation without first deriving the distribution of g(X).

Linearity of expectation

For constants a and b and random variables X and Y for which the expectations exist,

No independence assumption is required. This fact is particularly useful for total cost, total production and aggregate defect counts.

For a sum of n variables,

Even when production stages are dependent, the expected total remains the sum of expected stage values.

Conditional expectation

If information represented by an event A is available, the conditional expectation is

in the discrete case. Conditional expectation given a random variable Y is written E[X|Y]. It is itself a random variable and represents the best mean-square prediction of X based on Y.

The law of total expectation states that, for mutually exclusive and exhaustive categories A₁,…,Aₖ,

This is useful when a process contains identifiable strata, such as shifts, suppliers or machine types.

2. Variance, Covariance and Dependence

The variance of X is

where μ = E[X]. An equivalent computational form is

Variance has squared units, so the standard deviation is often easier to interpret:

Variance is finite only when E[X²] is finite. A variable can have an expectation but no finite variance.

For constants a and b,

For two variables, covariance is

and the variance of a linear combination is

If X and Y are independent and have finite second moments, Cov(X,Y) = 0, giving the familiar addition rule. Zero covariance, however, does not generally imply independence.

The correlation coefficient is

when both standard deviations are positive. It lies between −1 and 1. Correlation describes linear association, not every form of dependence.

Law of total variance

For a conditioning variable Y,

The first term is average within-group variation; the second is between-group variation. In quality management, this decomposition separates random variation inside machines or shifts from systematic variation between them.

Worked example: inspection cost

Suppose an inspection lot contains 100 items. Each item has inspection cost £2.50, and an additional rework cost of £40 is incurred if at least one defect is found. Let D be the number of defective items, and suppose P(D ≥ 1) = 0.08. The total cost is C = 250 + 40I, where I is 1 when at least one defect is found and 0 otherwise. Therefore,

The expected cost is £253.20. Since Var(I) = 0.08(0.92),

and the standard deviation is approximately £10.85. The calculation does not require independence among item defects; it requires only the stated probability of at least one defect.

3. Generating and Transform Functions

Transforms convert a probability distribution into a function whose algebraic properties can simplify sums, moments and distributional analysis. They are not merely computational devices: they also provide compact distributional signatures.

Probability-generating function

For a non-negative integer-valued random variable X, the probability-generating function (PGF) is

for values of s where the series converges, normally |s| ≤ 1. Important properties include

and, when the derivative exists,

Also,

so that E[X²] = G″ₓ(1) + G′ₓ(1). If X and Y are independent non-negative integer-valued variables,

For a Poisson variable with mean λ > 0,

Moment-generating function

The moment-generating function (MGF) is

for those real t in an open interval around zero where the expectation is finite. If differentiable at zero,

In particular, M′ₓ(0) = E[X] and M″ₓ(0) = E[X²]. For independent X and Y,

The MGF uniquely determines a distribution when it exists in a neighbourhood of zero. It may fail to exist for heavy-tailed distributions, so it should not be assumed universally available.

The cumulant-generating function is Kₓ(t) = log Mₓ(t). Its first derivative at zero is the mean and its second derivative at zero is the variance. Cumulants add for independent sums, making K useful in aggregate-risk analysis.

Characteristic function

The characteristic function (CF) is

where i² = −1 and t is real. Unlike the MGF, the CF always exists because |eⁱᵗˣ| = 1. It uniquely determines the distribution. For independent variables,

The CF is especially important in limit theory, distribution identification and numerical inversion. Its practical interpretation is less direct than that of a PGF or MGF, but its universal existence is a major advantage.

4. Fundamental Probability Inequalities

Inequalities provide bounds rather than exact probabilities. Their value is greatest when the distribution is only partially known.

Markov's inequality

If X is non-negative and a > 0, then

The bound follows because X is at least a whenever the event X ≥ a occurs. Markov requires non-negativity and a finite expectation. It can be conservative.

For a non-negative function g(X),

This generalized form is often useful in reliability and loss analysis.

Chebyshev's inequality

If X has finite mean μ and finite variance σ², then for k > 0,

Equivalently, for d > 0,

Chebyshev is distribution-free. It does not require normality, but it treats all distributions with the same mean and variance alike and can therefore be loose.

If X is known to be one-sided relative to its mean, Cantelli's inequality can improve the upper-tail bound:

Jensen's inequality

If φ is convex on an interval containing the values of X and E[|φ(X)|] exists, then

For concave φ, the inequality reverses. A function is convex when its graph lies below its chords; twice differentiability with φ″(x) ≥ 0 is a sufficient condition.

For example, since x² is convex,

which is equivalent to non-negative variance. Jensen explains why variability can increase expected nonlinear costs. If a failure penalty grows quadratically with deviation, reducing variation can lower expected penalty even when the mean remains unchanged.

Chernoff bounds

For any random variable X and t > 0 for which Mₓ(t) is finite,

This follows by applying Markov's inequality to eᵗˣ. Optimizing over t gives

For a sum of independent variables S = X₁ + ··· + Xₙ,

Chernoff bounds are often substantially tighter than Chebyshev bounds for sums, particularly when the variables have light tails. They require a suitable MGF and independence or another valid method for controlling the joint transform.

Worked example: a defect-count bound

Suppose the number D of defects in a batch is non-negative and E[D] = 4. Without knowing the distribution, Markov gives

If, in addition, Var(D) = 4, Chebyshev gives a two-sided statement around μ = 4:

Since D ≥ 10 implies |D − 4| ≥ 6, the one-sided event also has probability at most approximately 0.1111. This bound uses more information than Markov and is therefore smaller, although it may still be conservative.

5. Selecting a Bound in Quality Analysis

The analyst should distinguish four situations:

1. A validated distribution is available. Calculate the exact probability, while checking parameter uncertainty and model fit.
2. Only a mean is credible. Markov may be appropriate for non-negative variables.
3. Mean and variance are credible. Chebyshev or Cantelli can provide distribution-free guarantees.
4. An MGF and independence structure are credible. Chernoff or related exponential bounds may be effective.

A bound is not an estimate. If a bound says a risk is at most 0.20, the true risk may be 0.01 or 0.20; the result does not imply that the risk is close to 0.20. Conservative bounds are useful for screening, warranty reserves and safety decisions, but may lead to excessive inspection if interpreted as forecasts.

6. Illustrative Case: Supplier Defect Risk

Background

A manufacturer receives components from three suppliers. Historical records indicate different average defect rates, and production volume varies by supplier. Management must determine whether the probability of more than 12 defective components in a daily receipt is sufficiently small to permit reduced incoming inspection.

Problem

Let Xⱼ be the number of defects from supplier j on a day. The total is S = X₁ + X₂ + X₃. The organization has reliable estimates of E[Xⱼ] and Var(Xⱼ), but independence is uncertain because suppliers may be affected by the same raw-material disruption.

Analysis

Suppose the means are 2, 3 and 1, so E[S] = 6. Suppose the variances are 2, 3 and 1. If independence were defensible, Var(S) = 6. Without independence, the covariance terms must be included:

If management cannot validate the covariance terms, it should not report 6 as the variance of S. If a conservative variance estimate of 10 is adopted, Chebyshev gives

This result is a screening bound, not evidence that the actual probability is 27.8%. If supplier counts are well modelled as independent Poisson variables, then S is Poisson with mean 6, using the PGF product property. The exact probability can then be calculated from the Poisson distribution, and a Chernoff bound can be used as an analytically transparent check.

Approach and solution

The quality team should:

• stratify observations by supplier and production condition;
• test whether dependence changes during common disruptions;
• estimate uncertainty in means, variances and covariances;
• compare exact model-based probabilities with distribution-free bounds;
• retain increased inspection whenever the consequences of underestimating risk exceed the cost of inspection.

Outcome and learning points

The central decision is not simply which formula produces the smallest number. It is whether the assumptions behind that formula are justified. Transform methods are powerful when the count model is credible; inequalities are safer when distributional assumptions are weak. Governance requires that the method, assumptions, data period and uncertainty be documented so that a future decision-maker can reproduce and challenge the result.

7. Limitations, Governance and Contemporary Practice

Probability calculations can create false confidence when data are biased, definitions change or dependence is ignored. Common problems include under-reporting of defects, survivorship bias in reliability records, mixing unlike process strata and treating estimated parameters as known constants.

In AI-assisted quality analytics, transforms and inequalities can support explainable risk limits. For example, an automated monitoring system may trigger investigation when an upper confidence or concentration bound exceeds a tolerance. However, AI-generated recommendations must not be accepted without validating data provenance, distributional assumptions, subgroup performance and the cost of false alarms. A mathematically correct bound based on contaminated data remains operationally misleading.

Best practice includes:

• define the random variable and measurement unit before calculating;
• state whether the result is an exact probability, estimate or bound;
• verify moment existence and independence assumptions;
• use sensitivity analysis for uncertain means, variances and covariances;
• compare multiple methods where decisions are consequential;
• communicate conservative bounds honestly and avoid presenting them as forecasts;
• preserve an audit trail of data, code, parameters and approval decisions.

Chapter Summary

Expectation is the probability-weighted long-run value and is linear even without independence. Variance measures dispersion, while covariance records dependence and determines how variability combines. Conditional expectation and the laws of total expectation and total variance support stratified process analysis.

PGFs are natural for non-negative integer counts, MGFs encode moments and simplify independent sums when they exist, and CFs always exist and uniquely identify distributions. Markov's inequality uses non-negativity and a mean; Chebyshev's inequality uses mean and variance; Jensen's inequality describes the effect of convex or concave transformations; and Chernoff bounds use exponential moments to control tails. Each result has assumptions and may be conservative.

In TQM, these tools help evaluate defect counts, inspection costs, reliability risks and supplier variation. Sound practice requires matching the method to the available evidence, explicitly addressing dependence and clearly distinguishing model-based probabilities from distribution-free guarantees.

Key Terms

Expectation: Probability-weighted long-run average of a random variable.

Variance: Expected squared deviation from the mean.

Standard deviation: Square root of variance, expressed in the units of the variable.

Covariance: Measure of joint linear variation between two variables.

Correlation: Standardized covariance lying between −1 and 1.

Conditional expectation: Expected value given specified information.

Law of total expectation: Decomposition of an expectation across exhaustive groups.

Law of total variance: Decomposition into within-group and between-group variation.

Probability-generating function: Transform for non-negative integer-valued variables.

Moment-generating function: Transform whose derivatives can produce moments.

Characteristic function: Fourier-transform representation that always exists.

Cumulant-generating function: Logarithm of the MGF when the MGF exists.

Markov's inequality: Upper-tail bound for non-negative variables using the mean.

Chebyshev's inequality: Distribution-free deviation bound using variance.

Cantelli's inequality: One-sided variance-based bound.

Jensen's inequality: Relation between a nonlinear function of a mean and the mean of the function.

Chernoff bound: Exponential tail bound obtained by optimizing an MGF inequality.

Distribution-free bound: Guarantee requiring limited distributional assumptions.

Independence: Condition under which joint probabilities factor into marginal probabilities.

Concentration bound: Upper bound on the probability of a deviation or tail event.

Review Questions

5 Short Answer Questions

1. Define expectation for a discrete random variable.
2. Why is covariance relevant when combining process variances?
3. State the law of total variance.
4. What type of variable is most naturally analysed using a probability-generating function?
5. State the conditions required for Markov's inequality.

5 Long Answer Questions

1. Explain expectation, variance and standard deviation, including their units and interpretation in quality management.
2. Compare PGFs, MGFs and characteristic functions in terms of domain, purpose and limitations.
3. Derive the variance of aX + bY and explain the role of covariance.
4. Explain Jensen's inequality and show how process variability can affect expected nonlinear quality costs.
5. Compare Markov, Chebyshev, Cantelli and Chernoff inequalities, including the assumptions under which each is appropriate.

3 Analytical / Application Questions

1. A non-negative daily loss has mean £800. Use Markov's inequality to bound the probability that the loss is at least £4,000. Discuss why the bound may be conservative.
2. A process output has mean 50 units and variance 9 units². Use Chebyshev's inequality to bound the probability that output differs from its mean by at least 6 units. Interpret the result.
3. Three suppliers contribute defect counts to one batch. Explain how you would decide whether to use an exact Poisson-sum model, a Chebyshev bound or a Chernoff bound. Identify the data and assumptions you would verify first.

Multiple Choice Questions with Answers

Question 1

Which statement about linearity of expectation is correct?

A. It requires all variables to be independent.

B. It applies to sums when the relevant expectations exist, without requiring independence.

C. It applies only to normally distributed variables.

D. It applies only when all variances are equal.

Correct Answer: B. It applies to sums when the relevant expectations exist, without requiring independence.

Question 2

If E[X] = 5 and Var(X) = 4, what is E[X²]?

A. 9

B. 20

C. 25

D. 29

Correct Answer: D. 29

Question 3

Which transform is defined for every real-valued random variable because its modulus is always one?

A. Probability-generating function

B. Moment-generating function

C. Characteristic function

D. Cumulant-generating function

Correct Answer: C. Characteristic function

Question 4

A non-negative random variable has mean 6. Markov's inequality gives which upper bound for P(X ≥ 15)?

A. 0.04

B. 0.20

C. 0.40

D. 0.90

Correct Answer: C. 0.40

Question 5

If Var(X) = 9, Chebyshev's inequality bounds P(|X − E[X]| ≥ 6) by at most:

A. 0.0833

B. 0.25

C. 0.50

D. 0.6667

Correct Answer: B. 0.25

Question 6

For a convex function φ, Jensen's inequality states that:

A. E[φ(X)] ≤ φ(E[X])

B. φ(E[X]) ≤ E[φ(X)]

C. E[φ(X)] = 0

D. φ must be linear

Correct Answer: B. φ(E[X]) ≤ E[φ(X)]

Question 7

If X and Y are independent, which relation holds for their moment-generating functions?

A. Mₓ₊ᵧ(t) = Mₓ(t) + Mᵧ(t)

B. Mₓ₊ᵧ(t) = Mₓ(t) − Mᵧ(t)

C. Mₓ₊ᵧ(t) = Mₓ(t)Mᵧ(t)

D. Mₓ₊ᵧ(t) = Mₓ(t)/Mᵧ(t)

Correct Answer: C. Mₓ₊ᵧ(t) = Mₓ(t)Mᵧ(t)

Question 8

Which additional information is required to calculate Var(X + Y) from Var(X) and Var(Y)?

A. The median of X

B. The covariance of X and Y

C. The mode of Y

D. The PGF of X only

Correct Answer: B. The covariance of X and Y

Question 9

Chernoff bounds are obtained by applying Markov's inequality to which transformation?

A. X²

B. |X|

C. eᵗˣ

D. log X

Correct Answer: C. eᵗˣ

Question 10

A probability bound of 0.30 should be interpreted as:

A. The true probability is exactly 0.30.

B. The event will occur in 30% of all future cases.

C. The probability is guaranteed not to exceed 0.30 under the stated assumptions.

D. The event is impossible.

Correct Answer: C. The probability is guaranteed not to exceed 0.30 under the stated assumptions.