Sampling Distributions and Properties of Estimators

Introduction and Context

Statistical quality management rarely observes every unit produced. Instead, managers inspect a sample and use its results to infer characteristics of a process or population. The reliability of such inference depends not only on the observed data but also on the probability distribution of the statistic calculated from the sample. This distribution is called a sampling distribution.

Sampling distributions provide the theoretical foundation for confidence intervals, hypothesis tests, control-chart limits, acceptance sampling and process-capability studies. They also allow a quality professional to evaluate competing estimators of an unknown population parameter. An estimator should be accurate on average, become reliable as sample size increases, use the available information effectively and remain reasonably dependable when assumptions are imperfect.

Learning Objectives

By the end of this chapter, the reader should be able to:

• distinguish a population, sample, parameter, statistic and estimator;
• derive sampling distributions for common statistics;
• calculate and interpret standard errors;
• explain bias, consistency, efficiency, sufficiency, robustness and mean squared error;
• use the central limit theorem and finite-population correction appropriately;
• compare estimators in quality-management applications;
• identify the assumptions and limitations behind sampling-based decisions.

1. Populations, Samples and Statistics

A population is the complete set of units about which a conclusion is required. It may be finite, such as all items produced during a shift, or conceptual, such as all future outputs of a stable process. A parameter is a fixed but usually unknown numerical characteristic of the population, such as the mean μ, variance σ² or defect proportion p.

A sample is a subset selected according to a sampling design. A statistic is a function of sample observations and therefore varies from sample to sample. If X₁, X₂, …, Xₙ are observations, the sample mean is:

Here, n is sample size and X̄ is a statistic. When X̄ is used to estimate μ, it is an estimator; its observed numerical value is an estimate. This distinction is essential: an estimator is a random variable before sampling, whereas an estimate is the result obtained after sampling.

The sampling mechanism must support the intended inference. Simple random sampling gives each unit an equal selection probability. Stratified sampling divides the population into relatively homogeneous groups and samples within each group. Cluster sampling selects groups of units. These designs produce different sampling distributions and variances, even when their nominal sample sizes are equal.

2. What Is a Sampling Distribution?

A sampling distribution is the probability distribution of a statistic over all possible samples selected under a specified sampling design. It describes three fundamental features:

1. Centre: the expected value of the statistic;
2. Spread: the variance or standard error of the statistic;
3. Shape: whether the distribution is normal, t-shaped, skewed, discrete or otherwise structured.

For independent and identically distributed observations with population mean μ and variance σ²,

and

The standard error, SE, measures the typical sampling fluctuation of X̄ around μ. It is not the standard deviation of individual observations; it is the standard deviation of the estimator.

If σ is unknown, it is commonly replaced by the sample standard deviation S:

The estimated standard error is then S/√n. The divisor n − 1 provides an unbiased estimator of σ² under the usual independent random-sampling assumptions.

Finite-Population Correction

When sampling without replacement from a finite population of size N, and the sampling fraction n/N is not negligible, the variance of the sample mean is:

Thus,

The factor involving N is the finite-population correction. It approaches 1 when N is very large relative to n and becomes 0 when the entire population is observed. It should not be applied automatically to independent sampling from an effectively infinite process.

3. Important Sampling Distributions

3.1 Normal Distribution of the Sample Mean

If X₁, …, Xₙ are independent normal observations with mean μ and variance σ², then the sample mean is exactly normal:

Consequently,

This result is exact when the population is normal and σ is known.

3.2 Central Limit Theorem

The central limit theorem states that, under broad conditions, the standardized sample mean approaches a standard normal distribution as n increases:

The observations should generally be independent, identically distributed and have finite variance. The theorem does not say that individual observations become normal. It concerns the distribution of the mean. For strongly skewed or heavy-tailed populations, a larger sample may be necessary, and dependence can invalidate the usual result.

3.3 Student’s t Distribution

When the population is normal and σ is unknown, replacing σ by S introduces additional uncertainty. The statistic

has a Student’s t distribution with n − 1 degrees of freedom. The t distribution is symmetric and centred at zero but has heavier tails than the standard normal distribution. As n increases, it approaches the normal distribution.

For a two-sided confidence interval for μ under normal sampling:

where t₍α/2,n−1₎ is the critical t value leaving probability α/2 in each tail.

3.4 Chi-Square Distribution for Variance

For a normal population,

This result supports inference about process variance. It depends strongly on normality; the sample variance does not generally have this chi-square distribution for non-normal observations.

3.5 F Distribution

If U and V are independent chi-square variables divided by their respective degrees of freedom, their ratio has an F distribution. In particular, for two independent normal samples,

has an F distribution with n₁ − 1 and n₂ − 1 degrees of freedom. This distribution supports comparison of two variances, although modern quality analysis should also consider robust methods when normality is doubtful.

3.6 Sample Proportions

If X is the number of nonconforming units in n independent Bernoulli trials, then X has a binomial distribution with parameter p. The sample proportion is p̂ = X/n, and

For sufficiently large n, p̂ is approximately normal when both np and n(1 − p) are adequately large. The approximate standard error is √[p(1 − p)/n], or √[p̂(1 − p̂)/n] when p is unknown. For small counts or proportions close to 0 or 1, exact or improved interval methods are preferable to a basic normal approximation.

4. Properties of Estimators

4.1 Bias

The bias of an estimator θ̂ for parameter θ is:

An estimator is unbiased if its expected value equals the parameter. The sample mean is unbiased for μ. The divisor n − 1 in S² makes S² unbiased for σ², whereas the version using divisor n is biased downward for finite n.

Bias is a property of the estimator and sampling procedure, not of one particular estimate. An estimate can be above or below θ even when the estimator is unbiased.

4.2 Mean Squared Error

Mean squared error combines systematic error and random variation:

The decomposition follows by expanding the square and using E(θ̂ − θ) = Bias(θ̂). MSE is measured in squared units of θ. A slightly biased estimator may be preferable if its variance is sufficiently smaller. Therefore, unbiasedness alone is not a complete decision rule.

4.3 Consistency

An estimator θ̂ₙ is consistent for θ if it converges in probability to θ as sample size increases:

for every ε > 0. Unbiasedness is not necessary for consistency. A useful sufficient condition is MSE(θ̂ₙ) → 0. Consistency concerns large samples and does not guarantee good performance at the sample sizes used in practice.

4.4 Efficiency

For unbiased estimators, an estimator with smaller variance is more efficient. Relative efficiency may be expressed as:

when θ̂₁ and θ̂₂ estimate the same parameter and the comparison is defined accordingly. Efficiency comparisons must specify the model and class of estimators. An estimator can be efficient under a normal model but poor under contamination or heavy tails.

The Cramér–Rao lower bound gives a theoretical lower bound for the variance of unbiased estimators under regularity conditions:

where I(θ) is the Fisher information in the sample. Equality is not always attainable, and the regularity conditions must be checked rather than assumed.

4.5 Sufficiency

A statistic T(X) is sufficient for θ if, conditional on T, the sample contains no additional information about θ. The factorization theorem provides a practical criterion: a statistic is sufficient if the likelihood can be written as

where h does not depend on θ. For a normal population with known variance, the sample mean is sufficient for μ. For a Bernoulli model, the total number of successes is sufficient for p. Sufficiency is model-dependent; it does not mean that a statistic is always best for every population.

4.6 Robustness

An estimator is robust when moderate departures from assumptions or a small number of unusual observations do not cause unacceptable changes in its results. The mean has high sensitivity to extreme values. The median has a higher breakdown resistance and is often preferable for skewed cycle times, delivery delays or contaminated measurements, though it may be less efficient under an exactly normal model.

Robustness has several dimensions, including resistance to outliers, stability under non-normal distributions and sensitivity to dependence or heteroscedasticity. It should be evaluated in relation to the decision consequences, not treated as an absolute label.

5. Worked Example: Estimating a Filling-Process Mean

A quality engineer takes a random sample of n = 25 bottles. The process standard deviation is known to be σ = 4 ml, and the target population mean is μ = 500 ml.

The sampling standard error of the sample mean is:

If the process is normal, X̄ has a normal sampling distribution with mean 500 ml and standard deviation 0.8 ml. A sample mean of 501.6 ml is therefore 2 standard errors above the population mean:

The important interpretation is not that every bottle is within 0.8 ml of 500 ml. Rather, repeated samples of 25 bottles would produce sample means whose standard deviation is 0.8 ml. Confusing process variation with sampling variation can lead to inappropriate corrective action.

If σ were unknown, the engineer would use S and a t distribution with 24 degrees of freedom. If the bottles were selected from a very small finite lot without replacement, the finite-population correction would also be considered.

6. Estimator Choice in Quality Management

Estimator selection should follow the measurement scale, sampling design and loss associated with errors. Common choices include:

For a quality characteristic with a highly skewed distribution, reporting only the mean and normal-theory standard error can hide operational risk. A histogram, quantile summary, run chart and investigation of measurement-system quality should accompany estimation.

Measurement error also affects estimators. If observed value Y equals true value X plus independent error E with mean zero and variance σₑ², then the observed variance is inflated:

This identity assumes independence and zero covariance between X and E. Gauge repeatability and reproducibility studies are therefore important before interpreting process variation.

7. Illustrative Case: Supplier Defect Estimation

Background. A manufacturer receives a lot of 10,000 components from a supplier. Inspecting every component is costly. A simple random sample of 200 components identifies 8 nonconforming units.

Problem. Management wants to estimate the supplier’s defect rate and decide whether escalation is warranted.

Analysis. The observed defect proportion is:

The approximate standard error, using p̂, is:

A simple approximate 95% interval is p̂ ± 1.96SE, giving approximately 0.04 ± 0.0272, or 1.28% to 6.72%. Because the number of observed defects is only eight, the normal approximation may be inaccurate; an exact binomial or Wilson interval is more defensible. Since the lot is finite, a hypergeometric analysis can also be relevant, particularly when the sampling fraction is substantial.

Approach and solution. The quality team should verify that the sample was genuinely random, confirm the inspection system’s sensitivity and specificity, classify defects by severity and compare the interval with the contractual acceptance criterion. The sample should not be expanded selectively after observing an unfavourable result without documenting the revised sampling rule.

Outcome. The point estimate alone does not establish that the supplier’s true defect rate is exactly 4%. It indicates sampling uncertainty. A decision based on risk categories, confidence limits and the cost of false acceptance versus false rejection is more appropriate.

Learning points. Sampling distributions translate a sample result into uncertainty; finite-lot structure can matter; inspection error can bias estimates; and statistical evidence must be integrated with technical and contractual consequences.

8. Limitations, Governance and Contemporary Practice

Sampling theory assumes a defined target population and a defensible selection mechanism. Convenience samples, voluntary responses and undocumented exclusions can produce selection bias that no standard-error formula repairs. Nonresponse, time trends, autocorrelation and changing process conditions also undermine simple independent-sample models.

Quality professionals should document the population definition, sampling frame, randomization method, sample-size rationale, missing observations, measurement system and analysis model. They should distinguish exploratory analysis from confirmatory claims and avoid presenting a confidence level as the probability that a fixed parameter lies in a particular interval under the classical interpretation.

Software and AI can automate simulation, bootstrap standard errors, anomaly detection and distributional diagnostics. However, software cannot determine whether the sample represents the process, whether observations are independent or whether an outlier is a recording error or a real failure mode. AI-generated statistical recommendations require validation, reproducibility, access controls and human accountability. Confidential supplier and production data should not be transferred to unapproved systems.

Bootstrap methods are especially useful when the sampling distribution is analytically difficult, but they reproduce features of the observed sample and may perform poorly for very small, dependent or unrepresentative samples. Resampling must respect the design—for example, block bootstrap for serial dependence or cluster resampling for grouped data.

Best-Practice Checklist

• Define the population and estimand before collecting data.
• Use a probability-based sampling design whenever population inference is intended.
• Separate process variation from sampling variation and measurement error.
• Report the estimator, standard error, assumptions and uncertainty interval.
• Check skewness, outliers, dependence and finite-population effects.
• Compare estimators using MSE and robustness, not unbiasedness alone.
• Use exact or design-based methods when normal approximations are doubtful.
• Preserve an auditable record of sampling, analysis and decision rules.

Chapter Summary

A sampling distribution is the probability distribution of a statistic across repeated samples. The sample mean is unbiased for the population mean under random sampling, has variance σ²/n for independent observations and becomes approximately normal under the central limit theorem. The t, chi-square, F, binomial and approximate normal distributions support inference about means, variances, proportions and variance ratios under stated assumptions.

Estimator quality involves several distinct properties. Bias measures systematic departure from the target; MSE combines variance and squared bias; consistency describes convergence with increasing sample size; efficiency compares precision within a specified model; sufficiency concerns information preservation; and robustness concerns stability under departures from assumptions. In quality management, these properties must be evaluated alongside sampling design, measurement-system error, operational risk and the consequences of incorrect decisions.

Key Terms

Population: Complete set of units relevant to an investigation.

Parameter: Fixed numerical characteristic of a population.

Sample: Subset selected from a population.

Statistic: Numerical function of sample observations.

Estimator: Statistic used to estimate a population parameter.

Estimate: Observed value of an estimator.

Sampling distribution: Distribution of a statistic over repeated samples.

Standard error: Standard deviation of an estimator’s sampling distribution.

Bias: Difference between an estimator’s expected value and the target parameter.

Mean squared error: Variance plus squared bias.

Consistency: Convergence of an estimator to its target as sample size increases.

Efficiency: Relative precision of an estimator under a specified model.

Sufficiency: Property of retaining all sample information about a parameter under a model.

Robustness: Resistance to outliers or assumption violations.

Central limit theorem: Result describing approximate normality of standardized means under broad conditions.

Finite-population correction: Variance reduction caused by sampling without replacement from a finite population.

Measurement error: Difference between an observed value and the underlying true value.

Bootstrap: Resampling method used to approximate an estimator’s sampling distribution.

Review Questions

5 Short Answer Questions

1. Distinguish an estimator from an estimate.
2. What does the standard error of a sample mean measure?
3. State the bias and MSE formulas for an estimator.
4. When is the t distribution used instead of the standard normal distribution for a mean?
5. What is the purpose of the finite-population correction?

5 Long Answer Questions

1. Explain the central limit theorem and discuss its assumptions and limitations in quality-control data.
2. Compare unbiasedness, consistency, efficiency and robustness as criteria for estimator selection.
3. Derive the variance of the sample mean for independent observations and interpret its dependence on sample size.
4. Explain the roles of the t, chi-square and F distributions in statistical inference.
5. Discuss how measurement-system error can distort estimates of process variation.

3 Analytical / Application Questions

1. A process has known σ = 6 units. Compare the standard errors for samples of 36 and 144 observations, and explain the managerial meaning of the result.
2. A defect sample contains 3 nonconforming units out of 100. Explain why a normal approximation may be questionable and recommend a more suitable analysis.
3. A median and a mean are available for a strongly skewed delivery-time process containing two extreme delays. Which estimator would you report first, and what additional evidence would you examine before making a process decision?

Multiple Choice Questions with Answers

Question 1

For independent observations with population variance σ², what is Var(X̄)?

A. σ²n

B. σ²/n

C. σ/n²

D. n/σ²

Correct Answer: B. σ²/n

Question 2

An estimator’s bias is defined as:

A. E(θ̂²) − θ²

B. Var(θ̂) − θ

C. E(θ̂) − θ

D. θ − Var(θ̂)

Correct Answer: C. E(θ̂) − θ

Question 3

Which property describes convergence of an estimator to the target as sample size increases?

A. Sufficiency

B. Consistency

C. Robustness

D. Unbiasedness

Correct Answer: B. Consistency

Question 4

If σ is unknown and the population is normal, the standardized sample mean follows which distribution?

A. Chi-square with n degrees of freedom

B. F with n degrees of freedom

C. Student’s t with n − 1 degrees of freedom

D. Uniform with n − 1 degrees of freedom

Correct Answer: C. Student’s t with n − 1 degrees of freedom

Question 5

A sample of 64 observations comes from a process with σ = 8. What is the standard error of X̄?

A. 0.125

B. 0.5

C. 1

D. 8

Correct Answer: C. 1

Question 6

The MSE of an estimator equals:

A. Bias + variance

B. Variance − squared bias

C. Variance + squared bias

D. Standard error + bias

Correct Answer: C. Variance + squared bias

Question 7

Which estimator is generally more resistant to extreme observations?

A. Mean

B. Median

C. Sample variance

D. Range

Correct Answer: B. Median

Question 8

The finite-population correction is relevant primarily when sampling:

A. With replacement from an infinite population

B. Without replacement from a finite population

C. Only from a normal population

D. Only when σ is unknown

Correct Answer: B. Without replacement from a finite population

Question 9

For a Bernoulli sample proportion p̂, the model-based variance is:

A. p/n

B. p(1 − p)/n

C. p²/n

D. n p(1 − p)

Correct Answer: B. p(1 − p)/n

Question 10

A sufficient statistic is one that:

A. Always has zero variance

B. Contains all sample information about a parameter under the model

C. Is necessarily unbiased

D. Is unaffected by outliers

Correct Answer: B. Contains all sample information about a parameter under the model