Exact Sampling Distributions for Normal Populations

Introduction and Context

In quality management, decisions are often based on samples rather than complete populations. A process engineer may estimate the mean tensile strength from 10 specimens, compare the variability of two production lines, or determine whether a process variance exceeds a specification. Such decisions require probability models for statistics calculated from samples.

When the parent population is normal, several sampling distributions are available in exact form. These distributions provide the theoretical foundation for confidence intervals, hypothesis tests, analysis of variance, regression, design of experiments, and statistical process control. The central distributions are:

• the normal distribution for a standardized sample mean when the population variance is known;
• Student’s t distribution for a standardized sample mean when the population variance is unknown;
• the chi-square distribution for inference about one population variance; and
• the F distribution for comparing two population variances.

The results in this chapter are exact under the stated normality and independence assumptions. They should not be confused with large-sample approximations.

Learning Objectives

After studying this chapter, you should be able to:

1. derive the sampling distribution of the sample mean from a normal population;
2. explain the distribution of the sample variance and its degrees of freedom;
3. derive the Student’s t statistic and identify when it is appropriate;
4. use chi-square and F distributions for inference about variances;
5. construct and interpret confidence intervals and hypothesis tests;
6. diagnose the consequences of violating normality, independence, and equal-variance assumptions; and
7. apply these distributions to quality-management decisions.

1. Foundations and Notation

Let X₁, X₂, ..., X_n be a random sample of size n from a normal population with mean μ and variance σ²:

The observations are assumed to be independent and identically distributed. The principal statistics are:

Here, X̄ is the sample mean, S² is the unbiased sample variance, and S is the sample standard deviation. The denominator n − 1 is used because one degree of freedom is lost when deviations are calculated from the estimated mean rather than the known population mean.

The sample mean has the same measurement units as X_i. Variance has squared units, while standard deviation returns to the original units. Dimensional consistency is therefore essential when interpreting formulas.

2. Sampling Distribution of the Sample Mean

2.1 Derivation

Because a linear combination of independent normal random variables is normal, the sample mean is normally distributed. Its expectation is

Its variance is

where independence makes the covariance terms zero. Therefore,

and the standard error of the mean is

The standard error measures the typical sampling fluctuation of sample means around the population mean. Increasing the sample size reduces it at the rate 1/√n, not at the rate 1/n. To halve the standard error, the sample size must be multiplied by four.

2.2 Standardization and the normal distribution

If σ is known, the standardized statistic is

This result supports exact tests and confidence intervals for μ. For a two-sided confidence level 1 − α, the confidence interval is

where z_α/2 is the standard-normal critical value satisfying P(Z > z_α/2) = α/2. For example, z_0.025 ≈ 1.96 for 95% confidence.

A hypothesis test of H₀: μ = μ₀ uses

The p-value is calculated from the standard-normal distribution according to whether the alternative is lower-tailed, upper-tailed, or two-tailed.

Worked Example 1: Known process standard deviation

A filling process has known standard deviation σ = 4 grams. A random sample of n = 64 containers has mean X̄ = 502 grams. Find a 95% confidence interval for the population mean.

The standard error is

The margin of error is 1.96 × 0.5 = 0.98 grams. Therefore,

The interval procedure has 95% long-run coverage under the model. It does not mean that there is a 95% probability that this already calculated fixed interval contains μ.

3. The Sample Variance and Cochran’s Theorem

The central result for normal samples is that the sample mean and sample variance are independent. More specifically,

where χ²_n−1 denotes a chi-square random variable with n − 1 degrees of freedom. In addition,

This result is sometimes presented through Cochran’s theorem. The total standardized sum of squares can be decomposed as

The left side has a chi-square distribution with n degrees of freedom. The first term has n − 1 degrees of freedom, and the second term has one degree of freedom. Their independence is a distinctive property of normal populations.

The sample variance is unbiased:

However, S is not generally an unbiased estimator of σ. Unbiasedness of S² does not imply that every transformation of S² is unbiased.

Why one degree of freedom is lost

The n deviations X_i − X̄ always sum to zero:

Once n − 1 deviations are known, the last is determined. Only n − 1 independent pieces of information remain for estimating variation.

4. Student’s t Distribution

4.1 Derivation

When σ is unknown, it is replaced by S. The statistic

has a Student’s t distribution with n − 1 degrees of freedom:

To see this, write

The numerator is standard normal, the square of the denominator is related to an independent chi-square variable, and the independence of X̄ and S² satisfies the defining construction of the t distribution.

The t distribution is symmetric about zero and has heavier tails than the standard normal distribution. With increasing degrees of freedom, its tails become lighter and it approaches N(0,1). Using the normal distribution when σ is estimated can understate uncertainty, especially for small samples.

4.2 Inference for a population mean

A (1 − α) confidence interval for μ is

where t_α/2,n−1 is the t critical value with n − 1 degrees of freedom.

For testing H₀: μ = μ₀, use

The test is exact when the observations are independent and normally distributed. For a one-sample t procedure, the sample should represent the target process and should not be selected in a way that preferentially includes successful or unsuccessful units.

Worked Example 2: Unknown standard deviation

A random sample of eight batches gives an average impurity concentration of X̄ = 2.40% and sample standard deviation S = 0.16%. Construct a 95% confidence interval for the mean concentration.

Here n = 8, so the degrees of freedom are seven. The standard error is

Using t_0.025,7 ≈ 2.365, the margin of error is

Thus the interval is

The inference is exact under the normal-population assumption. With only eight observations, a normality check and investigation of possible outliers are particularly important.

5. The Chi-Square Distribution and One Variance

5.1 Definition and properties

If Z₁, ..., Z_ν are independent standard-normal variables, then

has a chi-square distribution with ν degrees of freedom. It is nonnegative and generally right-skewed, although the skewness decreases as ν increases.

For a normal sample,

This permits inference about the population variance σ² or standard deviation σ.

5.2 Confidence interval for variance

Let χ²_p,ν denote the chi-square quantile with cumulative probability p. A (1 − α) confidence interval for σ² is

The order of the quantiles is reversed because the statistic is in the denominator. A confidence interval for σ is obtained by taking the positive square root of each endpoint.

5.3 Hypothesis tests for variance

For H₀: σ² = σ₀², the test statistic is

Under the null hypothesis it follows χ²_n−1. For an upper-tailed alternative H₁: σ² > σ₀², large values provide evidence against H₀. For a lower-tailed alternative, small values are critical. A two-sided test uses both tails, which are unequal in numerical distance because the chi-square distribution is skewed.

Worked Example 3: Testing process variability

A quality standard requires a process standard deviation no greater than 3 units. A random sample of 15 observations has S = 4.2 units. Test H₀: σ² = 9 against H₁: σ² > 9 at the 5% level.

The statistic is

The degrees of freedom are 14. The upper 5% critical value is approximately χ²_0.95,14 = 23.685. Since 27.44 > 23.685, reject H₀. There is statistically significant evidence that the process variance exceeds 9 square units, assuming normality and random sampling.

Statistical significance does not by itself establish that the process is operationally unacceptable; the organization must compare the result with specification limits, risk, and the cost of corrective action.

6. The F Distribution and Ratio of Variances

6.1 Definition

If U ~ χ²_ν1 and V ~ χ²_ν2 are independent, then

The F distribution is nonnegative and right-skewed. It has two degrees-of-freedom parameters: numerator degrees of freedom ν₁ and denominator degrees of freedom ν₂. Interchanging them changes the distribution.

For two independent normal samples,

has distribution

when the null hypothesis is σ₁² = σ₂². More generally, S₁²/S₂² estimates the ratio σ₁²/σ₂².

6.2 Confidence interval for a variance ratio

Let F_{p,ν₁,ν₂} denote an F quantile. A (1 − α) confidence interval for σ₁²/σ₂² is

where ν₁ = n₁ − 1 and ν₂ = n₂ − 1. The interval is asymmetric because the F distribution is skewed.

6.3 Assumptions and alternatives

The classical F procedure assumes that the two samples are independent and each comes from a normal population. It is highly sensitive to nonnormality, particularly outliers. If normality is doubtful, robust or resampling-based procedures may be preferable. For comparing means with unequal variances, Welch’s t method is often more reliable than first conducting a separate variance test and then choosing a pooled method.

7. Integrated Quality-Management Application

Illustrative Case: Comparing curing-process consistency

Background. A manufacturer operates two curing ovens. Management wants to know whether Oven A is more consistent than Oven B, because inconsistency increases rework and complicates downstream scheduling.

Problem. Independent random samples are collected under comparable operating conditions. Oven A produces n₁ = 12 observations with S₁ = 1.8 minutes, while Oven B produces n₂ = 10 observations with S₂ = 1.2 minutes. Assume the measurements are approximately normal.

Analysis. The observed variance ratio is

The relevant degrees of freedom are 11 and 9. A one-sided test of whether Oven A is more variable uses

The analyst should compare 2.25 with the upper-tail critical value at the selected significance level, or calculate the corresponding p-value. Before relying on the result, the analyst should inspect time order, control charts, histograms, and possible special causes. If observations are autocorrelated because the process drifts over time, the independent-sample F model is not valid.

Approach and solution. If the test indicates a meaningful excess of variability, the team should not merely adjust the statistical model. It should investigate oven temperature uniformity, sensor calibration, loading patterns, operator practices, and maintenance records. A designed experiment may then separate controllable factors from random variation.

Outcome. The statistical analysis identifies whether the observed difference is inconsistent with equal variances, while process investigation identifies why the difference exists. A statistically significant result without an engineering cause may lead to ineffective action; conversely, a practically important difference may fail to reach significance when samples are small.

Learning points. Sampling distributions support evidence-based decisions, but they do not replace process knowledge. Independence, rational sampling, normality, effect size, and practical consequences must be considered together.

8. Assumptions, Diagnostics, and Limitations

Normality

Exact t, chi-square, and F results depend on normal populations. Mild nonnormality may have limited impact on the sample mean when samples are moderate and distributions are not heavily skewed. Variance procedures are more fragile. Use probability plots, residual plots, subject-matter knowledge, and outlier investigations rather than relying on a single normality test.

Independence

Independence can be threatened by repeated measurements on the same unit, time-series dependence, batch effects, or spatial correlation. Increasing sample size does not correct a biased or dependent sampling design. In process data, rational subgrouping and time-order plots are especially important.

Randomness and representativeness

A random sample from the wrong population is still uninformative about the desired population. Sampling should define the process, period, products, and operating conditions to which the conclusion will apply.

Statistical versus practical significance

A very large sample can detect a trivial difference, while a small sample can miss an operationally important effect. Report estimates, confidence intervals, and practical thresholds, not only p-values.

Multiple comparisons and selective reporting

Repeatedly testing many variances, means, or process stages increases the probability of a false positive. Pre-specify primary comparisons, control the familywise or false-discovery error when appropriate, and report all material analyses. Data exclusion and model selection should be documented.

Technology and software governance

Statistical software calculates quantiles and p-values efficiently, but it does not verify sampling design or assumptions. Analysts should record software version, input data, parameterization of quantiles, degrees of freedom, and reproducible code. AI-assisted analytics can help identify patterns or generate diagnostic suggestions, but final decisions require human validation, traceability, privacy protection, and domain expertise.

9. Best-Practice Workflow

1. Define the population, parameter, decision, and acceptable practical error.
2. Design an independent and representative sampling plan.
3. Inspect data quality, time order, missingness, and measurement-system adequacy.
4. Select the exact distribution appropriate to the parameter and assumptions.
5. State the null hypothesis, alternative, significance level, and degrees of freedom before calculating.
6. Report the estimate, standard error or variability measure, confidence interval, test statistic, and p-value.
7. Examine assumptions and sensitivity to outliers or nonnormality.
8. Translate statistical evidence into process action, risk assessment, and follow-up measurement.

Chapter Summary

For a random sample from a normal population, the sample mean has an exact normal distribution with mean μ and variance σ²/n. When σ is known, the standardized sample mean uses the standard-normal distribution. When σ is unknown, replacing it with S produces Student’s t with n − 1 degrees of freedom.

The sample variance satisfies (n−1)S²/σ² ~ χ²_n−1, and it is independent of the sample mean. This result supports exact confidence intervals and tests for one population variance. For two independent normal samples, the ratio S₁²/S₂² follows an F distribution under equal population variances, supporting inference about variance ratios.

These methods are central to quality-management inference, but their validity depends on normality, independence, and representative sampling. Sound professional practice combines distributional calculations with diagnostics, process knowledge, effect-size interpretation, transparent reporting, and appropriate governance.

Key Terms

Normal population: A population whose measurements follow a normal probability distribution.

Random sample: A sample whose observations are selected according to a design that supports the stated probability model.

Sample mean: The arithmetic average X̄ of sample observations.

Sample variance: The unbiased estimator S² of population variance using denominator n−1.

Standard error: The standard deviation of a sampling distribution.

Degrees of freedom: The number of independent pieces of information available for estimating a quantity.

Standard normal distribution: The normal distribution with mean zero and variance one.

Student’s t distribution: A symmetric distribution used for standardized means when population variance is estimated.

Chi-square distribution: The distribution of a sum of squared independent standard-normal variables.

F distribution: The distribution of a ratio of two independent chi-square variables after division by their degrees of freedom.

Population mean: The parameter μ describing the center of a population.

Population variance: The parameter σ² describing population dispersion.

Variance ratio: The ratio σ₁²/σ₂² of two population variances.

Critical value: A quantile defining a rejection-region boundary.

p-value: The probability, under the null model, of observing a result at least as extreme as the one obtained.

Confidence interval: An interval produced by a procedure with stated long-run coverage probability.

Independence: A condition in which observations provide no probabilistic information about one another.

Normality assumption: The assumption that observations arise from a normal population or that the relevant model is adequately approximated by normality.

Practical significance: The operational importance of an effect, independent of statistical significance alone.

Review Questions

5 Short Answer Questions

1. What is the sampling distribution of the sample mean from a normal population?
2. Why does the sample variance use n − 1 in its denominator?
3. State the distribution of (n − 1)S²/σ².
4. When should Student’s t be used instead of the standard-normal distribution?
5. What are the two degrees-of-freedom parameters of an F distribution?

5 Long Answer Questions

1. Derive the mean and variance of the sample mean for independent observations from N(μ,σ²).
2. Explain the independence of X̄ and S² and its importance for t inference.
3. Derive the confidence interval for a normal-population variance using the chi-square distribution.
4. Compare the normal, t, chi-square, and F distributions in terms of purpose, parameters, and assumptions.
5. Discuss how nonnormality, dependence, and nonrepresentative sampling affect exact inference in quality management.

3 Analytical / Application Questions

1. A process sample of 16 units has X̄ = 48, S = 5. Construct a 95% confidence interval for the population mean and explain the role of the t distribution.
2. A sample of 20 observations has S² = 12. Test whether the population variance exceeds 8 at the 5% level. State the statistic, degrees of freedom, rejection rule, and conclusion using an appropriate chi-square table.
3. Two independent processes have sample standard deviations 6 and 4, with sample sizes 11 and 13. Formulate an F-based investigation of whether their population variances differ, and identify the assumptions and practical issues that must be checked before acting on the result.

Multiple Choice Questions with Answers

Question 1

For a random sample from N(μ,σ²), what is the distribution of X̄?

A. N(μ,σ²)

B. N(μ,σ²/n)

C. t_n−1

D. χ²_n−1

Correct Answer: B. N(μ,σ²/n)

Question 2

If σ is unknown and the population is normal, which statistic is used to test H₀:μ=μ₀?

A. (X̄−μ₀)/(σ/√n)

B. (X̄−μ₀)/(S/√n)

C. (n−1)S²/σ²

D. S₁²/S₂²

Correct Answer: B. (X̄−μ₀)/(S/√n)

Question 3

The statistic (n−1)S²/σ² has which distribution under normal sampling?

A. Standard normal with n degrees of freedom

B. Student’s t with n−1 degrees of freedom

C. Chi-square with n−1 degrees of freedom

D. F with n and n−1 degrees of freedom

Correct Answer: C. Chi-square with n−1 degrees of freedom

Question 4

Why does the sample variance have n−1 degrees of freedom?

A. One observation is always discarded

B. The population variance is assumed to be zero

C. The deviations from the sample mean sum to zero

D. The sample mean has n−1 units

Correct Answer: C. The deviations from the sample mean sum to zero

Question 5

What happens to the t distribution as its degrees of freedom increase?

A. It approaches the standard-normal distribution

B. It approaches the chi-square distribution

C. It becomes concentrated only on positive values

D. Its variance becomes exactly zero

Correct Answer: A. It approaches the standard-normal distribution

Question 6

A 95% confidence interval for a normal population mean with unknown variance is generally based on which critical value?

A. z_0.025

B. t_0.025,n−1

C. χ²_0.95,n−1

D. F_{0.95,n−1,n−1}

Correct Answer: B. t_0.025,n−1

Question 7

Under equal population variances, the ratio S₁²/S₂² follows which distribution for two independent normal samples?

A. Chi-square

B. Student’s t

C. F

D. Poisson

Correct Answer: C. F

Question 8

A sample has n=10 and S²=4. To test H₀:σ²=4, what is the chi-square statistic?

A. 2.25

B. 9

C. 10

D. 36

Correct Answer: C. 10

Question 9

Which condition is especially important for the exact classical F test comparing two variances?

A. Both samples must have equal means

B. Both populations should be normal and samples independent

C. Both samples must have the same sample size

D. The population standard deviations must be known

Correct Answer: B. Both populations should be normal and samples independent

Question 10

If the standard error of a sample mean is to be reduced by one-half, what sample-size change is required, all else equal?

A. Double the sample size

B. Increase the sample size by 50%

C. Multiply the sample size by four

D. Divide the sample size by four

Correct Answer: C. Multiply the sample size by four