Sampling Distributions and the Central Limit Theorem

Introduction

Business decisions are usually made about populations, but analysts commonly observe only samples. A retailer may want to estimate the average spending of all customers from a sample of transactions; a bank may estimate the proportion of applicants who repay loans from a sample of borrowers; and a manufacturer may assess average delivery time from selected shipments. The central statistical challenge is that different samples produce different results.

This chapter develops the theory needed to understand and measure that variation. It introduces sampling distributions, standard error, the law of large numbers, and the central limit theorem (CLT). These concepts provide the foundation for confidence intervals, hypothesis tests, quality-control procedures, survey analysis, and many predictive and operational decisions.

A central distinction is between a population parameter and a sample statistic. A parameter, such as the population mean μ or population proportion p, is generally fixed but unknown. A statistic, such as the sample mean x̄ or sample proportion p̂, is calculated from data and varies from sample to sample.

Learning Objectives

By the end of this chapter, you should be able to:

1. Explain why sample statistics vary across random samples.
2. Define and derive sampling distributions for means and proportions.
3. Calculate and interpret standard errors.
4. Explain the law of large numbers and the central limit theorem.
5. Identify conditions required for using normal approximations.
6. Apply sampling-distribution results to business analytics decisions.
7. Recognize sources of bias and limitations that sampling theory cannot correct.

1. Population, Sample, Parameter, and Statistic

A population is the complete set of units relevant to a study. It may consist of all customers, invoices, employees, products, or transactions in a defined context. A sample is a subset selected for analysis.

A parameter describes a population. Common parameters include:

• Population mean: μ
• Population variance: σ²
• Population standard deviation: σ
• Population proportion: p

A statistic describes a sample. Common statistics include:

• Sample mean: x̄
• Sample variance: s²
• Sample standard deviation: s
• Sample proportion: p̂

For a sample of n numerical observations x₁, x₂, …, xₙ, the sample mean is:

Here, n is the sample size and xᵢ is the value of observation i. The sample mean has the same units as the observations.

If X is a binary variable, taking the value 1 when an event occurs and 0 otherwise, then the sample proportion is:

where X is the number of observed successes. For example, if 84 of 200 customers make a repeat purchase, p̂ = 84 ÷ 200 = 0.42, or 42%.

A statistic is often used to estimate a parameter. However, an estimate is not expected to equal the parameter exactly. The difference arises from sampling variability, not necessarily from an error in calculation.

2. Sampling Variability and Sampling Distributions

Suppose a population contains many possible observations. If we repeatedly select random samples of the same size and calculate a statistic for every sample, the resulting collection of statistic values forms a sampling distribution.

For the sample mean, the sampling distribution describes the possible values of x̄. For the sample proportion, it describes the possible values of p̂. A sampling distribution is theoretical when it describes all possible samples, but it can also be approximated through simulation.

Sampling variability depends mainly on:

• The amount of natural variation in the population.
• The sample size.
• The sampling design.
• Whether observations are independent.

Larger samples usually produce less variable statistics. This is why a mean calculated from 1,000 transactions is generally more stable than a mean calculated from 10 transactions. Larger samples do not automatically eliminate bias, however. A very large sample selected from the wrong population can produce a very precise but systematically misleading result.

The mean and variance of a sampling distribution

If X₁, X₂, …, Xₙ are independent observations from a population with mean μ and variance σ², then:

and

Therefore, the standard deviation of the sampling distribution of x̄ is:

The quantity σₓ̄ is called the standard error of the mean. It measures the typical sampling variation of sample means around μ. The formula assumes random sampling or an equivalent independent-sampling process and is most directly applicable when the population standard deviation σ is known.

The expression shows an important result: increasing n by a factor of 4 reduces the standard error by a factor of 2, because the denominator contains √n rather than n.

When σ is unknown, it is usually estimated by s. The estimated standard error is:

This estimate is appropriate when the sample is random and observations are independent. For small samples from a normally distributed population, inference about μ generally uses the t distribution rather than the standard normal distribution.

3. The Law of Large Numbers

The law of large numbers (LLN) states, informally, that as the sample size increases, the sample average tends to get closer to the population mean, provided observations are appropriately sampled and have a finite expected value.

In probability notation, under standard conditions:

The symbol → indicates convergence, while a.s. means almost surely. At an introductory level, the practical interpretation is more important than the notation: larger random samples tend to provide more stable estimates of population averages.

The LLN does not say that every large sample will equal μ, nor does it specify how quickly convergence occurs. It also does not correct systematic sampling bias. For example, if a company surveys only its most loyal customers, increasing the number of surveyed loyal customers may produce a more precise estimate of loyal-customer satisfaction, but not of satisfaction among all customers.

The LLN and the CLT answer different questions:

4. The Central Limit Theorem

The central limit theorem is one of the most important results in statistics. Let X₁, X₂, …, Xₙ be independent and identically distributed observations with mean μ and finite variance σ². As n becomes sufficiently large, the standardized sample mean approaches a standard normal distribution:

Here, N(0, 1) denotes a normal distribution with mean 0 and variance 1. Equivalently, the sampling distribution of x̄ is approximately:

The CLT is powerful because the original population need not be normal when n is sufficiently large. A skewed population, for example, can produce an approximately normal sampling distribution of the mean when the sample is large enough.

The phrase “sufficiently large” has no universal numerical threshold. It depends on population shape, outliers, dependence, and the purpose of the analysis. A sample of 30 may be adequate for a mildly skewed population, whereas a highly skewed population with extreme outliers may require a much larger sample. If the population itself is normal, the sampling distribution of x̄ is normal for every sample size.

Why the CLT matters in business analytics

The CLT permits analysts to:

• Approximate probabilities involving sample means.
• Construct confidence intervals.
• Test claims about means.
• Compare average performance across groups.
• Quantify uncertainty in dashboards and forecasts.
• Make operational decisions without observing the entire population.

The CLT concerns the distribution of a statistic across repeated samples. It does not claim that the raw observations become normal. Individual transaction values may remain strongly skewed even when the distribution of sample means is approximately normal.

5. Sampling Distribution of the Sample Mean

Assume a random sample of n independent observations from a population with mean μ and standard deviation σ. The sampling distribution of x̄ has:

If the population is normal, then:

If the population is not normal but n is sufficiently large, then the CLT gives the approximation:

To calculate a probability about x̄ when σ is known, standardize the value:

Worked Example 1: Average delivery time

A logistics firm knows from historical records that delivery times have mean μ = 48 hours and standard deviation σ = 12 hours. A random sample of n = 36 deliveries is selected. What is the approximate probability that the sample mean exceeds 51 hours?

First calculate the standard error:

Then standardize 51 hours:

Thus, the probability is P(Z > 1.5), which is approximately 0.0668, or 6.68%.

The correct interpretation is: if repeated random samples of 36 deliveries were selected, approximately 6.68% of their sample means would exceed 51 hours, assuming the historical population model remains valid. It is not correct to say that 6.68% of individual deliveries exceed 51 hours.

Finite population correction

When sampling without replacement from a finite population, observations are not fully independent if the sample is a substantial fraction of the population. If the population size is N and n/N is more than approximately 5%, the finite population correction may be used:

The correction reduces the standard error because sampling a large fraction of a finite population leaves less uncertainty about the remaining units. It is not normally needed when the population is very large relative to the sample.

6. Sampling Distribution of a Sample Proportion

Suppose each observation is classified as a success or failure. Let p be the population proportion of successes and let p̂ be the sample proportion. For a random sample of n independent observations:

and

Therefore, the standard error is:

Because p is usually unknown, analysts often estimate the standard error using p̂:

For a normal approximation to the sampling distribution of p̂, the expected numbers of successes and failures should be sufficiently large. A common rule is:

When p is unknown, p̂ may be used for an initial check. Some applications use a more conservative threshold, such as 5 or 10, depending on the required accuracy.

Under the success–failure condition:

To calculate a probability about p̂ when p is specified:

Worked Example 2: Repeat purchase rate

An online retailer believes that p = 0.40 of customers make a repeat purchase within 30 days. A random sample of n = 400 customers is observed. What is the approximate probability that the sample repeat-purchase rate is at least 0.45?

The expected numbers are:

Both exceed 10, so the normal approximation is reasonable.

The standard error is:

The standardized value is:

The upper-tail probability P(Z ≥ 2.04) is approximately 0.0207, or 2.07%. Thus, a sample rate of at least 45% would be relatively unusual if the true rate were 40%.

7. Standard Error, Bias, and Practical Interpretation

The standard error measures the typical random variation of a statistic across repeated samples. It is not the same as the standard deviation of individual observations.

For example, customer spending may have a standard deviation of $80, while the standard error of the mean spending for a sample of 400 customers may be:

The individual spending values vary by about $80 around the population mean, but sample means of 400 customers typically vary by about $4 around μ.

Standard error should also be distinguished from bias. Bias is systematic departure from the target parameter. If a sample mean has expected value different from μ, then it is biased. Random sampling variability may be reduced by increasing n; sampling bias is not reliably reduced by increasing n.

A useful conceptual decomposition is:

This is a conceptual expression rather than an exact universal identity for every error metric. It emphasizes that precision and validity are different. A narrow standard error indicates precision, not necessarily accuracy.

8. Conditions for Statistical Inference

Sampling-distribution methods depend on assumptions. Before applying a normal approximation or a confidence interval, analysts should evaluate the following conditions.

Randomness and representativeness

The sample should be obtained through a probability-based method or a defensible process approximating random selection. Convenience samples, voluntary-response surveys, and samples restricted to easily accessible users may not represent the target population.

Independence

Observations should be independent, or the dependence should be explicitly modeled. Independence may be reasonable when sampling with replacement or when sampling without replacement from a very large population. It may fail when:

• Several observations come from the same customer.
• Employees are sampled within the same branch.
• Orders are collected in time clusters.
• Repeated measurements are taken from the same unit.
• A time series contains serial dependence.

When observations are positively correlated, the usual standard error formula tends to be too small, creating excessive confidence.

Sample size and population shape

For means, examine the population or sample distribution. Strong skewness and outliers require larger samples or robust methods. For proportions, check the expected success and failure counts.

Measurement quality

A statistically valid sampling distribution cannot repair poor measurement. Ambiguous survey questions, inconsistent definitions, missing data, faulty sensors, and duplicate records may produce misleading results even with a large sample.

Independence from the sampling frame

The sampling frame—the operational list from which units are selected—should correspond adequately to the target population. Excluding mobile-only customers from an online survey, for example, may create coverage bias.

9. Simulation as a Learning and Validation Tool

Simulation makes sampling variability visible. Suppose a population distribution is stored in a database. An analyst can repeatedly draw random samples of size n, calculate x̄ for each sample, and plot the resulting means. The histogram of means approximates the sampling distribution.

A basic simulation procedure is:

1. Define the population or a plausible generating distribution.
2. Select a sample size n.
3. Draw a random sample.
4. Calculate the statistic.
5. Repeat many times.
6. Examine the center, spread, and shape of the simulated statistics.

Simulation can demonstrate that:

• The center of the sampling distribution is near the population parameter.
• Larger n produces a narrower distribution.
• The distribution of means becomes more bell-shaped.
• Extreme population skewness may delay normal approximation.

Simulation does not replace sound sampling design. If the simulated population is biased or the repeated samples do not reflect the real design, the result may be visually persuasive but substantively wrong.

10. Applied Case: Evaluating a Customer-Service Target

Background

Illustrative case: a national telecommunications company promises that customer-service calls will be answered within 4 minutes on average. Management wants to determine whether recent performance is consistent with this target.

Problem

A random sample of 64 calls has mean waiting time x̄ = 4.5 minutes and sample standard deviation s = 2.4 minutes. The population standard deviation is unknown. The analyst must assess the precision of the estimate and identify an appropriate inferential approach.

Analysis

The estimated standard error is:

The observed mean is 0.5 minutes above the target. Relative to the estimated standard error, this difference is:

Because σ is unknown, a t-based method is appropriate. With n − 1 = 63 degrees of freedom, the analyst would compare this result with a t distribution rather than automatically using a standard normal distribution. A one-sided test could be formulated as:

The decision should depend on a preselected significance level, the sampling process, the distribution of waiting times, and whether calls are independent. If calls are collected during one unusually busy shift, the sample may not represent normal operations.

Approach and solution

The analyst should verify that calls were randomly selected across days and times, check for extreme waiting times, confirm that each call is a distinct customer interaction, and report the estimate with a confidence interval or p-value. If waiting times are highly skewed, a larger sample, transformation, bootstrap method, or robust analysis may be appropriate.

Outcome and learning points

The sample suggests performance may be above the four-minute target, but the conclusion must account for sampling uncertainty and design quality. A point estimate alone is insufficient. Managers should distinguish “the observed average was 4.5 minutes” from “the long-run population average exceeds 4 minutes.”

The case illustrates five principles:

1. A sample mean is variable.
2. Standard error quantifies that variability.
3. Unknown σ generally leads to t-based inference for a mean.
4. Sampling design affects validity.
5. Statistical evidence should be combined with operational knowledge.

11. Sampling in Technology-Enabled Business Analytics

Modern organizations generate large volumes of data through transactions, sensors, applications, and digital platforms. Large data volume does not automatically create a valid sample. Platform users may differ from nonusers, automated systems may record some events more accurately than others, and repeated observations may be clustered by user or device.

Analytics platforms can automate random sampling, bootstrap resampling, simulation, and standard-error calculations. Machine-learning systems may also use sampled training data. Governance is needed to document the target population, sampling frame, inclusion rules, missing-data treatment, and version of the data used.

Artificial intelligence can help detect unusual observations or suggest strata for stratified sampling, but automated recommendations should be reviewed. An algorithm trained on a biased historical sample may reproduce or amplify that bias. In high-stakes applications such as credit, employment, insurance, and healthcare, analysts should examine representativeness, fairness, privacy, and explainability.

12. Limitations and Best Practices

Sampling-distribution theory has important limitations:

• It assumes a clearly defined target population.
• Standard formulas may fail under dependence or severe nonresponse.
• Normal approximations can be poor for small samples or extreme skewness.
• A large sample cannot correct systematic bias.
• Statistical significance does not establish practical importance.
• Data quality problems may dominate sampling error.

Best practices include:

1. Define the population and estimand before collecting data.
2. Use a probability sample where feasible.
3. Document the sampling frame and selection process.
4. Check independence and clustering.
5. Examine distributions and outliers.
6. Report estimates together with standard errors or intervals.
7. Distinguish statistical from practical significance.
8. Use finite population corrections when appropriate.
9. Conduct sensitivity or simulation analyses when assumptions are uncertain.
10. Protect privacy and disclose important limitations.

Chapter Summary

Sampling variability is the natural variation among statistics calculated from different samples. The sampling distribution describes this variation. For independent observations, the sample mean is centered at μ and has standard error σ ÷ √n; when σ is unknown, s ÷ √n estimates the standard error. The law of large numbers explains why sample averages tend toward the population mean as sample size increases.

The central limit theorem explains why the sampling distribution of the mean is approximately normal for sufficiently large samples, even when the population is not normal. For sample proportions, the mean is p and the standard error is √[p(1 − p) ÷ n], with normal approximation commonly requiring sufficiently large expected counts of successes and failures.

Standard error measures random sampling variation, whereas bias reflects systematic error. Reliable inference requires attention to random sampling, independence, sample size, population shape, measurement quality, and the sampling frame. These principles support confidence intervals, hypothesis tests, quality control, survey analysis, and responsible business decision-making.

Key Terms

Population: The complete set of units relevant to a study.

Sample: A subset of the population selected for analysis.

Parameter: A numerical characteristic of a population.

Statistic: A numerical characteristic calculated from a sample.

Sampling variability: Natural variation in a statistic across repeated samples.

Sampling distribution: The probability distribution of a statistic across possible samples.

Sample mean: The arithmetic average of sample observations.

Sample proportion: The fraction of sample observations classified as successes.

Standard error: The standard deviation of a statistic’s sampling distribution.

Law of large numbers: The principle that sample averages tend toward the population mean as sample size grows.

Central limit theorem: The result that standardized sums or means become approximately normal under broad conditions.

Finite population correction: An adjustment that reduces standard error when sampling a substantial fraction of a finite population.

Independence: The condition that one observation does not provide information about another.

Bias: Systematic deviation of an estimator from the target parameter.

Normal approximation: Use of a normal distribution to approximate another sampling distribution.

Success–failure condition: A check that expected successes and failures are sufficiently numerous for proportion inference.

Sampling frame: The operational list or source from which a sample is selected.

Representativeness: The degree to which a sample reflects the target population.

Precision: The degree of random variation, often summarized by a standard error or interval width.

Review Questions

5 Short Answer Questions

1. Distinguish a parameter from a statistic and provide one example of each.
2. What does the standard error of a sample mean measure?
3. State the practical meaning of the law of large numbers.
4. Why does increasing the sample size reduce the standard error of a mean?
5. What conditions support a normal approximation for a sample proportion?

5 Long Answer Questions

1. Explain the central limit theorem and discuss why it is important for business analytics.
2. Derive the mean and variance of the sampling distribution of the sample mean under independent sampling.
3. Compare sampling variability with sampling bias. Explain why a large sample does not necessarily eliminate bias.
4. Discuss how dependence, skewness, outliers, and finite population size affect statistical inference.
5. Explain how simulation can be used to investigate a sampling distribution and evaluate the usefulness of a normal approximation.

3 Analytical / Application Questions

1. A process has standard deviation σ = 18 units. Compare the standard errors of sample means for n = 36 and n = 144. Interpret the result for an operations manager.
2. A survey finds that 180 of 500 respondents support a proposed service. Assess whether a normal approximation for p̂ is reasonable and calculate the estimated standard error.
3. A company reports a highly precise estimate based on a very large convenience sample. Evaluate the claim and identify the additional evidence needed before generalizing to all customers.

Multiple Choice Questions with Answers

Question 1

Which statement best describes a sampling distribution?

A. The distribution of all observations in the population

B. The distribution of a statistic across repeated possible samples

C. The distribution of measurement errors in a single observation

D. The distribution of sample sizes selected by an analyst

Correct Answer: B. The distribution of a statistic across repeated possible samples

Question 2

A population has standard deviation σ = 20. What is the standard error of the sample mean for n = 100 independent observations?

A. 0.2

B. 2

C. 10

D. 200

Correct Answer: B. 2

Question 3

What does the law of large numbers imply?

A. Every large sample is normally distributed

B. A sample mean tends to approach the population mean as sample size increases

C. Sampling bias disappears when n exceeds 30

D. The population standard deviation becomes zero

Correct Answer: B. A sample mean tends to approach the population mean as sample size increases

Question 4

A population has μ = 50 and σ = 10. For n = 25, what is the standard deviation of the sampling distribution of x̄?

A. 0.4

B. 2

C. 5

D. 10

Correct Answer: B. 2

Question 5

Which condition is most directly relevant to a normal approximation for a sample proportion?

A. The sample mean must equal the population mean

B. The population must have zero variance

C. Expected successes and failures should both be sufficiently large

D. The sample must contain exactly 30 observations

Correct Answer: C. Expected successes and failures should both be sufficiently large

Question 6

If the sample size is multiplied by 4, what happens to the standard error of a sample mean, all else equal?

A. It doubles

B. It is divided by 2

C. It is divided by 4

D. It does not change

Correct Answer: B. It is divided by 2

Question 7

A sample contains 84 successes out of 200 observations. What is the sample proportion?

A. 0.084

B. 0. twenty-one

C. 0.42

D. 2.38

Correct Answer: C. 0.42

Question 8

Which statement correctly distinguishes standard error from bias?

A. Standard error is systematic, while bias is random

B. Standard error measures random sampling variation, while bias is systematic deviation

C. They are two names for the same quantity

D. Bias can always be eliminated by increasing sample size

Correct Answer: B. Standard error measures random sampling variation, while bias is systematic deviation

Question 9

When the population standard deviation is unknown and a sample mean is used for inference, which distribution is commonly used, especially for smaller samples?

A. Uniform distribution

B. Poisson distribution

C. t distribution

D. Exponential distribution

Correct Answer: C. t distribution

Question 10

A sample of 400 customers is drawn from a population with p = 0.40. Which pair gives the expected numbers of successes and failures?

A. 40 and 360

B. 160 and 240

C. 200 and 200

D. 400 and 0

Correct Answer: B. 160 and 240