Sampling Distributions and Statistical Estimation

Introduction

Business decisions are usually made from samples rather than complete populations. A retailer may survey 600 customers instead of all customers; a bank may examine a sample of loans rather than its entire portfolio; and a digital platform may evaluate a sample of users exposed to a new interface. The central statistical problem is therefore not merely to describe observed data, but to quantify how much sample-based conclusions may vary from one sample to another.

Sampling distributions provide the theoretical foundation for this task. They describe the behaviour of a statistic across repeated samples from a population. Statistical estimation uses that behaviour to infer unknown population quantities, express uncertainty through confidence intervals, and determine how much data are needed for a desired level of precision. Bootstrap methods extend these ideas when analytical formulas are difficult or unreliable.

Learning Objectives

By the end of this chapter, readers should be able to:

1. Distinguish populations, samples, parameters and statistics.
2. Select and evaluate probability and non-probability sampling designs.
3. Explain the central limit theorem and its relevance to business analytics.
4. Calculate and interpret standard errors for common estimators.
5. Construct and critically interpret confidence intervals.
6. Apply bootstrap methods to estimate uncertainty.
7. Determine sample sizes for means and proportions.
8. Recognize sources of sampling error, bias and misleading precision.
9. Communicate statistical uncertainty responsibly in managerial settings.

1. The Inferential Setting

A population is the complete set of units relevant to a question. A target population is the conceptual population to which conclusions are intended to apply, whereas a sampling frame is the operational list from which observations are selected. A sample is the subset actually observed.

A parameter is a fixed but usually unknown population characteristic, such as the population mean μ, variance σ², or proportion p. A statistic is a quantity calculated from sample data, such as the sample mean x̄ or sample proportion p̂. An estimator is a rule for using sample data to estimate a parameter; an estimate is the numerical result produced by that rule.

The distinction between variability and bias is fundamental. Sampling variability arises because different random samples produce different values. Sampling bias arises when the design systematically favours some population units or measurements. A very large biased sample can produce a highly precise estimate of the wrong quantity.

2. Sampling Designs

2.1 Probability sampling

In probability sampling, each population unit has a known, non-zero probability of selection. This permits design-based inference and supports valid estimates of sampling error.

Simple random sampling gives every sample of a specified size an equal chance of selection. It is conceptually simple but may be expensive and requires a complete sampling frame.

Systematic sampling selects every kth unit after a random starting point, where k is approximately N ÷ n, N is the frame size and n is the desired sample size. It is efficient, but periodic patterns in the frame can create bias.

Stratified sampling divides the population into relatively homogeneous strata, such as regions or customer tiers, and samples within each stratum. Proportional allocation samples in proportion to stratum size. Disproportionate allocation can improve precision for small or strategically important groups, but weights are then required for population estimates.

Cluster sampling selects naturally occurring groups, such as stores, branches or schools, and observes units within selected clusters. It reduces field costs but often increases sampling variance because units in the same cluster tend to resemble one another.

Multistage sampling combines stages, for example selecting regions, then stores, then customers. Its variance must reflect all relevant stages.

2.2 Non-probability sampling

Convenience, quota, purposive and voluntary-response samples are often practical, particularly in exploratory research. However, selection probabilities are unknown, so conventional confidence intervals may not represent repeated-sampling performance. Weighting, calibration and sensitivity analysis can reduce—but do not automatically eliminate—coverage and self-selection problems.

A professional analyst should document the target population, frame, eligibility rules, selection mechanism, response rate, replacements and weighting procedure. These design details are as important as the final calculation.

3. Sampling Distributions

The sampling distribution of a statistic is the probability distribution of that statistic over all possible samples of a specified design and size. It is not the distribution of individual observations.

For a simple random sample of n independent observations with population mean μ and variance σ², the sample mean is

Its expected value and variance are

and its standard deviation, called the standard error when estimated or theoretically specified for an estimator, is

These results assume independent observations with a common finite variance. For sampling without replacement from a finite population of size N, the finite population correction may be relevant:

The correction is usually negligible when the sampling fraction n ÷ N is small, commonly below about 5%.

If the population is normally distributed, x̄ is exactly normally distributed for every n. If the population is not normal, the distribution of x̄ may still be approximately normal for sufficiently large n under suitable regularity conditions.

4. The Central Limit Theorem

The central limit theorem (CLT) states, in its common independent-identically-distributed form, that as n becomes large, the standardized sample mean approaches a standard normal distribution, provided the population has a finite mean and variance:

The CLT does not say that raw observations become normal. It says that the distribution of the sample mean becomes approximately normal. The required sample size depends on skewness, heavy tails, dependence and the presence of outliers. A sample of 30 is not a universal guarantee.

For a sample proportion p̂ based on n independent Bernoulli trials,

where X is the number of successes. Its mean is p and its variance is p(1 − p) ÷ n. A normal approximation is generally appropriate when both np and n(1 − p) are sufficiently large; practical rules often use at least 10 for each, though exact or simulation-based methods are preferable for small counts.

Dependence changes the effective amount of information. If observations are clustered or serially correlated, the nominal n overstates information and the usual standard error is too small. Analysts should use cluster-robust methods, design effects, time-series methods or an appropriate hierarchical model.

5. Standard Errors and Estimator Quality

The standard error measures the typical sampling variation of an estimator, not the spread of individual observations. For an estimated mean with unknown σ, the usual standard error is

where s is the sample standard deviation calculated with denominator n − 1.

For an independent difference between two sample means,

This unequal-variance form is associated with Welch inference and is generally preferable to assuming equal population variances. For an independent difference in proportions,

provided the independent-sample approximation is suitable.

An estimator is unbiased if E(T) = θ, where T estimates parameter θ. It is consistent if it converges in probability to θ as sample size increases. It is efficient relative to another estimator if it has smaller variance under the relevant model. In practice, an estimator should be judged jointly on bias, variance, robustness, interpretability and the quality of the sampling design.

6. Point Estimation

Common point estimators include x̄ for μ, s² for σ², and p̂ for p. A point estimate alone is incomplete because it does not communicate precision. For example, a campaign conversion estimate of 8% has a different managerial meaning when based on 80 observations than when based on 80,000 observations.

The method of moments equates sample moments to population moments. The maximum likelihood estimator (MLE) chooses the parameter value that makes the observed data most plausible under a specified probability model. MLEs can have excellent large-sample properties, but their validity depends on an appropriate model and reliable optimization.

For a normal population with unknown mean and variance, the sample mean is the MLE of μ and the maximum-likelihood variance uses denominator n. The unbiased sample variance uses denominator n − 1. These are different objectives and should not be confused.

7. Confidence Intervals

A 100(1 − α)% confidence interval is an interval-producing procedure that captures the fixed parameter in 100(1 − α)% of repeated samples when its assumptions hold. It is not correct to say that there is a 95% probability that a fixed parameter lies in the particular interval after it has been calculated, under the frequentist interpretation.

7.1 Mean with known or effectively known σ

Here z₁₋α/₂ is the standard normal quantile. For a 95% interval, z₀.₉₇₅ ≈ 1.96.

7.2 Mean with unknown σ

When the population is approximately normal and σ is unknown, use the t distribution:

The degrees of freedom are n − 1. The t multiplier is larger than 1.96 for small samples, reflecting additional uncertainty from estimating σ.

7.3 Proportion

A simple large-sample interval is

This Wald interval can perform poorly for small n or proportions near 0 or 1. Wilson or exact binomial intervals are often better choices. An interval should not be truncated mechanically to [0,1] without acknowledging that the underlying approximation was inadequate.

Worked Example: Mean revenue

A random sample of 64 transactions has mean revenue $52 and sample standard deviation $16. Assuming independent observations and an approximately normal mean, a 95% t interval uses 63 degrees of freedom. Taking t₀.₉₇₅,₆₃ ≈ 2.00,

Thus the margin of error is approximately 2.00 × 2 = $4, and the interval is $52 ± $4, or ($48, $56). The interpretation is that the procedure used would capture the population mean in approximately 95% of repeated random samples. It does not assert that 95% of individual transactions lie in this interval.

8. Bootstrap Methods

The bootstrap approximates a sampling distribution by repeatedly resampling from the observed sample, with replacement. It is useful when the estimator is complicated, the population distribution is unknown, or an analytical standard error is difficult to derive.

For data x₁, …, xₙ:

1. Draw a bootstrap sample of size n with replacement.
2. Compute the statistic T* for that sample.
3. Repeat B times, often at least several thousand for stable interval estimates.
4. Use the empirical distribution of T* to estimate standard error or interval limits.

The bootstrap standard error is the standard deviation of the B bootstrap statistics:

A percentile interval uses the α ÷ 2 and 1 − α ÷ 2 empirical quantiles of the bootstrap distribution. Alternatives include the basic, bias-corrected and accelerated (BCa), and studentized intervals. Percentile intervals are easy to explain but may be inaccurate for strongly biased, highly skewed or boundary-constrained statistics.

Bootstrap resampling must respect the data-generating structure. A time series should generally be bootstrapped using blocks or a model-based method, and clustered data should be resampled at the cluster level. Resampling individual observations from dependent data can create false precision.

9. Sample-Size Determination

Sample size should be determined before data collection, using a specified confidence level, margin of error, variability assumption and design.

For a mean with known planning standard deviation σ and desired margin of error E,

Round upward. If σ is unknown, use a pilot estimate, historical data or a conservative planning value.

For a proportion with anticipated proportion p and margin E,

When p is unknown, p = 0.5 maximizes p(1 − p), producing a conservative size. For a 95% interval with E = 0.03 and p = 0.5,

Therefore at least 1,068 completed responses are required under the simple random-sampling approximation.

If the expected response rate is r, invitations should be approximately n ÷ r. For cluster designs, multiply by the design effect. A common approximation is

where m is average cluster size and ρ is the intracluster correlation. The effective sample size is approximately n ÷ DEFF. Sample-size planning should also consider power for a specific hypothesis test, multiple comparisons, attrition, subgroup analysis and operational cost.

10. Applied Case: Estimating Customer Churn

Background

Illustrative case: A subscription company wants to estimate the proportion of customers likely to cancel within the next quarter. Its customer database contains 240,000 active accounts across four regions.

Problem

A voluntary online poll produced a churn estimate of 14%, but respondents were disproportionately dissatisfied and mobile users were underrepresented. Management needs a defensible estimate with a margin of error near ±2 percentage points.

Analysis and approach

The analytics team defines the target population as all active accounts at the beginning of the quarter. It stratifies the frame by region and customer tenure, samples accounts randomly within strata, and applies proportional weights. It anticipates p = 0.20 and uses a 95% confidence level. The simple-random preliminary size is

Because the design uses stores or service teams as clusters, a design effect of 1.4 is conservatively assumed, producing approximately 2,152 completed observations. With an expected response rate of 70%, the team contacts about 3,075 accounts. Nonresponse patterns are monitored, and calibration weights align the sample with known regional and tenure distributions.

The team reports the weighted estimate, its design-based standard error, response rate, weighting procedure and a sensitivity analysis under alternative nonresponse assumptions. A bootstrap procedure resamples primary clusters, not individual customers, to reflect the design.

Outcome and learning points

The resulting estimate is 11.8%, with a 95% interval of approximately 10.2% to 13.4%. The interval is narrower and more credible than the voluntary poll’s informal estimate because selection, coverage and uncertainty were explicitly addressed. The case demonstrates that sample size alone cannot repair a defective design; representativeness and dependence matter equally.

11. Limitations, Governance and Best Practices

Confidence intervals quantify sampling uncertainty, not all uncertainty. They do not automatically account for coverage error, nonresponse bias, measurement error, model misspecification, data leakage or adaptive business processes. Analysts should distinguish a statistical margin of error from a broader uncertainty assessment.

Ethically, organizations should avoid presenting a precise estimate when the frame excludes vulnerable groups or when response is strongly selective. Survey participation, privacy, consent and secure handling of customer records require governance controls. Weighting variables should be documented, and sensitive subgroup estimates should be suppressed or carefully qualified when sample sizes are small.

Recommended practice includes: define the estimand before analysis; use probability sampling where generalization is required; preserve a reproducible sampling record; inspect distributions and outliers; use design-appropriate standard errors; report point estimates with intervals; conduct sensitivity analyses; pre-specify important decisions; and explain assumptions in language decision-makers can understand.

Modern statistical software makes bootstrap simulation, survey weighting and automated sample-size analysis accessible. Automation does not remove the need to understand the estimand, dependence structure or data-generating process. Artificial intelligence may assist with frame construction, response prediction and simulation, but it can also amplify historical coverage bias. Human review, audit trails and validation against known population totals remain essential.

Chapter Summary

Sampling distributions describe the repeated-sample behaviour of statistics and form the basis of statistical inference. Probability sampling supports defensible generalization, while non-probability designs require explicit caution about selection and coverage bias. The central limit theorem explains why sample means and many other statistics can often be approximated by normal distributions, but its conditions—including independence and adequate effective sample size—must be assessed.

Standard errors measure estimator variability; they are not the same as the standard deviation of individual observations. Point estimators should be evaluated for bias, variance, consistency, efficiency and robustness. Confidence intervals combine an estimate with a critical value and standard error, but their interpretation concerns long-run procedure performance. Bootstrap methods provide flexible uncertainty estimates when analytical methods are difficult, provided resampling respects dependence and the sampling design. Sample-size determination requires assumptions about variability, confidence, precision, response, design effects and analytic objectives. Reliable business inference depends as much on design and governance as on formulas.

Key Terms

Population: Complete set of units relevant to an investigation.

Target population: Population to which conclusions are intended to apply.

Sampling frame: Operational list from which a sample is selected.

Sample: Observed subset of a population.

Parameter: Fixed population quantity, such as μ or p.

Statistic: Quantity calculated from sample data.

Estimator: Rule used to estimate a parameter.

Sampling distribution: Distribution of a statistic over repeated samples.

Standard error: Standard deviation of an estimator’s sampling distribution.

Central limit theorem: Result describing approximate normality of standardized sample means under suitable conditions.

Bias: Systematic difference between an estimator’s expected value and the parameter.

Consistency: Convergence of an estimator to its target as sample size increases.

Confidence interval: Interval produced by a procedure with specified long-run coverage.

Bootstrap: Resampling method used to approximate a sampling distribution.

Stratified sampling: Sampling separately within defined population strata.

Cluster sampling: Sampling groups of population units rather than individuals directly.

Design effect: Variance inflation caused by a complex sampling design.

Margin of error: Half-width of an interval under a specified procedure.

Nonresponse bias: Bias caused by systematic differences between respondents and nonrespondents.

Finite population correction: Adjustment for sampling without replacement from a non-negligible fraction of a finite population.

Review Questions

Short Answer Questions

1. Distinguish a parameter from a statistic and give one business example of each.
2. What does the central limit theorem say about the sample mean?
3. Why can a large sample still produce a biased estimate?
4. Explain the difference between standard deviation and standard error.
5. What is the purpose of the finite population correction?

Long Answer Questions

1. Compare simple random, stratified, cluster and systematic sampling, including their advantages and risks.
2. Explain the construction and interpretation of a t-based confidence interval for a population mean.
3. Discuss the assumptions and limitations of the normal approximation for a sample proportion.
4. Describe the bootstrap algorithm and explain when ordinary resampling is inappropriate.
5. Explain how response rate, design effect and subgroup analysis influence sample-size planning.

Analytical / Application Questions

1. A sample of 100 customers has mean monthly spending $75 and standard deviation $20. Construct an approximate 95% confidence interval for the population mean using a t critical value of 1.984, and interpret it correctly.
2. A company wants to estimate a proportion with 95% confidence and margin of error 0.025, with no prior estimate of p. Determine the required simple-random sample size and explain why p = 0.5 is used.
3. A survey samples 40 branches and 25 customers per branch. The intrabranch correlation is estimated as 0.06. Calculate the approximate design effect and explain its consequence for effective sample size.

Multiple Choice Questions with Answers

Question 1

Which statement best defines a sampling distribution?

A. The distribution of all individual observations in a population

B. The distribution of a statistic across repeated samples

C. The distribution of errors in a regression model only

D. The distribution of population parameters across firms

Correct Answer: B. The distribution of a statistic across repeated samples

Question 2

A sample mean is calculated from n independent observations with population standard deviation σ. Its standard error is:

A. σ × √n

B. σ ÷ n

C. σ ÷ √n

D. σ² ÷ n

Correct Answer: C. σ ÷ √n

Question 3

What is the principal purpose of stratified sampling?

A. To ensure that every selected unit comes from one cluster

B. To sample separately from defined subgroups, potentially improving representation and precision

C. To eliminate all nonresponse bias

D. To avoid the need for a sampling frame

Correct Answer: B. To sample separately from defined subgroups, potentially improving representation and precision

Question 4

A 95% confidence interval means that:

A. 95% of individual observations lie in the interval

B. There is a 95% probability that the fixed parameter is random and lies in this interval

C. The procedure used captures the fixed parameter in about 95% of repeated samples

D. The sample statistic is correct with probability 0.95

Correct Answer: C. The procedure used captures the fixed parameter in about 95% of repeated samples

Question 5

For a 95% proportion estimate with no prior value of p, which planning value is conservative?

A. p = 0.01

B. p = 0.10

C. p = 0.50

D. p = 0.99

Correct Answer: C. p = 0.50

Question 6

A bootstrap sample is ordinarily generated by:

A. Sampling from the population without replacement using a smaller sample size

B. Resampling the observed data with replacement

C. Replacing all observations by their population means

D. Drawing only observations above the sample median

Correct Answer: B. Resampling the observed data with replacement

Question 7

If sample size increases from 100 to 400 while σ remains constant, the standard error of a sample mean approximately:

A. Doubles

B. Is halved

C. Is quartered

D. Does not change

Correct Answer: B. Is halved

Question 8

Which issue most directly causes ordinary individual-level bootstrap resampling to be misleading for clustered data?

A. It preserves too much cluster dependence

B. It may treat correlated observations as independent

C. It always produces a biased point estimate

D. It cannot calculate a sample mean

Correct Answer: B. It may treat correlated observations as independent

Question 9

For a mean with known σ, desired margin of error E and confidence multiplier z, the required sample size is:

A. zσE

B. (zσ/E)²

C. zE/σ²

D. σ²/(zE)

Correct Answer: B. (zσ/E)²

Question 10

If average cluster size is m and intracluster correlation is ρ, a common design-effect approximation is:

A. 1 − (m − 1)ρ

B. mρ only

C. 1 + (m − 1)ρ

D. ρ ÷ m

Correct Answer: C. 1 + (m − 1)ρ