Statistical Estimation and Hypothesis Testing

Introduction

Management decisions are usually made with incomplete information. A firm may wish to estimate average customer lifetime value, determine whether a training programme improves productivity, compare two suppliers, or assess whether a new pricing policy changes demand. In each situation, a sample is observed, while the managerial decision concerns a larger population or a future process.

Statistical inference provides a disciplined way to move from sample evidence to population conclusions. Two central activities are estimation, which quantifies plausible values of unknown population parameters, and hypothesis testing, which evaluates claims about those parameters. Sound inference requires more than obtaining a small p-value. Managers must also consider effect size, statistical power, uncertainty, data quality, assumptions, costs of error and practical significance.

This chapter develops confidence intervals, parametric tests, non-parametric tests and the interpretation of significance. It also addresses common inferential errors, reproducible analytical practice and the responsible use of statistical software and artificial intelligence.

Learning Objectives

By the end of this chapter, the reader should be able to:

1. Distinguish populations, samples, parameters and statistics.
2. Construct and interpret confidence intervals for means, proportions and differences.
3. Formulate null and alternative hypotheses and select suitable tests.
4. Apply and interpret one-sample, two-sample, paired, correlation, regression and chi-square procedures.
5. Use non-parametric methods when measurement or distributional assumptions are unsuitable.
6. Interpret p-values, effect sizes, confidence intervals and statistical power together.
7. Identify inferential errors and translate results into managerial recommendations.

1. Foundations of Statistical Inference

A population is the complete set of units of interest, such as all customers who purchased a product in a year. A sample is a subset observed from that population. A parameter is a numerical characteristic of the population, such as the population mean μ or proportion p. A statistic is calculated from the sample, such as the sample mean x̄ or sample proportion p̂.

A statistic is used to estimate a parameter. An estimator is desirable when it is approximately unbiased, consistent and efficient. An unbiased estimator has an expected value equal to the parameter. Consistency means that the estimator approaches the parameter as sample size increases. Efficiency concerns the estimator’s variance relative to competing unbiased estimators.

The standard error measures sampling variability. For a sample mean from a population with standard deviation σ:

where n is sample size. When σ is unknown, it is estimated by the sample standard deviation s:

The formula assumes independent observations and is most reliable when the population is normal or the sample is sufficiently large for the central limit theorem to apply. Increasing n reduces the standard error at a rate proportional to 1 ÷ √n; therefore, quadrupling the sample size approximately halves sampling error.

A sampling distribution describes how a statistic would vary across repeated random samples. This concept is essential: inferential procedures quantify uncertainty arising from sampling, not all possible sources of uncertainty such as measurement error, non-response or model misspecification.

2. Confidence Intervals

A point estimate gives one value, but a confidence interval provides a range of plausible parameter values. Its general form is:

A 95% confidence interval is constructed by a procedure that captures the true parameter in approximately 95% of repeated samples, assuming the model and sampling process are appropriate. It is not correct to say that there is a 95% probability that a fixed parameter lies inside a particular interval in the classical frequentist interpretation.

2.1 Confidence interval for a population mean

When σ is known, or when a normal approximation is justified, a confidence interval for μ is:

When σ is unknown, the usual interval is:

Here α is the total significance level, z is a standard normal critical value, t is a Student’s t critical value and n − 1 is the degrees of freedom. For a 95% interval, α = 0.05. The t distribution has heavier tails than the normal distribution, particularly for small samples, reflecting uncertainty in estimating σ.

Worked Example 1: Mean delivery time

A random sample of 25 deliveries has mean delivery time 42 minutes and standard deviation 8 minutes. Assuming independent observations and a reasonably normal population, a 95% confidence interval is based on t_(0.025,24) ≈ 2.064.

Thus:

The interval suggests that the population mean delivery time is plausibly between 38.70 and 45.30 minutes under the stated assumptions. A manager should not conclude that every delivery falls in this range; the interval concerns the population mean, not individual delivery times.

2.2 Confidence interval for a population proportion

For a sufficiently large sample, an approximate interval for a population proportion p is:

The approximation is generally assessed using expected counts such as n p̂ and n(1 − p̂), both preferably at least 10. For small samples or extreme proportions, exact or adjusted methods may be preferable.

2.3 Confidence interval for a difference

For two independent means, a confidence interval for μ₁ − μ₂ is based on the estimated difference x̄₁ − x̄₂ and its standard error. When equal population variances are not credible, Welch’s method is usually safer than the pooled-variance method. For paired observations, calculate within-pair differences dᵢ and construct an interval for the mean difference μ_d:

Pairing is appropriate when the same subjects are measured before and after an intervention or when matched units are deliberately compared.

2.4 Factors affecting interval width

An interval becomes wider when confidence level increases, variability increases or sample size decreases. Precision and confidence are distinct: a large sample can produce a narrow interval, while a small sample may produce a wide interval even at the same confidence level. A narrow interval is not automatically accurate if the sample is biased or the measurement process is defective.

3. Hypothesis Testing

Hypothesis testing begins with a claim about a population parameter. The null hypothesis, H₀, usually represents no difference, no association or a specified benchmark. The alternative hypothesis, H₁ or Hₐ, represents the effect or relationship of interest.

For example, a company may test:

A two-sided alternative is used when deviations in either direction matter. A one-sided alternative, such as Hₐ: μ > 50, should be selected before observing the data and only when a directional decision is substantively justified.

The test statistic measures how far the sample result is from the null value relative to its standard error. For a one-sample mean with unknown σ:

where μ₀ is the null-hypothesized mean. Under H₀ and the relevant assumptions, this statistic follows a t distribution with n − 1 degrees of freedom.

The p-value is the probability, assuming H₀ is true, of observing a result at least as extreme as the one obtained, according to the selected alternative. It is not the probability that H₀ is true, nor the probability that the result occurred by chance in an informal sense.

A decision rule compares the p-value with α, commonly 0.05. If p ≤ α, H₀ is rejected; otherwise, H₀ is not rejected. “Fail to reject” does not prove H₀. It indicates that the evidence is insufficient under the selected design and model.

3.1 Type I and Type II errors

A Type I error rejects a true H₀. Its long-run probability is controlled by α under the model. A Type II error fails to reject a false H₀. Its probability is β, and statistical power is:

Power depends on sample size, variability, α, the true effect size and whether the test is one- or two-sided. A statistically non-significant result may reflect a genuinely negligible effect or inadequate power. A statistically significant result may be too small to matter operationally.

3.2 Relationship between confidence intervals and tests

For a two-sided test at α = 0.05, a null value is rejected if and only if it lies outside the corresponding 95% confidence interval, provided the same model and method are used. This relationship helps managers see both the decision and the plausible magnitude of the effect.

4. Parametric Tests and Models

Parametric procedures make assumptions about population parameters or distributions. Their strength is often statistical efficiency when assumptions are reasonable.

4.1 One-sample t-test

The one-sample t-test evaluates whether a population mean differs from a benchmark. Important assumptions are independent observations, a quantitative outcome and approximate normality of the population for small samples. Strong outliers can materially distort the mean and test statistic.

4.2 Independent two-sample tests

An independent two-sample t-test compares means from two unrelated groups. Welch’s t-test does not require equal variances and is generally the default choice when group variances or sample sizes differ. Independence must concern the observations, not merely the group labels.

A statistically significant difference in group means does not establish that group membership caused the difference. Causal interpretation requires an appropriate experimental or quasi-experimental design and control of confounding variables.

4.3 Paired t-test

The paired t-test compares two measurements for each unit. It tests whether the mean difference differs from zero. Pairing can reduce noise caused by stable differences between units, but it does not correct for trends, regression to the mean or lack of a control group.

4.4 Analysis of variance

One-way analysis of variance (ANOVA) tests whether k group means are all equal:

The F statistic is:

where MS_between measures variation among group means and MS_within measures variation within groups. Under H₀ and suitable assumptions, F follows an F distribution with k − 1 and N − k degrees of freedom.

ANOVA assumptions include independent observations, approximately normal errors within groups and homogeneous variances. A significant omnibus test indicates that at least one mean differs; it does not identify which groups differ. Planned contrasts or multiplicity-adjusted post-hoc comparisons are required. Repeatedly inspecting pairwise tests without adjustment inflates the familywise Type I error rate.

4.5 Correlation and regression

Pearson’s correlation coefficient measures linear association:

Its value lies between −1 and +1. Correlation is sensitive to outliers and does not imply causation.

In simple linear regression:

where yᵢ is the outcome, xᵢ is the predictor, β₀ is the intercept, β₁ is the expected change in y for a one-unit increase in x, and εᵢ is the error term. Inference about β₁ commonly assumes independent errors, linearity, constant error variance and approximately normal errors for small-sample tests. Time-series observations may violate independence and require specialised methods.

The coefficient of determination, R², describes the proportion of sample variation in y explained by the fitted linear model; it is not a causal measure and does not by itself establish managerial usefulness.

5. Non-Parametric Methods

Non-parametric procedures are useful when outcomes are ordinal, distributions are strongly skewed, samples are small, or outliers make mean-based procedures inappropriate. They often test distributions, ranks or medians rather than means. Their assumptions are reduced, not eliminated.

The Mann–Whitney test evaluates whether observations from one group tend to be larger than observations from another. It should not automatically be described as a test of medians unless the distributions have comparable shapes and spread. The Wilcoxon signed-rank test uses the ranks of non-zero paired differences and assumes a suitably symmetric distribution of differences for its common interpretation.

The Kruskal–Wallis test is an omnibus test for k independent groups. A significant result requires follow-up comparisons with appropriate multiplicity control. The Friedman test is designed for related or repeated observations across conditions.

5.1 Chi-square tests

For a contingency table, the Pearson chi-square statistic is:

where Oᵢ is an observed cell count and Eᵢ is the expected count under H₀. For a test of independence, expected counts are:

The test assumes independent cases and sufficiently large expected counts for the chi-square approximation. Sparse tables may require exact tests or category consolidation, provided the consolidation is substantively defensible and not data-driven.

6. Effect Size, Power and Managerial Relevance

Statistical significance depends strongly on sample size. Effect size describes the magnitude of a difference or relationship independently, as far as possible, from sample size.

For a one-group mean difference, Cohen’s d is often written:

For two independent groups, a standardised mean difference may use a pooled standard deviation when equal-variance assumptions are defensible. For paired data, the standard deviation of the paired differences is relevant. Interpretation of “small,” “medium” and “large” benchmarks should be contextual rather than mechanical.

For a two-group proportion comparison, report the absolute difference, relative risk or odds ratio where appropriate. For ANOVA, eta-squared or partial eta-squared can describe variance associated with group effects, but definitions and denominators must be stated. For chi-square analysis, Cramér’s V is commonly used:

where N is the total sample size and r and c are the numbers of rows and columns.

A useful managerial report combines:

• the estimated effect;
• a confidence interval;
• the p-value and test direction;
• an effect-size measure;
• the sample and study design;
• assumptions and limitations; and
• a comparison with a business-relevant threshold.

A minimum practically important difference (MPID) is the smallest effect judged worthwhile after considering revenue, cost, risk, customer experience or regulatory consequences. A result can be statistically significant but below the MPID. Conversely, a large estimated effect with a wide interval may justify further data collection even if p > 0.05.

Power analysis should normally be conducted before data collection. A simplified planning relationship for a one-sample mean, when σ and a target difference δ are specified, is that larger σ, higher desired power and smaller δ require larger n. Exact calculations depend on the test, α, alternative, allocation and assumptions. Post hoc claims that a non-significant result “proves no effect” are especially unsafe; confidence intervals and equivalence or non-inferiority tests may be more appropriate.

7. A Disciplined Testing Workflow

A robust workflow is as follows:

1. Define the population, decision and estimand before analysing data.
2. Specify the outcome, predictors, comparison, direction and practical threshold.
3. Inspect sampling, missingness, measurement quality and possible dependence.
4. Visualise distributions and identify outliers without automatically deleting them.
5. State H₀, Hₐ, α and the analysis plan before viewing inferential results.
6. Select a method whose assumptions and scale match the research question.
7. Calculate an estimate, confidence interval, test statistic and p-value.
8. Assess effect size, power, robustness and practical importance.
9. Adjust for multiplicity when many outcomes, groups or models are examined.
10. Report limitations and translate the result into a conditional managerial action.

Statistical software can calculate results quickly, but it cannot decide whether the sample is representative, whether a variable was measured validly or whether a statistically detectable effect matters. AI-assisted analysis should therefore be governed by documented data provenance, reproducible code or syntax, versioned outputs, human review and protection of confidential information. AI-generated interpretations must be independently checked, especially for causal claims and assumption violations.

8. Illustrative Case: Evaluating a Customer-Service Training Programme

Background

A retail bank introduces a service-training programme. Management wants to know whether average customer waiting time has fallen below the operational benchmark of 12 minutes. A random sample of 36 post-training service episodes has mean waiting time 10.8 minutes and standard deviation 3.6 minutes. The bank defines a reduction of at least 1 minute as operationally meaningful.

Problem and analysis plan

The estimand is the post-training population mean waiting time. The hypotheses are:

A one-sided test is justified because only reduction below the benchmark supports the proposed operational decision, and the direction was specified before analysis. The sample is treated as independent and approximately representative. A one-sample t-test is used.

Calculation

Degrees of freedom are 35. The one-sided p-value is approximately 0.026. At α = 0.05, the null hypothesis is rejected. A 95% two-sided confidence interval is based on t_(0.025,35) ≈ 2.030:

Thus, the interval is approximately (9.58, 12.02) minutes. The observed reduction is 1.2 minutes. A standardised effect estimate is:

This is a modest standardised effect. Although the one-sided test provides evidence of a reduction, the confidence interval includes values close to the 12-minute benchmark and does not establish that the reduction exceeds the 1-minute MPID with high confidence.

Managerial interpretation and outcome

The evidence supports treating the programme as promising, but not as conclusively meeting the business target. Management should continue implementation while collecting a larger, representative sample and preferably comparing trained branches with similar untrained branches over the same period. The comparison would help distinguish training effects from seasonal demand, staffing changes or regression to the mean.

Learning points

The case illustrates why a p-value alone is inadequate. The direction of the test must be justified in advance; the confidence interval expresses uncertainty; the effect size describes magnitude; and the MPID determines whether the result is useful. A statistically significant benchmark comparison is not automatically evidence of causation or sufficient evidence for a large operational investment.

9. Common Inferential Errors

Confusing significance with importance. A tiny effect can have p < 0.05 in a very large sample. Always report magnitude and context.

Interpreting p as the probability that H₀ is true. The p-value is conditional on H₀ and the model; it does not reverse that conditional probability.

Treating non-significance as proof of no effect. Examine power and the confidence interval. Equivalence testing may be needed to support a claim of practical similarity.

Using a one-sided test after seeing the data. This can artificially reduce the p-value and should be avoided.

Ignoring dependence. Repeated observations from the same customer, branch or employee are not independent simply because they appear in separate rows.

Running many tests without adjustment. Multiple comparisons increase the chance of false positives. Pre-specification, familywise control or false-discovery-rate procedures may be appropriate.

Deleting outliers automatically. Investigate whether an outlier is an error, a valid extreme case or evidence that the model is inadequate.

Assuming normality is always required. For means, the normality of the raw data is less critical with large independent samples, but dependence, skewness and influential observations remain important.

Equating correlation with causation. Confounding, reverse causality and selection effects can produce associations without causal impact.

Reporting excessive precision. Rounding should reflect measurement quality and uncertainty; a p-value such as 0.000 should be reported as p < 0.001, not as an impossible probability of zero.

10. Governance and Ethical Considerations

Inferential decisions can affect hiring, credit, pricing, performance evaluation and access to services. Analysts should document the sampling frame, exclusions, missing-data treatment, variable definitions, test choices and deviations from the original plan. Selective reporting, p-hacking and undisclosed subgroup searches undermine trust.

Fairness requires examining whether aggregate results conceal materially different effects across relevant groups. Subgroup analysis should be substantively motivated and interpreted cautiously because small subgroup samples reduce precision and repeated analysis increases false discoveries. Sensitive data must be handled according to applicable privacy, data-protection and sectoral requirements. Statistical evidence is one input to a decision, not a substitute for ethical judgment or legal compliance.

Chapter Summary

Statistical estimation uses sample information to quantify plausible values of population parameters, while hypothesis testing evaluates claims under an explicit null model. Confidence intervals communicate both point estimates and uncertainty. t-tests, ANOVA, regression and chi-square procedures are useful when their design and assumptions are appropriate; non-parametric methods provide alternatives for ordinal, skewed or rank-based data but still require careful interpretation.

A p-value measures compatibility of the observed data with H₀ under specified assumptions. It does not measure the probability that H₀ is true, the size of an effect or its managerial value. Responsible inference combines estimates, confidence intervals, p-values, effect sizes, power and practical thresholds. Sound practice also addresses sampling bias, dependence, multiplicity, outliers, causal limitations, ethical governance and reproducibility.

Key Terms

Population: Complete set of units about which inference is intended.

Sample: Observed subset drawn from a population.

Parameter: Numerical characteristic of a population.

Statistic: Numerical quantity calculated from a sample.

Estimator: Rule used to estimate a population parameter.

Standard error: Estimated sampling variability of a statistic.

Confidence interval: Range produced by a procedure designed to capture a parameter at a stated long-run rate.

Null hypothesis: Default claim evaluated by a statistical test.

Alternative hypothesis: Claim representing a difference, relationship or directional effect.

p-value: Probability, assuming H₀ and the model, of data at least as extreme as observed.

Significance level: Preselected Type I error rate, denoted α.

Type I error: Rejection of a true null hypothesis.

Type II error: Failure to reject a false null hypothesis.

Statistical power: Probability of detecting a specified effect; 1 − β.

Effect size: Quantitative measure of the magnitude of an effect.

Parametric test: Procedure relying on specified distributional or parameter assumptions.

Non-parametric test: Procedure relying less directly on distributional parameters, often using ranks.

Multiplicity: Inflation of false-positive risk from conducting multiple analyses.

Practical significance: Importance of an effect for a real decision or operational objective.

Estimand: Precisely defined quantity a study seeks to estimate.

Review Questions

Short Answer Questions

1. Distinguish a parameter from a statistic and provide one managerial example of each.
2. What does a 95% confidence interval mean in repeated-sampling terms?
3. Explain the difference between a p-value and the probability that the null hypothesis is true.
4. State two conditions under which a non-parametric test may be preferred.
5. Why should effect size be reported alongside statistical significance?

Long Answer Questions

1. Explain the construction and interpretation of a confidence interval for a population mean when the population standard deviation is unknown.
2. Compare Type I error, Type II error and statistical power, including their managerial consequences.
3. Discuss the assumptions, interpretation and limitations of one-way ANOVA.
4. Compare independent t-tests, paired t-tests and Welch’s t-test in terms of design and assumptions.
5. Explain how confidence intervals, effect sizes and minimum practically important differences improve managerial interpretation.

Analytical / Application Questions

1. A manager observes a statistically significant 0.3-minute reduction in average processing time from a sample of 20,000 transactions. Explain why the result may still be operationally unimportant and identify additional evidence required.
2. A before-and-after evaluation uses the same 30 employees, but the organisation also changed its software during the study. Identify the appropriate basic test and discuss why causal interpretation remains limited.
3. A survey examines whether product preference is associated with age group using a sparse contingency table. Recommend an analytical approach, identify the relevant assumptions and explain how the result should be reported.

Multiple Choice Questions with Answers

Question 1

A 95% confidence interval is best interpreted as:

A. There is a 95% probability that the fixed population parameter lies in this specific interval.

B. The procedure used will capture the true parameter in approximately 95% of repeated samples.

C. 95% of individual observations lie inside the interval.

D. The null hypothesis has a 95% probability of being true.

Correct Answer: B. The procedure used will capture the true parameter in approximately 95% of repeated samples.

Question 2

A sample of 25 observations has a mean of 42 and standard deviation of 8. Using t = 2.064 for 24 degrees of freedom, the approximate 95% confidence interval for the mean is:

A. (38.70, 45.30)

B. (40.40, 43.60)

C. (37.94, 46.06)

D. (41.68, 42.32)

Correct Answer: A. (38.70, 45.30)

Question 3

A p-value is:

A. The probability that the null hypothesis is true.

B. The probability that the alternative hypothesis is true.

C. The probability, assuming the null hypothesis and model, of observing a result at least as extreme as the sample result.

D. The probability that the result has managerial importance.

Correct Answer: C. The probability, assuming the null hypothesis and model, of observing a result at least as extreme as the sample result.

Question 4

Which test is most appropriate for comparing two measurements taken from the same employees before and after training?

A. Independent two-sample t-test

B. Paired t-test

C. One-way ANOVA for unrelated groups

D. Pearson chi-square test

Correct Answer: B. Paired t-test

Question 5

For a one-way ANOVA, a significant omnibus F-test establishes that:

A. Every group mean differs from every other group mean.

B. At least one group mean differs, but further comparisons are needed to identify which ones.

C. The largest group has the highest population mean.

D. All group variances are equal in the population.

Correct Answer: B. At least one group mean differs, but further comparisons are needed to identify which ones.

Question 6

Which method is a common non-parametric alternative to the independent two-sample t-test?

A. Mann–Whitney U test

B. Friedman test

C. Paired t-test

D. One-sample z-test

Correct Answer: A. Mann–Whitney U test

Question 7

If β is the probability of a Type II error, statistical power equals:

A. α

B. β

C. 1 − α

D. 1 − β

Correct Answer: D. 1 − β

Question 8

A statistically significant result with a very small effect size indicates that:

A. The effect is necessarily important for management.

B. The result cannot be replicated.

C. The effect is detectable under the model but may not be practically important.

D. The sample must be biased.

Correct Answer: C. The effect is detectable under the model but may not be practically important.

Question 9

In a chi-square test of independence, the expected count for a cell is calculated as:

A. Cell count divided by sample size

B. Row total multiplied by column total, divided by grand total

C. Grand total divided by the number of cells

D. Row total plus column total

Correct Answer: B. Row total multiplied by column total, divided by grand total

Question 10

Which statement is correct about failing to reject the null hypothesis?

A. It proves that the null hypothesis is true.

B. It proves that the effect is zero.

C. It indicates that the evidence is insufficient to reject the null under the specified design and model.

D. It means the study had no measurement error.

Correct Answer: C. It indicates that the evidence is insufficient to reject the null under the specified design and model.