Statistical Estimation and Hypothesis Testing

Introduction

Business analytics rarely observes an entire population. A retailer may have millions of transactions, a bank may have thousands of customers, and a manufacturer may produce continuously. Analysts therefore use a sample to learn about an unknown population. Statistical estimation quantifies plausible values for population parameters, while hypothesis testing provides a structured method for deciding whether sample evidence is sufficiently inconsistent with a stated claim.

These methods support decisions such as whether a marketing campaign changed conversion rates, whether two branches have different average service times, or whether process variability has increased. They do not prove that a claim is true or false with certainty. Instead, they measure evidence under explicitly stated assumptions.

Learning Objectives

By the end of this chapter, you should be able to:

• distinguish point estimates from interval estimates;
• construct and interpret confidence intervals for means and proportions;
• perform one- and two-sample tests for means, proportions, and variances;
• explain null and alternative hypotheses, p-values, significance levels, and critical regions;
• distinguish Type I error, Type II error, statistical significance, and practical significance;
• explain statistical power and factors that affect it;
• select suitable procedures and communicate results responsibly.

1. The Estimation Framework

A population is the complete set of units of interest. A sample is the subset observed by the analyst. A parameter describes a population, whereas a statistic describes a sample.

A point estimate is a single value used to estimate a parameter. For example, x̄ estimates μ and p̂ estimates p. A point estimate alone does not show sampling uncertainty. An interval estimate gives a range of plausible parameter values together with a confidence level.

For a parameter θ, the general form of a confidence interval is:

The standard error measures the typical sampling variation of an estimator. The critical value reflects the desired confidence level and the relevant probability distribution.

Sampling distributions and the central limit theorem

A sampling distribution is the probability distribution of a statistic over repeated samples of the same size. For independent observations with population mean μ and standard deviation σ, the sample mean has:

where n is sample size. If σ is unknown, it is commonly estimated by s, giving SE(x̄) = s/√n. For sufficiently large samples, the central limit theorem states that the distribution of x̄ is approximately normal under broad conditions, even when individual observations are not normal. For small samples, normality of the underlying population is more important.

2. Confidence Intervals

A 95% confidence interval is constructed by a method that would capture the fixed population parameter in approximately 95% of repeated samples. It is not technically correct to say that there is a 95% probability that a fixed parameter lies in one already-calculated interval. The parameter is fixed; the interval-producing procedure is random.

In practical communication, analysts often say that they are 95% confident that the interval contains the parameter, provided the sampling design and model assumptions are appropriate.

2.1 Confidence interval for a population mean

When the population standard deviation σ is known, the z-based interval is:

Here, α = 1 − confidence level, z_(α/2) is the standard normal critical value, and n is sample size. This procedure assumes independent observations and a known σ, conditions that are uncommon in routine business data.

When σ is unknown, use the t interval:

where t_(α/2, n−1) is the Student’s t critical value with n − 1 degrees of freedom. This is appropriate for independent observations from a normal population, especially when n is small. For larger samples, moderate departures from normality are often tolerable, but extreme skewness and outliers remain problematic.

Worked example. A sample of 25 delivery times has x̄ = 42 minutes and s = 8 minutes. A 95% interval uses t_(0.025,24) ≈ 2.064.

The analyst can report: “Using this procedure, we estimate the population mean delivery time to be between 38.70 and 45.30 minutes at the 95% confidence level.” The interval’s margin of error is 3.30 minutes.

2.2 Confidence interval for a population proportion

For a sample proportion p̂ = x/n, the large-sample Wald interval is:

where x is the number of successes and n is the sample size. This approximation is most defensible when both np̂ and n(1 − p̂) are sufficiently large, commonly at least 10. For small counts or proportions near zero or one, Wilson or exact methods are preferable.

Worked example. In a survey, 180 of 400 customers recommend a service. Thus p̂ = 0.45. A 95% approximate interval is:

The estimated recommendation rate is therefore approximately 40.1% to 49.9% using the stated approximation.

2.3 Confidence interval for a variance

If a random sample comes from a normal population, the statistic involving sample variance follows a chi-square distribution:

A two-sided 100(1 − α)% confidence interval for σ² is:

The order of the chi-square quantiles is reversed because the variance appears in the denominator. This interval is sensitive to non-normality; it should not be used mechanically for heavily skewed data.

2.4 What controls interval width?

An interval becomes narrower when sample size increases, variability decreases, or the confidence level decreases. For a mean, the standard error decreases at the rate 1/√n. Consequently, quadrupling sample size approximately halves the margin of error. Greater confidence requires a larger critical value and therefore a wider interval.

3. Hypothesis Testing

Hypothesis testing begins with a claim about a population parameter. The null hypothesis, H₀, usually represents a status quo, benchmark, or no-effect condition. The alternative hypothesis, H₁ or Hₐ, represents the effect or difference being investigated.

Examples include:

• H₀: μ = 30 versus Hₐ: μ ≠ 30 (two-sided test);
• H₀: p = 0.20 versus Hₐ: p > 0.20 (right-tailed test);
• H₀: μ₁ − μ₂ = 0 versus Hₐ: μ₁ − μ₂ ≠ 0.

The null hypothesis is not normally “accepted.” The appropriate conclusions are to reject H₀ or fail to reject H₀. Failure to reject does not establish that H₀ is true; it indicates that the sample has not supplied sufficient evidence against it.

A disciplined testing procedure

1. Define the population, parameter, and practical question.
2. State H₀ and Hₐ before examining results.
3. Select α, commonly 0.05, and identify a suitable test statistic.
4. Check independence, measurement, distributional, and sample-size assumptions.
5. Calculate the test statistic and p-value, or compare with a critical value.
6. Make a statistical decision.
7. Translate the result into business or operational language, including effect size and uncertainty.

Test statistics and p-values

A test statistic measures how far the sample result is from the null value in standard-error units. For a mean with unknown σ:

where μ₀ is the hypothesized mean and the statistic follows a t distribution with n − 1 degrees of freedom under H₀, assuming the model conditions hold.

A p-value is the probability, assuming H₀ is true, of obtaining a result at least as inconsistent with H₀ as the observed result, in the direction specified by Hₐ. It is not the probability that H₀ is true, nor the probability that the result occurred by chance in an informal sense.

If p ≤ α, reject H₀. If p > α, fail to reject H₀. The threshold α must be selected before seeing the data; changing it after viewing the p-value undermines the analysis.

3.1 One-sample test for a mean

For a normal population with unknown σ, use the one-sample t test:

Example. A call centre claims that the average waiting time is 6 minutes. A random sample of 16 calls gives x̄ = 6.8 and s = 1.6. Test H₀: μ = 6 against Hₐ: μ ≠ 6 at α = 0.05.

With df = 15, the two-sided critical value is approximately ±2.131. Since |2.00| < 2.131, fail to reject H₀. The sample does not provide sufficient evidence at the 5% level that the mean waiting time differs from 6 minutes. This does not demonstrate that the true mean is exactly 6 minutes.

3.2 One-sample test for a proportion

For a hypothesized proportion p₀, the large-sample z test is:

The null value p₀, rather than p̂, is used in the standard error because the distribution is evaluated under H₀. Large-sample conditions should be checked. Exact binomial methods are appropriate when counts are small.

3.3 Two-sample tests for means

For two independent groups, the parameter is μ₁ − μ₂. The preferred general procedure is Welch’s t test, which does not require equal population variances:

Here Δ₀ is the hypothesized difference, often zero. The degrees of freedom are approximated by the Welch–Satterthwaite formula:

A paired t test should be used instead when each observation in one condition is naturally matched with an observation in the other condition, such as the same employee’s productivity before and after training. Define differences dᵢ, then test the mean difference using a one-sample t test.

Example. Branch A has n₁ = 40, x̄₁ = 12.4, s₁ = 2.0; Branch B has n₂ = 35, x̄₂ = 11.5, s₂ = 2.4. The estimated difference is 0.9 units. Its standard error is:

Thus t ≈ 0.9/0.527 = 1.71. Whether this is statistically significant depends on the Welch degrees of freedom and the selected alternative; at the conventional 5% two-sided level it would generally not be significant. The confidence interval for μ₁ − μ₂ should also be reported because it shows plausible effect sizes.

3.4 Two-sample tests for proportions

For two independent proportions, test H₀: p₁ − p₂ = 0 using the pooled estimate:

and

The pooled estimate is used for the null test. For estimation, a confidence interval for p₁ − p₂ generally uses an unpooled standard error. Small counts require exact or improved interval/test methods.

3.5 Tests for variances

For a normal population, testing H₀: σ² = σ₀² uses:

with df = n − 1. For two independent normal populations, the ratio test is:

with df₁ = n₁ − 1 and df₂ = n₂ − 1 under H₀: σ₁² = σ₂². These tests are highly sensitive to non-normality. In many practical analytics settings, robust methods or graphical checks are safer than relying solely on an F test.

4. Errors, Power, and Practical Significance

A hypothesis decision can be wrong in two principal ways.

A Type I error is a false positive: concluding that an effect exists when it does not. A Type II error is a false negative: failing to detect a real effect. The power of a test is 1 − β, the probability of rejecting H₀ for a specified alternative.

Power increases with larger sample size, larger true effect, lower variability, and a higher α. One-sided tests can have more power in the specified direction, but only when the direction is justified in advance. Power decreases when the minimum effect of interest is small relative to noise.

Statistical significance and practical significance are different. With a very large sample, a trivial improvement may produce p < 0.001. With a small sample, a commercially important effect may fail to reach p < 0.05. Analysts should report the estimated effect, confidence interval, costs, risks, and operational relevance, not only a p-value.

Useful effect measures include a mean difference, percentage-point difference, relative risk, odds ratio, or standardized mean difference. For two means, Cohen’s d is commonly defined as:

where s_p is a pooled standard deviation under the equal-variance convention. The measure is dimensionless, but it should be interpreted with domain context rather than rigid universal cut-offs.

5. Choosing a Procedure and Avoiding Common Errors

The choice of test depends on the outcome type, design, parameter, independence, sample size, and distributional assumptions.

Common errors include treating a confidence level as a probability about a fixed parameter, interpreting p as the probability that H₀ is true, testing many outcomes without adjustment, ignoring dependence, using a t test for strongly skewed small samples, and reporting significance without effect size.

Multiple testing deserves particular caution. If many independent hypotheses are each tested at α = 0.05, the probability of at least one false positive can exceed 5%. Analysts may pre-specify a primary outcome, control the family-wise error rate, or control the false discovery rate, depending on the purpose of the study.

6. Applied Case: Evaluating a Digital Checkout Redesign

Background. A fictional online retailer redesigns its checkout page. Management wants to know whether the redesign increases completed purchases and whether the improvement is large enough to justify implementation costs.

Problem. Before the redesign, the conversion rate was 18%. After launch, 1,000 randomly selected eligible sessions produce 205 purchases. The business considers an increase of at least 3 percentage points operationally worthwhile.

Analysis. The sample proportion is p̂ = 205/1,000 = 0.205. For the statistical test H₀: p = 0.18 against Hₐ: p > 0.18:

Calculation gives a standard error of approximately 0.01215 and z ≈ 2.06. The one-sided p-value is approximately 0.020. At α = 0.05, the evidence supports rejecting H₀ in favour of an increase.

For estimation, an approximate 95% interval is:

The estimated increase is 2.5 percentage points. Although the result is statistically significant, the interval includes increases below the 3-point operational threshold. Management should therefore consider implementation cost, uncertainty, seasonality, device mix, and whether the sessions were randomly assigned. A randomized A/B design would provide stronger causal evidence than a simple before–after comparison.

Outcome and learning points. The redesign provides evidence of improvement, but statistical significance alone does not prove that it meets the business target. The decision should combine a confidence interval, a pre-defined practical threshold, experimental design, and cost–benefit analysis. Analysts should also monitor performance after rollout because effects can change as customers adapt.

7. Technology, Governance, and Ethical Practice

Statistical software can calculate confidence intervals, test statistics, p-values, bootstrap intervals, and power analyses rapidly. However, software does not determine whether the sample is representative, whether observations are independent, or whether the chosen alternative reflects the decision problem. Analysts should document data preparation, inclusion criteria, missing-data treatment, test selection, assumptions, and any multiplicity adjustments.

Automated and AI-assisted analytics introduce additional risks. Repeatedly searching many model specifications until a desirable p-value appears is a form of analytical flexibility that inflates false-positive risk. A responsible workflow separates exploratory analysis from confirmatory testing, preserves analysis logs, uses preregistered hypotheses where appropriate, and validates important findings on new data.

Privacy and fairness also matter. Testing conversion or approval rates across demographic groups can reveal disparities, but small subgroup samples may yield unstable estimates. Analysts should protect personal information, avoid unnecessary identification, report uncertainty for each group, and investigate whether measurement or sampling processes disadvantage particular populations.

Best-Practice Checklist

• Define the target population and sampling unit.
• Match the method to the outcome and study design.
• State hypotheses and α before inspecting inferential results.
• Check independence, outliers, normality, and sample-size conditions.
• Report estimates, confidence intervals, p-values, and practical effect sizes.
• Distinguish failure to reject from proof of no effect.
• Consider power before collecting data.
• Address multiple comparisons and selective reporting.
• Communicate assumptions and limitations in plain language.
• Connect statistical evidence to costs, risks, and decisions.

Chapter Summary

Statistical estimation uses sample information to learn about unknown population parameters. Point estimates provide single values, while confidence intervals quantify uncertainty through an estimate, standard error, and critical value. Means commonly use t intervals when population variability is unknown; proportions use binomial or large-sample procedures; and variance intervals rely on the chi-square distribution under normality.

Hypothesis testing compares observed evidence with a null model. Correct analysis requires clearly stated hypotheses, an appropriate test statistic, a preselected significance level, assumption checks, and careful interpretation of the p-value. One- and two-sample procedures differ according to the outcome and whether observations are independent or paired. Type I errors are false positives, Type II errors are false negatives, and power is the probability of detecting a specified real effect. Statistical significance must be distinguished from practical importance. Sound analytics combines uncertainty, effect size, study design, ethical governance, and business context.

Key Terms

Parameter: A numerical characteristic of a population.

Statistic: A numerical characteristic calculated from a sample.

Point estimate: A single sample-based estimate of a population parameter.

Confidence interval: An interval produced by a procedure designed to capture a parameter at a stated long-run confidence rate.

Standard error: The standard deviation of an estimator’s sampling distribution.

Null hypothesis: A benchmark claim evaluated by a statistical test.

Alternative hypothesis: The claim supported when evidence against the null is sufficiently strong.

Significance level: The chosen maximum probability of a Type I error, denoted α.

p-value: The probability, under H₀, of evidence at least as extreme as that observed.

Critical value: A distribution-based boundary separating rejection and non-rejection regions.

Type I error: Rejection of a true null hypothesis.

Type II error: Failure to reject a false null hypothesis.

Power: The probability of rejecting H₀ for a specified alternative; 1 − β.

Welch’s t test: A two-sample mean test that does not assume equal population variances.

Paired test: A test based on differences between naturally matched observations.

Chi-square distribution: A distribution used in inference about normal-population variances.

Statistical significance: Evidence judged sufficiently inconsistent with H₀ under a chosen α.

Practical significance: The operational, financial, or substantive importance of an effect.

Multiple testing: Conducting several hypothesis tests, which can increase false-positive risk.

Review Questions

5 Short Answer Questions

1. Distinguish a parameter from a statistic.
2. What does a 95% confidence procedure mean in repeated sampling?
3. Why is “fail to reject H₀” preferable to “accept H₀”?
4. Define a p-value without describing it as the probability that H₀ is true.
5. State two factors that increase statistical power.

5 Long Answer Questions

1. Explain the construction and interpretation of a t confidence interval for a population mean, including its assumptions.
2. Compare one-sample, independent two-sample, and paired t tests.
3. Explain Type I error, Type II error, power, and their relationship to α.
4. Discuss why statistical significance and practical significance may lead to different conclusions.
5. Explain the assumptions and limitations of chi-square and F tests for variances.

3 Analytical / Application Questions

1. A sample of 36 service times has mean 14.2 minutes and standard deviation 3.6 minutes. Construct and interpret an approximate 95% confidence interval using the appropriate t critical value.
2. A campaign produces 145 purchases from 800 visitors, compared with a historical rate of 15%. Formulate a one-sided test, calculate the test statistic, and explain what additional information is needed before making a managerial recommendation.
3. Two independent teams have different average processing times. Design an analysis plan that addresses sampling, independence, choice between Welch and paired procedures, effect size, confidence intervals, power, and practical relevance.

Multiple Choice Questions with Answers

Question 1

Which quantity is a point estimate of the population mean μ?

A. σ

B. x̄

C. α

D. β

Correct Answer: B. x̄

Question 2

A 95% confidence interval is best interpreted as:

A. A 95% probability that the fixed parameter is in this particular interval

B. A procedure that captures the parameter in about 95% of repeated samples

C. A guarantee that 95% of observations lie in the interval

D. A probability that the null hypothesis is true

Correct Answer: B. A procedure that captures the parameter in about 95% of repeated samples

Question 3

When population σ is unknown and the sample is from a normal population, which interval is generally appropriate for μ?

A. Student’s t interval

B. F interval

C. Binomial interval only

D. Poisson interval

Correct Answer: A. Student’s t interval

Question 4

A p-value is:

A. The probability that H₀ is true

B. The probability that the result is practically important

C. The probability, assuming H₀, of evidence at least as extreme as observed

D. The probability of a Type II error in every situation

Correct Answer: C. The probability, assuming H₀, of evidence at least as extreme as observed

Question 5

If p = 0.03 and α = 0.05, the correct statistical decision is to:

A. Reject H₀

B. Accept H₀ as proven

C. Increase the sample size automatically

D. Conclude that the effect is practically large

Correct Answer: A. Reject H₀

Question 6

Which test is generally preferred for two independent means when equal variances cannot be assumed?

A. Paired t test

B. Welch’s t test

C. One-sample proportion test

D. Chi-square variance test

Correct Answer: B. Welch’s t test

Question 7

A Type I error occurs when an analyst:

A. Fails to reject a false H₀

B. Rejects a true H₀

C. Uses a confidence interval

D. Reports an effect size

Correct Answer: B. Rejects a true H₀

Question 8

Which change usually increases statistical power, all else equal?

A. Reducing sample size

B. Increasing measurement variability

C. Increasing sample size

D. Reducing the true effect to zero

Correct Answer: C. Increasing sample size

Question 9

For a one-sample proportion test, the standard error under H₀ uses:

A. Only p̂ and never p₀

B. The hypothesized proportion p₀

C. The sample standard deviation s

D. The population variance σ² of a continuous outcome

Correct Answer: B. The hypothesized proportion p₀

Question 10

A statistically significant result can still be practically unimportant because:

A. A very large sample can detect a very small effect

B. A p-value always measures financial value

C. Confidence intervals contain no information about size

D. Power is unrelated to sample size

Correct Answer: A. A very large sample can detect a very small effect