Hypothesis Testing and Statistical Decision Making

Introduction

Statistical inference is concerned with learning about a population from incomplete information contained in a sample. Estimation describes plausible values of unknown parameters; hypothesis testing addresses claims or decisions about those parameters. In artificial intelligence and machine learning, hypothesis testing is used to compare models, evaluate interventions, detect distributional changes, assess fairness, and determine whether an observed improvement is likely to reflect more than sampling variation.

A statistical test does not prove that a hypothesis is true. Rather, it quantifies how incompatible the observed data are with a precisely stated null model. Sound statistical decision making therefore requires more than calculating a p-value. It requires formulating appropriate hypotheses, selecting a test statistic, checking assumptions, distinguishing statistical from practical significance, considering power and effect size, and accounting for multiple comparisons.

Learning Objectives

By the end of this chapter, students should be able to:

• Formulate null and alternative hypotheses for common research and business questions.
• Explain test statistics, sampling distributions, p-values, significance levels, and confidence intervals.
• Distinguish Type I and Type II errors and explain statistical power.
• Conduct and interpret one-sample, two-sample, and proportion tests.
• Explain effect size and practical significance.
• Describe likelihood-ratio tests and their relationship to common tests.
• Recognize the dangers of multiple testing and select appropriate corrections.
• Apply hypothesis testing to an AI/ML decision while considering assumptions, ethics, and governance.

1. The Logic of Statistical Hypothesis Testing

A hypothesis is a claim about a population, probability distribution, or data-generating process. The null hypothesis, denoted \(H_0\), represents a reference position, such as no difference, no association, or a specified parameter value. The alternative hypothesis, denoted \(H_1\) or \(H_A\), represents the claim supported when the data provide sufficiently strong evidence against \(H_0\).

For a population mean \(\mu\), examples include:

• Two-sided test: \(H_0:\mu=\mu_0\), \(H_A:\mu\ne\mu_0\).
• Upper-tailed test: \(H_0:\mu\leq\mu_0\), \(H_A:\mu>\mu_0\).
• Lower-tailed test: \(H_0:\mu\geq\mu_0\), \(H_A:\mu<\mu_0\).

The equality is conventionally placed in the null hypothesis because the sampling distribution used to calibrate the test is specified under the boundary or equality case. The direction of the alternative must be chosen before inspecting the results; changing from a two-sided to a one-sided test after seeing the data inflates the false-positive risk.

A test typically follows this sequence:

1. Define the population, parameter, hypotheses, and decision context.

2. Choose a test statistic and its null distribution.

3. Set a significance level \(\alpha\) before analyzing the result.

4. Collect data and calculate the statistic.

5. Compute a p-value or compare the statistic with a critical value.

6. Reject or fail to reject \(H_0\).

7. Report the effect size, uncertainty, assumptions, and practical implications.

“Fail to reject” does not mean “accept” or “prove” the null hypothesis. It means that the data do not provide sufficient evidence against the null at the selected threshold.

2. Test Statistics, Null Distributions, and p-Values

A test statistic summarizes evidence in the data relative to the null hypothesis. A useful statistic usually measures the estimated departure from the null value in units of its standard error.

For a sample mean with known population standard deviation \(\sigma\), sample size \(n\), and hypothesized mean \(\mu_0\), the one-sample z statistic is

\[

z=\frac{\bar{x}-\mu_0}{\sigma/\sqrt{n}}.

\]

Here, \(\bar{x}\) is the sample mean and \(\sigma/\sqrt n\) is the standard error of the mean. This formula is appropriate when observations are independent, \(\sigma\) is known, and the sampling distribution of the mean is normal or approximately normal, often through a sufficiently large sample.

When \(\sigma\) is unknown, it is estimated by the sample standard deviation \(s\), giving the one-sample t statistic:

\[

t=\frac{\bar{x}-\mu_0}{s/\sqrt n},

\]

which, under independent normally distributed observations, follows a t distribution with \(n-1\) degrees of freedom under \(H_0\). For large samples, the t distribution approaches the standard normal distribution.

A p-value is the probability, assuming \(H_0\) is true, of observing a test statistic at least as extreme as the one obtained, in the direction specified by the alternative. For a two-sided z test with observed value \(z_{obs}\),

\[

p=2P(Z\geq |z_{obs}|),

\]

where \(Z\) has a standard normal distribution. A p-value is not the probability that \(H_0\) is true, nor is it the probability that the result occurred “by chance.” It is conditional on the null model and measures extremeness under that model.

The significance level \(\alpha\) is the maximum long-run probability of rejecting a true null hypothesis that the procedure is designed to tolerate. Common choices are 0.05 and 0.01, but the appropriate level depends on consequences, prior evidence, and the decision environment. If \(p\leq\alpha\), the result is conventionally called statistically significant and \(H_0\) is rejected. If \(p>\alpha\), \(H_0\) is not rejected.

A critical-value approach is equivalent. For a two-sided z test at \(\alpha=0.05\), the critical values are approximately \(-1.96\) and \(1.96\). Values outside this interval lead to rejection.

3. Type I Error, Type II Error, and Power

A decision can be represented by the following possibilities:

| Reality | Decision: reject \(H_0\) | Decision: fail to reject \(H_0\) |

|---|---|---|

| \(H_0\) true | Type I error | Correct decision |

| \(H_0\) false | Correct decision | Type II error |

A Type I error is rejecting a true null hypothesis. Its probability is controlled by \(\alpha\), provided the assumptions and testing procedure are valid. A Type II error is failing to reject a false null hypothesis. Its probability is denoted \(\beta\), although it varies with the actual parameter value, sample size, variability, and test design.

Power is the probability of rejecting \(H_0\) when a specified alternative is true:

\[

\text{Power}=1-\beta.

\]

Power is not a single universal number unless a particular alternative effect has been specified. It generally increases when:

• the sample size increases;
• the true effect is larger;
• measurement variability decreases;
• the significance level increases; or
• a genuinely appropriate one-sided test is used instead of a two-sided test.

There is a trade-off between \(\alpha\) and \(\beta\). Reducing false positives by lowering \(\alpha\) often reduces power unless the sample size is increased. A power analysis conducted before data collection can determine the sample size needed to detect a practically important effect with a chosen power, commonly 80% or 90%.

For a two-sample comparison of means with equal group sizes, a rough planning relationship is

\[

n_{group}\approx \frac{2(z_{1-\alpha/2}+z_{1-\beta})^2\sigma^2}{\Delta^2},

\]

where \(n_{group}\) is the required sample size per group, \(\sigma\) is the anticipated common standard deviation, \(\Delta\) is the smallest effect worth detecting, and \(z_q\) is the qth standard-normal quantile. This approximation assumes independent observations, approximately normal means, equal group sizes, and a two-sided test. More precise calculations should use the intended test and software.

4. Common Tests and Their Assumptions

4.1 One-Sample Mean Test

A one-sample t test evaluates whether a population mean differs from a specified value when the population standard deviation is unknown. The observations should be independent, and the population should be approximately normal for small samples. Strong skewness, outliers, dependence, or a poorly defined sampling process can invalidate the result.

4.2 Two Independent Means

To compare means from two independent groups, a common statistic is Welch’s t statistic:

\[

t=\frac{\bar{x}_1-\bar{x}_2}{\sqrt{s_1^2/n_1+s_2^2/n_2}}.

\]

Here \(\bar{x}_j\), \(s_j\), and \(n_j\) are the sample mean, sample standard deviation, and sample size in group \(j\). Welch’s test does not require equal population variances and is generally preferred to the pooled-variance test when variance equality is uncertain. The degrees of freedom are estimated using the Welch–Satterthwaite formula, usually supplied by statistical software.

4.3 Paired Data

When each observation in one condition is naturally matched with an observation in another condition—for example, the same user before and after a system change—the analysis should use within-pair differences. A paired t test tests whether the mean difference differs from zero. Treating paired observations as independent discards information and can produce incorrect standard errors.

4.4 Proportions

For a sample proportion \(\hat p=x/n\), a large-sample test of \(H_0:p=p_0\) uses

\[

z=\frac{\hat p-p_0}{\sqrt{p_0(1-p_0)/n}}.

\]

The denominator is the null standard error. This approximation requires adequate expected counts; for small or sparse samples, exact binomial methods or carefully chosen alternatives are more appropriate. For two proportions, methods should account for the sampling design and, where relevant, use confidence intervals or regression models for adjusted comparisons.

4.5 Association and Independence

A chi-square test can assess whether two categorical variables are independent. It compares observed cell counts with expected counts under independence:

\[

\chi^2=\sum_{i,j}\frac{(O_{ij}-E_{ij})^2}{E_{ij}},

\]

where \(O_{ij}\) is an observed count and \(E_{ij}\) is the expected count under the null. The approximation is unreliable when expected counts are very small; exact or model-based methods may be preferable.

5. Worked Example: Evaluating a Model Improvement

An AI team claims that a new fraud-detection model reduces the mean processing time below the current benchmark of 120 milliseconds. A random sample of 25 independent transactions processed by the new system has mean time \(\bar{x}=116\) ms and sample standard deviation \(s=10\) ms.

The hypotheses are

\[

H_0:\mu\geq120, \qquad H_A:\mu<120.

\]

Using the boundary value \(\mu_0=120\), the test statistic is

\[

t=\frac{116-120}{10/\sqrt{25}}=\frac{-4}{2}=-2.00.

\]

The test has 24 degrees of freedom. The one-sided p-value for \(t=-2.00\) is approximately 0.028. At \(\alpha=0.05\), the team rejects \(H_0\) and reports evidence that the mean processing time is below 120 ms.

However, statistical significance is not the complete decision. The estimated improvement is 4 ms, and the standardized effect estimate is approximately \(4/10=0.40\), a moderate difference under a common descriptive convention. The team should also report a confidence interval, examine latency variability and tail latency, verify that the sample represents production traffic, and assess whether throughput, accuracy, energy use, or fairness changed. A statistically significant reduction in mean latency might be operationally unimportant if the 99th-percentile latency worsens or if fraud detection declines.

6. Effect Size and Practical Significance

Statistical significance depends strongly on sample size. With a very large sample, a trivial difference may produce a very small p-value. With a small sample, an important difference may fail to reach significance. An effect size describes the magnitude of a difference or association independently, as far as possible, from sample size.

For a difference in means, an unstandardized effect is

\[

\Delta=\mu_1-\mu_2,

\]

measured in the original units. This is often the most useful quantity for engineering or managerial decisions. A standardized mean difference may be expressed as Cohen’s d:

\[

d=\frac{\bar{x}_1-\bar{x}_2}{s_{pooled}},

\]

where, for two independent groups,

\[

s_{pooled}=\sqrt{\frac{(n_1-1)s_1^2+(n_2-1)s_2^2}{n_1+n_2-2}}.

\]

Cohen’s d is meaningful when a common scale of variability is defensible; it should not replace the original-unit difference. For binary outcomes, useful measures include risk difference, relative risk, and odds ratio. For model evaluation, improvements in accuracy, recall, calibration error, false-positive rate, expected cost, or revenue may be more interpretable than a generic standardized effect.

Practical significance asks whether an effect is large enough to matter in the real decision context. Before testing, stakeholders should define a minimally important difference, such as a 5 ms latency reduction, a 1-percentage-point reduction in false negatives, or a specified monetary benefit. Confidence intervals are valuable because they show both plausible effect sizes and uncertainty. A result may be statistically significant but practically negligible, or practically promising but too imprecise to support deployment.

7. Likelihood-Ratio Tests

Many statistical models are specified through a likelihood. For data \(x\) and parameter \(\theta\), the likelihood is

\[

L(\theta\mid x)=f(x\mid\theta),

\]

viewed as a function of \(\theta\) for observed data \(x\). A likelihood-ratio test compares the best fit under the null hypothesis with the best fit under the unrestricted alternative.

For a simple null and a general alternative, the likelihood ratio is

\[

\Lambda(x)=\frac{\sup_{\theta\in\Theta_0}L(\theta\mid x)}{\sup_{\theta\in\Theta}L(\theta\mid x)},

\]

where \(\Theta_0\) is the parameter space allowed by \(H_0\), and \(\Theta\) is the full parameter space. Small values indicate that the null fits substantially worse than the unrestricted model. A common statistic is

\[

-2\log\Lambda.

\]

Under regularity conditions and for large samples, this statistic is approximately chi-square distributed with degrees of freedom equal to the difference in the number of free parameters between the alternative and null models. This result is associated with Wilks’ theorem. Regularity can fail at parameter boundaries, in mixture models, with non-identifiability, or in small samples, so the chi-square approximation should not be used automatically.

Likelihood-ratio tests are central in generalized linear models, logistic regression, survival analysis, and nested model comparison. A likelihood-ratio test compares nested models; comparing non-nested predictive models may require other methods, such as cross-validation, information criteria, or task-specific evaluation. In machine learning, a statistically significant likelihood improvement does not automatically imply better out-of-sample predictions.

8. Multiple Testing

If one test is conducted at \(\alpha=0.05\), the probability of a false rejection is controlled at approximately 5% under the stated procedure. If 20 independent true null hypotheses are each tested at 0.05, the probability of at least one false rejection is

\[

1-(1-0.05)^{20}\approx0.642.

\]

Thus, examining many metrics, subgroups, model versions, or features and reporting only significant findings can produce misleading results. This is the multiple-testing problem.

The family-wise error rate (FWER) is the probability of making at least one Type I error in a family of tests. The Bonferroni procedure tests each hypothesis at \(\alpha/m\), where \(m\) is the number of tests, and guarantees FWER control under arbitrary dependence, though it can be conservative. Holm’s step-down procedure usually provides stronger power while retaining FWER control.

The false discovery rate (FDR) is the expected proportion of false rejections among all rejections. The Benjamini–Hochberg procedure controls FDR under standard conditions and is often useful in exploratory high-dimensional work, such as genomics or feature screening. FDR control does not guarantee that every reported finding is true.

Best practices include specifying the primary outcome in advance, distinguishing confirmatory from exploratory analyses, reporting all tested outcomes or a transparent testing family, correcting for multiplicity when appropriate, and validating discoveries on new data. Holding out a test set protects predictive evaluation, but it does not eliminate all problems caused by repeated experimentation or adaptive research decisions.

9. Statistical Decision Making in AI and Organizations

A statistical test is part of a decision system rather than an isolated calculation. Decisions should connect the hypothesis to a loss function or consequence structure. A false positive might lead to unnecessary medical treatment, an unwarranted account suspension, or an expensive deployment. A false negative might allow fraud, deny a qualified applicant, or miss a safety failure. Therefore, the acceptable balance between Type I and Type II errors depends on context.

In AI evaluation, common questions include:

• Is a new classifier’s error rate lower than that of a baseline?
• Is the difference in performance stable across resampled test sets?
• Does a model’s false-positive rate differ across protected groups?
• Has a production data stream shifted from the training distribution?
• Does a treatment recommendation improve outcomes rather than merely prediction scores?

The unit of analysis must be chosen carefully. If many records come from the same user, hospital, device, or time period, observations may not be independent. Clustered data require grouped sampling, mixed-effects models, cluster-robust standard errors, or other designs. Reusing the same test set repeatedly can lead to overfitting the evaluation process. A pre-registered protocol, versioned data, locked evaluation scripts, and independent replication improve credibility.

Fairness-related testing requires particular caution. A statistically significant group difference may reflect a tiny operational disparity in a huge dataset, while a non-significant result may reflect low power in a small protected group. Researchers should report effect sizes and confidence intervals, examine intersectional groups where feasible, and involve domain experts and affected communities. Statistical parity alone cannot resolve legal, ethical, or social questions about justified use of an AI system.

10. Illustrative Case: A Credit-Risk Model

Background

A financial institution develops Model B to replace Model A for credit-risk screening. On a temporally separated validation sample, both models are evaluated. The organization hopes to reduce false negatives while maintaining a tolerable false-positive rate.

Problem

The team initially reports that Model B has a statistically significant 0.8-percentage-point improvement in overall accuracy. However, the validation set contains many more low-risk than high-risk applications, and the cost of the two error types is very different. A single accuracy test may therefore be inadequate.

Analysis

The team defines primary outcomes before further analysis: false-negative rate, false-positive rate, calibration, and expected cost under an agreed cost matrix. It compares paired predictions because both models classify the same applications. A paired method such as McNemar’s test is appropriate for the disagreement pattern between two binary classifiers, rather than treating the two accuracy estimates as independent. Confidence intervals for performance differences are also computed using a resampling method that preserves relevant applicant structure.

The team examines performance across legally and ethically relevant groups, but treats subgroup analyses as a planned testing family. It uses multiplicity adjustments for confirmatory subgroup claims and labels additional analyses as exploratory. It checks whether results persist in a later time period and assesses whether missing data or changes in application composition affect conclusions.

Approach and Outcome

Model B reduces the economically costly false-negative rate by 1.5 percentage points, with a confidence interval that excludes zero, while its false-positive rate increases by 0.3 percentage points. The increase is statistically detectable in the full sample but is below the institution’s defined operational tolerance. A subgroup analysis identifies a disparity requiring mitigation before deployment, despite the favorable aggregate result. The organization delays full deployment, recalibrates the model, and conducts a new prospective evaluation.

Learning Points

The case demonstrates that an appropriate test depends on paired design, outcome definition, error costs, multiplicity, and fairness obligations. Aggregate statistical significance cannot substitute for a decision-relevant effect size or responsible governance. A model comparison should evaluate generalization, not merely fit to one historical validation set.

11. Assumptions, Limitations, and Good Practice

Hypothesis tests can be misleading when the sampling process is biased, observations are dependent, measurements are selectively reported, or the model is misspecified. Normality tests alone are not a reliable substitute for substantive diagnostics; plots, subject-matter knowledge, residual analysis, and sensitivity analyses are important. Robust or nonparametric methods may help, but they do not automatically repair biased sampling or confounding.

Researchers should avoid p-hacking, which includes trying many analyses until one is significant, selectively excluding observations, changing outcomes after seeing results, or stopping data collection opportunistically. These practices can make nominal error rates meaningless. Pre-registration, analysis plans, transparent reporting, replication, and sharing code where privacy permits help distinguish planned confirmation from discovery.

A p-value should normally be reported with the test statistic, degrees of freedom when applicable, effect estimate, confidence interval, sample size, and relevant assumptions. Rounding a p-value to “significant” or “not significant” hides important information. Statistical significance is evidence relative to a model, not a measure of truth, importance, causality, or reproducibility.

12. Contemporary Developments

Modern data analysis increasingly uses resampling, randomization tests, Bayesian decision analysis, sequential monitoring, and causal inference alongside classical tests. Permutation tests construct a reference distribution by rearranging labels under an exchangeability assumption and can be useful when parametric assumptions are doubtful. Bayesian methods report posterior probabilities or credible intervals based on a prior and likelihood; they answer different questions from classical p-values and should not be interpreted interchangeably.

In adaptive experimentation and online A/B testing, repeated monitoring can inflate false-positive rates unless sequential methods or alpha-spending procedures are used. In large-scale ML, uncertainty estimation, distribution-shift detection, conformal prediction, and calibration assessment complement traditional hypothesis testing. These tools remain dependent on data quality, valid sampling, and clearly defined decision objectives.

Chapter Summary

Hypothesis testing provides a structured way to evaluate claims about populations and models. A null hypothesis defines a reference data-generating position, while an alternative specifies the departure of interest. Test statistics are interpreted through null sampling distributions, and p-values quantify the extremeness of observed data assuming the null. The significance level controls the planned Type I error rate, whereas Type II error and power depend on effect size, variability, sample size, and design.

Tests must match the data structure: one-sample, independent-group, paired, proportion, chi-square, and likelihood-ratio procedures have different assumptions and applications. Effect sizes and confidence intervals are essential for judging practical importance. Likelihood-ratio tests compare nested model fits, while multiple-testing procedures control family-wise error or false discovery rate. In AI and organizational decisions, statistical evidence must be connected to costs, fairness, generalization, and governance. Transparent planning, assumption checking, replication, and complete reporting are central to trustworthy inference.

Key Terms

Null hypothesis: A reference claim or restricted model tested against data.

Alternative hypothesis: A claim representing the departure from the null hypothesis.

Test statistic: A numerical summary used to compare data with a null distribution.

p-value: The probability, under the null model, of data at least as extreme as observed.

Significance level: The planned upper bound on the Type I error probability.

Critical value: A cutoff separating rejection and non-rejection regions.

Type I error: Rejection of a true null hypothesis.

Type II error: Failure to reject a false null hypothesis.

Power: Probability of rejecting the null when a specified alternative is true.

Effect size: A quantitative measure of the magnitude of a difference or association.

Practical significance: Importance of an effect for a real decision or application.

Confidence interval: A procedure-based interval expressing uncertainty about a parameter.

Standard error: The sampling standard deviation of an estimator.

Likelihood ratio: The ratio of the best null-constrained likelihood to the unrestricted likelihood.

Family-wise error rate: Probability of at least one false rejection in a family of tests.

False discovery rate: Expected proportion of false rejections among all rejections.

Welch’s t test: A two-sample mean test that does not require equal variances.

Power analysis: Planning analysis used to determine detectable effects or required sample size.

Statistical significance: Evidence judged incompatible with the null at a chosen threshold.

Pre-registration: Public specification of hypotheses and analysis plans before observing results.

Review Questions

Short Answer Questions

1. What is the difference between a null hypothesis and an alternative hypothesis?

2. Why is a p-value not the probability that the null hypothesis is true?

3. Define Type I error, Type II error, and statistical power.

4. Why should effect size accompany a significance test?

5. What problem is addressed by multiple-testing corrections?

Long Answer Questions

1. Explain the complete workflow of a hypothesis test, including assumptions and reporting requirements.

2. Compare one-sample, independent two-sample, and paired t tests, including when each is appropriate.

3. Explain likelihood-ratio testing and the conditions under which its chi-square approximation may fail.

4. Discuss how sample size, variability, significance level, and effect size influence power.

5. Explain why statistical significance and practical significance can lead to different conclusions in AI evaluation.

Analytical / Application Questions

1. A model’s average latency falls from 200 ms to 198 ms in a sample of 100,000 requests, with very low variability. Explain why statistical significance may not establish operational importance, and identify additional metrics to examine.

2. An experiment tests 30 independent outcomes at the 5% level and reports the smallest p-value. Identify the inferential problem and propose a more responsible analysis strategy.

3. Two credit models classify the same applicants, but one has a lower aggregate error rate and a higher false-negative rate for a protected group. Design a decision-making analysis that incorporates paired testing, effect sizes, uncertainty, error costs, and fairness governance.

Multiple Choice Questions with Answers

Question 1

Which statement correctly describes a p-value?

A. The probability that the null hypothesis is true

B. The probability of observing data at least as extreme, assuming the null hypothesis is true

C. The probability that the alternative hypothesis is false

D. The probability that a replication will succeed

Correct Answer: B. The probability of observing data at least as extreme, assuming the null hypothesis is true

Question 2

A Type I error occurs when a researcher:

A. Fails to reject a false null hypothesis

B. Rejects a true null hypothesis

C. Accepts a true alternative hypothesis

D. Uses a sample that is too large

Correct Answer: B. Rejects a true null hypothesis

Question 3

Which change generally increases statistical power, all else equal?

A. Reducing sample size

B. Increasing measurement variability

C. Increasing sample size

D. Reducing the true effect to zero

Correct Answer: C. Increasing sample size

Question 4

Which test is most appropriate for comparing measurements taken on the same individuals before and after an intervention?

A. Paired t test

B. Independent pooled-variance test

C. One-sample proportion test

D. Chi-square test of independence

Correct Answer: A. Paired t test

Question 5

For a two-sided z test at significance level 0.05, the approximate critical values are:

A. -1.28 and 1.28

B. -1.645 and 1.645

C. -1.96 and 1.96

D. -2.58 and 2.58

Correct Answer: C. -1.96 and 1.96

Question 6

What does practical significance assess?

A. Whether a result has any decimal places

B. Whether an effect is large enough to matter in the decision context

C. Whether the sample was randomly generated by software

D. Whether the null hypothesis is logically possible

Correct Answer: B. Whether an effect is large enough to matter in the decision context

Question 7

The Bonferroni procedure controls which quantity by testing each of m hypotheses at alpha divided by m?

A. The family-wise error rate

B. The population mean

C. The effect size

D. The statistical power at one fixed alternative

Correct Answer: A. The family-wise error rate

Question 8

A likelihood-ratio test primarily compares:

A. Two unrelated sample means only

B. The best fit under a null-constrained model with the best unrestricted fit

C. Two p-values without reference to models

D. Observed counts with their sample medians

Correct Answer: B. The best fit under a null-constrained model with the best unrestricted fit

Question 9

If a test has beta equal to 0.20 for a specified alternative, its power is:

A. 0.02

B. 0.20

C. 0.80

D. 1.20

Correct Answer: C. 0.80

Question 10

Why can repeated testing of many outcomes create misleading findings?

A. It always reduces the sample mean

B. It increases the chance of at least one false positive

C. It eliminates Type II errors

D. It makes all tests paired

Correct Answer: B. It increases the chance of at least one false positive