7. Hypothesis Testing and Statistical Decision-Making

Introduction and Context

Data science rarely ends with describing what has happened. In many professional settings, the more difficult question is whether an observed pattern is sufficiently convincing to justify action. A marketing analyst may observe a higher conversion rate after a campaign change; a hospital may find that one treatment group recovers more quickly; a manufacturer may record a reduction in defects after introducing a new process. In each case, the observed difference may reflect a genuine underlying effect, random sampling variation, measurement error, or a combination of these factors.

Hypothesis testing provides a formal framework for reasoning under uncertainty. It does not prove that a claim is true or false in an absolute sense. Rather, it evaluates how compatible observed data are with a specified assumption about a population or data-generating process. Statistical decision-making extends this reasoning by considering the size and practical importance of an effect, the costs of errors, available resources, and organizational objectives.

This distinction is essential. A very large dataset can produce a statistically significant result for an effect too small to matter operationally. Conversely, a meaningful effect may fail to achieve statistical significance when the sample is small or highly variable. Competent data scientists therefore interpret p-values alongside confidence intervals, effect sizes, power, study design, and business or policy consequences.

Learning Objectives

By the end of this chapter, the reader should be able to:

1. Formulate null and alternative hypotheses for one-sample and two-sample problems.

2. Explain p-values, significance levels, confidence intervals, and statistical evidence.

3. Conduct and interpret tests for means, proportions, and differences between groups.

4. Distinguish Type I and Type II errors and explain statistical power.

5. Estimate sample-size requirements and understand the influence of effect size and variability.

6. Select appropriate parametric and non-parametric testing procedures.

7. Recognize the problems created by multiple testing, optional stopping, and p-hacking.

8. Combine statistical evidence with effect sizes, costs, risks, and managerial objectives.

9. Apply hypothesis-testing principles in data-science and organizational settings.

From a Business Question to Statistical Hypotheses

A hypothesis test begins with a substantive question, not with a statistical formula. For example, “Does the redesigned checkout page improve purchasing?” must be translated into a measurable outcome, a target population, a comparison, and a decision rule. If the outcome is the proportion of visitors who purchase, the parameter of interest may be the difference between the new and existing conversion rates.

The null hypothesis, denoted by H₀, represents a reference position. It commonly states that there is no difference, no association, or no effect. The alternative hypothesis, denoted by H₁ or Hₐ, represents the claim supported if the data provide sufficient evidence against H₀. For a difference in average delivery time, the hypotheses might be:

H₀: μnew − μold = 0

Hₐ: μnew − μold < 0

The alternative is one-sided because the operational question is whether the new process reduces delivery time. If any difference would be relevant, the alternative would be two-sided:

Hₐ: μnew − μold ≠ 0.

The choice between one-sided and two-sided testing must be made before inspecting the results. A one-sided test should not be selected merely because the observed result happens to point in a convenient direction. In most exploratory and general-purpose analyses, two-sided tests are safer because effects in either direction may have implications.

A test statistic summarizes how far the sample result lies from the null expectation, relative to sampling variability. In a one-sample mean test with known or sufficiently well-estimated standard error, the general form is:

Test statistic = (observed estimate − null value) / standard error.

For a sample mean with unknown population standard deviation, the one-sample t-statistic is:

t = (x̄ − μ₀)/(s/√n),

where x̄ is the sample mean, μ₀ is the hypothesized mean, s is the sample standard deviation, and n is the sample size. Under suitable assumptions, this statistic follows a t-distribution with n − 1 degrees of freedom when H₀ is true.

P-Values, Significance Levels, and Evidence

The p-value is the probability, assuming that H₀ is true, of observing a test statistic at least as extreme as the one obtained. “At least as extreme” is determined by the alternative hypothesis. A small p-value indicates that the data would be unusual under the null model; it does not indicate the probability that H₀ is true.

For example, a p-value of 0.03 does not mean that there is a 3% probability that the null hypothesis is true. Nor does it mean that the alternative hypothesis has a 97% probability of being true. Such interpretations confuse a conditional probability about data given a hypothesis with a probability assigned to the hypothesis itself.

Before testing, the analyst usually specifies a significance level, α, such as 0.05. If p ≤ α, the null hypothesis is rejected. If p > α, the analyst fails to reject H₀. The phrase “fail to reject” is preferable to “accept H₀,” because a non-significant result may reflect insufficient information rather than evidence that the null hypothesis is exactly true.

Statistical significance is a decision convention, not a natural boundary between truth and falsehood. A result with p = 0.049 is not meaningfully different from one with p = 0.051. The p-value should therefore be reported as part of a broader interpretation that includes the estimated effect and its uncertainty.

A confidence interval gives a range of parameter values compatible with the data and the selected confidence procedure. A 95% confidence interval is often described informally as a range of plausible values. In repeated sampling, 95% of intervals constructed by the same valid procedure would contain the fixed parameter. It is not technically correct to say that there is a 95% probability that a particular fixed parameter lies in the already calculated interval under the frequentist interpretation.

For a two-sided test at α = 0.05, a 95% confidence interval that excludes the null value corresponds to rejecting H₀ at the 5% level, assuming the same model and procedure. The interval adds information that a p-value alone cannot provide: direction, magnitude, and precision.

One-Sample and Two-Sample Procedures

One-sample tests

A one-sample test compares a sample with a benchmark or hypothesized population value. A retailer might test whether the average order value differs from $50, or a sensor manufacturer might test whether the mean measurement bias differs from zero.

The one-sample t-test is appropriate for a mean when observations are independent, the outcome is quantitative, and the population is approximately normal or the sample is sufficiently large for the sampling distribution of the mean to be reliable. Severe skewness, extreme outliers, or dependence may invalidate a simple t-test.

For a population proportion, a one-sample proportion test compares an observed proportion p̂ with a hypothesized value p₀. For sufficiently large expected counts, a z approximation may be used:

z = (p̂ − p₀)/√[p₀(1 − p₀)/n].

When expected counts are small, exact binomial methods or carefully chosen alternatives are preferable.

Independent two-sample tests

An independent two-sample test compares outcomes from two unrelated groups, such as customers exposed to two different advertisements. The parameter may be a difference in means, proportions, or distributional characteristics.

The two-sample t-test evaluates a difference in means. Welch’s t-test is generally preferred to the equal-variance version when group variances or sample sizes differ, because it does not require the assumption of equal population variances. The estimated difference should be reported with a confidence interval.

For two proportions, the analyst may test whether conversion rates differ between groups. The test requires attention to independence and expected counts. In a randomized experiment, the difference in proportions is often more interpretable than a ratio alone, although relative measures such as risk ratios or odds ratios can also be useful.

Paired data

A paired design arises when the same units are measured twice or when units are deliberately matched. Examples include employee productivity before and after training, or blood pressure measured before and after treatment. The appropriate analysis is based on within-pair differences:

di = afteri − beforei.

A one-sample t-test can then be applied to the differences. Treating paired observations as independent discards information and may produce an incorrect standard error.

Categorical and frequency data

For categorical outcomes, chi-square tests can assess independence between variables or goodness of fit to a specified distribution. For example, a company may test whether product preference is associated with customer segment. The chi-square approximation depends on adequate expected frequencies; sparse tables may require exact methods or model-based approaches.

Non-parametric alternatives

When assumptions concerning normality or outliers are problematic, rank-based methods may be useful. The Mann–Whitney test compares the distributions or rank locations of two independent groups, while the Wilcoxon signed-rank test addresses paired data. These procedures do not automatically test differences in means, and their interpretation depends on the shape of the distributions. Non-parametric methods are not assumption-free; they replace some assumptions with others.

Assumptions and the Quality of Evidence

Hypothesis tests are conditional on assumptions. The most important include independence, appropriate sampling or assignment, correct measurement, suitable distributional approximations, and a correctly specified comparison. Independence is especially important: thousands of observations from the same customer, machine, household, or time series may contain far less information than thousands of independent observations.

Random sampling supports generalization from a sample to a target population. Random assignment in an experiment supports causal interpretation of differences between treatment groups. These are different ideas. A randomized experiment conducted on a narrow convenience sample may provide strong evidence about causality within the study but limited evidence about a broader population.

Data screening should be conducted before final testing. Analysts should inspect missingness, outliers, coding errors, group imbalance, and measurement definitions. Transformations may stabilize variance or reduce skewness, but they should be justified rather than used merely to obtain a desired p-value.

Type I Error, Type II Error, and Power

A Type I error occurs when H₀ is rejected even though it is true. Its probability is controlled by α under the assumptions of the test. A Type II error occurs when H₀ is not rejected even though a specified alternative is true. Its probability is commonly denoted by β. Statistical power is 1 − β: the probability of detecting an effect of a specified size under specified conditions.

The consequences of these errors depend on context. A false positive in a safety-critical medical screening system may be costly, while a false negative may delay an effective intervention. In exploratory product experimentation, the organization may tolerate more false positives during early discovery but require stronger evidence before large-scale deployment.

Power depends on the significance level, sample size, effect size, variability, and test design. Holding other factors constant:

• Larger samples increase power.
• Larger true effects increase power.
• Lower variability increases power.
• A higher α increases power but also increases Type I error risk.
• One-sided tests can have greater power in the specified direction, but only when scientifically justified.

A non-significant result does not establish that an effect is absent. Analysts should ask whether the confidence interval excludes effects that would matter operationally. If a study estimates a $2 reduction with a 95% interval from −$8 to $12, the evidence is inconclusive about both meaningful savings and meaningful losses. A larger sample may be needed.

Effect Size and Practical Significance

An effect size describes the magnitude of a phenomenon independently, or partly independently, of sample size. For mean differences, the raw effect may be directly meaningful: average processing time fell by 4.2 minutes. A standardized effect such as Cohen’s d expresses the difference relative to pooled variability:

d = (x̄1 − x̄2)/sp.

For proportions, useful measures include absolute risk difference, relative risk, odds ratio, and number needed to treat. The correct measure depends on the decision context. A rise in conversion from 10% to 11% is a one-percentage-point absolute increase but a 10% relative increase. Both statements are mathematically correct, yet they lead to different managerial impressions.

Practical significance requires domain knowledge. A statistically significant improvement of 0.1 seconds may be irrelevant in a low-volume process but important in a high-frequency automated system. Conversely, a small probability reduction may be highly valuable when the consequences of failure are severe. Effect sizes should therefore be translated into financial, operational, customer, safety, or social outcomes.

Sample Size Planning

Sample-size planning should occur before data collection whenever possible. The analyst specifies the desired significance level, target power, minimum detectable effect, variability or baseline rate, design structure, and likely attrition or missingness.

For a two-group comparison of means with equal group sizes, a simplified planning relationship is:

n per group ≈ 2(zα/2 + zβ)²σ²/Δ²,

where σ is the common standard deviation, Δ is the minimum meaningful difference, zα/2 reflects the selected two-sided significance level, and zβ reflects the desired power. The exact formula depends on the test and design.

The minimum detectable effect should not be chosen solely because it produces a convenient sample size. It should represent the smallest change that would justify action after considering costs and constraints. Underpowered studies waste resources and produce imprecise estimates; excessively large studies may detect trivial effects and expose unnecessary numbers of participants to intervention or data collection.

Sequential designs and interim analyses can be efficient, but repeated inspection of unadjusted p-values inflates the Type I error rate. Such designs require pre-specified stopping rules or sequential-testing methods.

Multiple Testing and Research Integrity

When many hypotheses are tested, the chance of at least one false positive increases. If 20 independent null hypotheses are each tested at α = 0.05, the probability of at least one false rejection is approximately 1 − 0.95²⁰, or about 64%, assuming all null hypotheses are true and independence holds.

The family-wise error rate is the probability of making one or more Type I errors within a family of tests. Bonferroni adjustment controls this rate by testing each hypothesis at α/m, where m is the number of tests. Holm’s step-down procedure is often more powerful while maintaining family-wise control. In large-scale exploratory analyses, controlling the false discovery rate—the expected proportion of false discoveries among rejected hypotheses—may be more appropriate. The Benjamini–Hochberg procedure is a widely used approach.

Multiple testing problems also arise through unplanned subgroup analyses, repeated model selection, trying several outcome definitions, and stopping data collection when a desired result appears. These practices can make nominal p-values misleading. Pre-registration, analysis plans, transparent reporting, replication, and separation of exploratory from confirmatory analyses improve credibility.

A Decision Framework for Data Scientists and Managers

Statistical evidence is one input into a decision, not the decision itself. A useful decision process contains five linked questions:

1. What population, intervention, and outcome are being studied?

2. What effect would be practically meaningful?

3. How precise and credible is the estimate?

4. What are the costs of false positives, false negatives, implementation, and delay?

5. What action is justified given the evidence and uncertainty?

A decision may be framed using expected loss. If action A produces benefit when an effect exists but imposes cost when it does not, the organization should compare expected benefits and costs under plausible states of nature. This may lead to different thresholds in a low-risk pilot and a high-risk production deployment.

For example, an A/B test may show that a recommendation algorithm increases click-through rate by 0.8 percentage points, with p = 0.01. Before deployment, management should examine whether the increase also improves revenue, whether long-term retention declines, whether the result generalizes across customer segments, and whether the algorithm introduces unfair exposure patterns. Statistical significance alone cannot answer these questions.

Illustrative Case Study: Evaluating a Customer-Service Intervention

Background

A fictional online retailer introduces an automated message intended to reduce repeat contacts to its customer-service center. Management wants to know whether the intervention reduces the average number of contacts per customer during the 30 days following a support interaction.

Situation and design

Customers are randomly assigned to either the existing communication process or the new automated message. The control group contains 2,400 customers and has a mean of 1.84 contacts, with a standard deviation of 2.10. The intervention group contains 2,400 customers and has a mean of 1.70 contacts, with a standard deviation of 2.00. The observed difference, intervention minus control, is −0.14 contacts.

Management defines a reduction of at least 0.10 contacts as operationally worthwhile because it would reduce workload enough to offset implementation costs. The analysis uses a two-sided Welch t-test at α = 0.05 because either an increase or decrease could be important.

Analysis

The estimated reduction is 0.14 contacts. With large samples, the standard error is sufficiently small that the p-value is below 0.05. Thus, the null hypothesis of no difference is rejected at the selected significance level. The result supports an association between the intervention and fewer subsequent contacts under the randomized design.

However, the analyst also reports a confidence interval. Suppose the 95% interval for the difference is −0.22 to −0.06 contacts. The interval suggests that the true average reduction may be as large as 0.22 contacts or as small as 0.06 contacts. Although the interval excludes zero, it includes effects below the pre-specified practical threshold of 0.10.

The analysis should also examine heterogeneous effects. The intervention may reduce contacts among customers with simple requests but increase them among customers with complex or accessibility-related needs. Segment analysis must be planned or adjusted for multiple comparisons, and subgroup estimates should be interpreted cautiously.

Decision and outcome

Management chooses a monitored phased rollout rather than an immediate universal deployment. The rollout includes a holdout group, monitoring of repeat contacts, resolution time, customer satisfaction, complaint rates, and accessibility indicators. After implementation, the organization can compare operational outcomes and detect unintended consequences.

Learning points

The case illustrates that statistical significance and practical significance are related but distinct. Random assignment strengthens causal interpretation, while the confidence interval reveals uncertainty about the size of the effect. A staged decision is appropriate when the evidence is promising but the consequences of implementation are not fully known. The analysis also demonstrates why a single headline metric should not determine a managerial decision.

Common Errors, Limitations, and Risks

Several errors recur in applied hypothesis testing. Treating p < 0.05 as a universal rule ignores effect size, context, and the cost of errors. Reporting only significant findings creates publication and selection bias. Testing assumptions mechanically, such as relying only on a normality test, can be misleading; graphical inspection, study design, robustness, and substantive knowledge matter as well.

Analysts should avoid changing the hypothesis after seeing the result without labeling the analysis as exploratory. They should not remove outliers solely because they weaken significance, nor should they test many models and report only the most favorable one. Correlation does not become causation through a small p-value. Statistical adjustment cannot repair severe selection bias, confounding, poor measurement, or data leakage.

A further limitation is that classical tests often answer a narrower question than decision-makers believe. A test may assess whether an average differs from a benchmark while management needs to know whether the effect is consistent across regions, profitable after costs, durable over time, or safe for vulnerable groups.

Ethics, Governance, and Digital Practice

Hypothesis testing can affect access to credit, healthcare, employment, insurance, and public services. Analysts must therefore consider privacy, consent, fairness, explainability, and the possibility that an apparently neutral outcome measure embeds unequal consequences. Statistical significance should not be used to legitimize discriminatory practices or to conceal uncertainty from stakeholders.

In automated analytics environments, reproducible pipelines should preserve the data version, feature definitions, sampling rules, code, model settings, and analysis plan. Dashboards that display p-values without confidence intervals or sample sizes can encourage overinterpretation. Automated experimentation platforms should include safeguards for multiple testing, monitoring, stopping rules, and metric manipulation.

Current practice increasingly combines classical inference with resampling, Bayesian analysis, causal inference, and sequential experimentation. Bootstrap intervals can be useful when analytic assumptions are doubtful. Bayesian methods express uncertainty through posterior distributions and can incorporate prior information. Causal methods such as randomized trials, matching, regression adjustment, and difference-in-differences address questions that a simple significance test cannot resolve. These methods do not eliminate the need for careful design and transparent assumptions.

Best Practices for Professional Analysis

A defensible hypothesis-testing workflow should:

• Define the decision, population, outcome, and practically meaningful effect before analysis.
• Select the test according to the design and measurement scale.
• Check independence, missing data, outliers, and relevant distributional assumptions.
• Report the estimate, confidence interval, p-value, sample size, and analytical method.
• Distinguish statistical significance from practical importance.
• Plan sample size and power before collecting data.
• Adjust for multiple comparisons when a family of tests is being interpreted together.
• Separate confirmatory analyses from exploratory findings.
• Conduct sensitivity and subgroup analyses transparently.
• Translate results into operational consequences and monitor post-decision outcomes.

Chapter Summary

Hypothesis testing evaluates the compatibility of observed data with a null model. It requires clearly formulated null and alternative hypotheses, an appropriate test statistic, and assumptions justified by the study design and data. The p-value measures the extremeness of the data under H₀; it is not the probability that H₀ is true. Confidence intervals and effect sizes provide essential information about magnitude and precision.

One-sample, independent two-sample, paired, proportion, categorical, and non-parametric procedures address different designs and outcome types. Type I and Type II errors represent different risks, while power expresses the probability of detecting a specified effect. Sample-size planning should be based on meaningful effects, variability, α, desired power, and resource constraints. Multiple testing, optional stopping, and selective reporting can inflate false discoveries.

For data-science practice, statistical evidence must be integrated with causal design, business value, ethical considerations, implementation costs, and the consequences of incorrect decisions. A statistically significant finding is a starting point for judgment, not a substitute for it.

Key Terms

Null hypothesis: A reference statement about a population parameter, commonly asserting no effect or no difference.

Alternative hypothesis: A statement describing the effect or difference considered plausible against the null hypothesis.

P-value: The probability, assuming the null hypothesis is true, of observing a result at least as extreme as the sample result.

Significance level: The pre-specified maximum probability of a Type I error, denoted by α.

Test statistic: A standardized quantity used to compare an observed estimate with a null value.

Type I error: Rejection of a true null hypothesis.

Type II error: Failure to reject a false null hypothesis.

Statistical power: The probability of rejecting the null hypothesis when a specified alternative effect is true.

Effect size: A quantitative measure of the magnitude of a difference or relationship.

Confidence interval: An interval produced by a procedure that captures the fixed parameter at a stated long-run frequency.

One-sample test: A test comparing a sample estimate with a specified population or benchmark value.

Independent two-sample test: A test comparing outcomes from two unrelated groups.

Paired test: A test based on differences within matched or repeated observations.

Welch’s t-test: A two-sample t-test that does not require equal population variances.

Parametric test: A test based on a specified distributional model and population parameters.

Non-parametric test: A test relying less directly on a particular population distribution, often using ranks.

Family-wise error rate: The probability of making at least one Type I error across a family of tests.

False discovery rate: The expected proportion of false rejections among all rejected hypotheses.

Minimum detectable effect: The smallest effect a study is designed to identify with specified power.

Practical significance: The importance of an effect for real-world decisions, independent of statistical significance alone.

Review Questions

Short Answer Questions

1. What is the difference between a null hypothesis and an alternative hypothesis?

2. Why is “fail to reject” usually preferable to “accept” the null hypothesis?

3. Define a p-value without interpreting it as the probability that the null hypothesis is true.

4. What factors influence statistical power?

5. Why are paired observations analyzed through within-pair differences?

Long Answer Questions

1. Explain how confidence intervals, p-values, and effect sizes complement one another in statistical interpretation.

2. Compare Type I and Type II errors and discuss how their relative costs may affect an organizational decision.

3. Describe the assumptions and appropriate applications of one-sample, independent two-sample, and paired t-tests.

4. Explain why multiple testing increases false-positive risk and compare family-wise error control with false-discovery-rate control.

5. Discuss how a data scientist should translate a statistically significant result into a managerial recommendation.

Analytical / Application Questions

1. An experiment estimates that a new interface increases conversion by 0.6 percentage points, with p = 0.002 and a 95% confidence interval from 0.2 to 1.0 percentage points. Explain what can and cannot be concluded, and identify additional information needed before rollout.

2. A study reports a non-significant difference between two treatments, but its confidence interval ranges from a clinically important benefit to a clinically important harm. How should the result be interpreted, and what would you recommend regarding sample size or further research?

3. An analyst tests 30 customer segments and reports the three segments with p-values below 0.05. Evaluate the evidential problem and propose a more defensible analysis plan.

22. Multiple Choice Questions with Answers

Question 1

Which interpretation of a p-value is correct?

A. It is the probability that the null hypothesis is true.

B. It is the probability that the alternative hypothesis is true.

C. Assuming the null hypothesis is true, it is the probability of observing a result at least as extreme as the one obtained.

D. It is the probability that the sample was selected randomly.

Correct Answer: C. Assuming the null hypothesis is true, it is the probability of observing a result at least as extreme as the one obtained.

Question 2

A study estimates a mean difference of 4 units with a 95% confidence interval from -2 to 10 units. At the 5% two-sided level, what is the appropriate conclusion?

A. Reject the null hypothesis because the estimated difference is positive.

B. Reject the null hypothesis because the interval contains values greater than zero.

C. Fail to reject the null hypothesis because the interval includes zero.

D. Accept the null hypothesis as exactly true.

Correct Answer: C. Fail to reject the null hypothesis because the interval includes zero.

Question 3

Which design most directly calls for a paired analysis?

A. Comparing two unrelated customer samples.

B. Comparing measurements from the same employees before and after training.

C. Comparing defect rates across two independent factories.

D. Comparing preferences across independent age groups.

Correct Answer: B. Comparing measurements from the same employees before and after training.

Question 4

Which change generally increases statistical power, all else being equal?

A. Reducing the sample size.

B. Increasing measurement variability.

C. Reducing the true effect size.

D. Increasing the sample size.

Correct Answer: D. Increasing the sample size.

Question 5

A Type I error occurs when an analyst:

A. Fails to reject a false null hypothesis.

B. Rejects a true null hypothesis.

C. Uses a sample that is larger than necessary.

D. Reports an effect size without a p-value.

Correct Answer: B. Rejects a true null hypothesis.

Question 6

Why can a very large dataset produce a statistically significant but practically unimportant result?

A. Large samples always violate independence.

B. The standard error can become very small, making tiny effects statistically detectable.

C. P-values become probabilities that the alternative is true.

D. Confidence intervals become wider as sample size increases.

Correct Answer: B. The standard error can become very small, making tiny effects statistically detectable.

Question 7

When comparing two means with unequal variances and unequal sample sizes, which procedure is generally appropriate?

A. Welch’s t-test.

B. A one-sample proportion test.

C. A paired t-test regardless of study design.

D. A chi-square goodness-of-fit test.

Correct Answer: A. Welch’s t-test.

Question 8

What is the main purpose of a multiple-testing adjustment?

A. To increase the observed effect size.

B. To remove all sampling variability.

C. To control the increased risk of false discoveries created by testing many hypotheses.

D. To guarantee that every rejected hypothesis is causally valid.

Correct Answer: C. To control the increased risk of false discoveries created by testing many hypotheses.

Question 9

An analyst repeatedly checks results and stops data collection as soon as p < 0.05 without a pre-specified rule. What is the principal concern?

A. The procedure may inflate the Type I error rate.

B. The procedure necessarily eliminates statistical power.

C. The sample mean becomes equal to the population mean.

D. The analysis becomes automatically non-parametric.

Correct Answer: A. The procedure may inflate the Type I error rate.

Question 10

Which statement best reflects sound managerial use of statistical evidence?

A. Any statistically significant result should be implemented immediately.

B. A non-significant result proves that no effect exists.

C. The decision should consider effect size, uncertainty, error costs, implementation consequences, and evidence quality.

D. The p-value alone determines whether an intervention is profitable.

Correct Answer: C. The decision should consider effect size, uncertainty, error costs, implementation consequences, and evidence quality.