Statistical Inference and Relationship Analysis

Introduction

Marketing decisions are commonly based on samples rather than complete populations. A retailer may survey 400 customers instead of every customer; a brand manager may compare two advertisements using an experiment; and an analyst may examine whether satisfaction predicts repurchase intention. Statistical inference provides disciplined methods for moving from sample evidence to carefully qualified conclusions about a target population.

This chapter examines hypothesis testing, confidence intervals, chi-square tests, t-tests, analysis of variance (ANOVA), correlation, and linear and logistic regression. It also addresses assumptions, statistical power, effect size, managerial significance, analytical errors, and ethical interpretation. The central principle is that statistical evidence informs managerial judgment; it does not replace it.

Learning Objectives

By the end of this chapter, students should be able to:

1. Formulate null and alternative hypotheses and interpret p-values and confidence intervals.
2. Select appropriate chi-square, t, ANOVA, correlation, and regression procedures.
3. Explain assumptions, statistical power, effect size, and practical significance.
4. Interpret coefficients and model results in marketing contexts.
5. Diagnose common analytical errors and communicate results responsibly.

1. Populations, Samples, and Inference

A population is the complete set of units relevant to a research question, such as all customers of a supermarket chain. A sample is the subset actually observed. A population quantity is a parameter; a sample quantity is a statistic. For example, μ denotes a population mean, whereas x̄ denotes a sample mean.

Sampling creates uncertainty. If repeated random samples were drawn from the same population, their means would differ. The standard error describes the typical sampling variation of a statistic. For a sample mean:

where s is the sample standard deviation and n is the sample size. This formula is appropriate when observations are independent and the sample is reasonably representative. Larger samples generally reduce sampling error, although they do not correct coverage bias, nonresponse bias, or poor measurement.

A confidence interval gives a range of plausible values for a population parameter. A general interval for a mean is:

where t* is a critical value from the t distribution for the chosen confidence level and degrees of freedom. A 95% confidence interval does not mean that there is a 95% probability that the already-fixed parameter lies in this particular interval. Rather, a procedure that produces such intervals will capture the true parameter in approximately 95% of repeated samples under its assumptions.

2. Hypothesis Testing

Hypothesis testing evaluates whether sample evidence is sufficiently inconsistent with a specified null model. The null hypothesis, H₀, usually represents no difference, no association, or a specified parameter value. The alternative hypothesis, H₁ or Hₐ, represents the research claim.

For example, a researcher testing whether a redesigned product page changes mean conversion rate might specify:

• H₀: μnew − μold = 0
• Hₐ: μnew − μold ≠ 0

A test statistic measures the distance between the sample result and the null value relative to sampling variability. A p-value is the probability, assuming H₀ is true and the model is appropriate, of obtaining a result at least as extreme as the observed result. A small p-value indicates incompatibility with H₀; it does not indicate the probability that H₀ is true, the size of an effect, or its managerial value.

At significance level α, researchers reject H₀ when p ≤ α. Common choices are .05 and .01. A Type I error rejects a true H₀; a Type II error fails to reject a false H₀. The probability of detecting a specified effect is power:

where β is the probability of a Type II error. Power increases with larger samples, larger true effects, lower measurement variability, and a less stringent α. A nonsignificant result may therefore reflect either no practically important effect or insufficient power.

Researchers should state hypotheses, outcomes, and analysis plans before examining results where possible. One-tailed tests require a defensible directional hypothesis established in advance. Using a one-tailed test after seeing the data inflates the risk of misleading significance.

3. Choosing a Statistical Test

The appropriate procedure depends on the measurement scale, research design, number of groups, and question being asked.

The test must match the design. For example, comparing the same customers before and after a campaign requires a paired analysis, not an independent-groups test.

4. Chi-Square Tests for Categorical Data

The chi-square test of independence examines whether two categorical variables are associated. Suppose a researcher records channel preference (social media, search, email) and age group. The null hypothesis states that channel preference and age group are independent.

The test compares observed counts, Oᵢⱼ, with expected counts, Eᵢⱼ, under independence:

For a contingency table:

The degrees of freedom are:

where r is the number of rows and c is the number of columns. A significant test indicates association, not causation. Analysts should inspect standardized residuals or conditional percentages to identify which cells contribute to the association.

Expected counts should generally be sufficiently large; a common guideline is that expected counts should be at least 5 in most cells and not below 1. Sparse tables may require combining substantively similar categories or using an exact test. The chi-square test is not appropriate for comparing means or for treating arbitrary numerical codes as continuous measurements.

A useful effect-size measure is Cramér’s V:

where N is the total sample size. Interpretation depends on context, table dimensions, and field conventions.

5. t-Tests

Independent-Samples t-Test

An independent-samples t-test compares means from two unrelated groups. With a pooled-variance assumption, the statistic is:

where x̄₁ and x̄₂ are sample means, n₁ and n₂ are sample sizes, and sₚ is the pooled standard deviation. In practice, Welch’s t-test is often preferred because it does not assume equal population variances. It is especially appropriate when group sizes or variances differ.

Assumptions include independent observations, a quantitative outcome, and distributions that are not severely distorted by outliers or extreme skewness. With moderate or large samples, the t procedure is often robust, but robustness should not be treated as permission to ignore the data structure.

Paired-Samples t-Test

A paired test analyzes within-unit differences, dᵢ, such as each respondent’s post-campaign score minus pre-campaign score:

where d̄ is the mean difference, s_d is the standard deviation of differences, and n is the number of pairs. The assumptions apply to the differences, not separately to the two raw measurements.

A confidence interval for the mean difference is often more informative than a p-value alone. Standardized mean differences, such as Cohen’s d, can supplement the analysis, but the chosen effect-size definition must match the design.

6. Analysis of Variance

One-way ANOVA tests whether three or more independent group means are equal. Its hypotheses are:

• H₀: μ₁ = μ₂ = … = μₖ
• Hₐ: at least one population mean differs

ANOVA partitions total variation into variation between groups and variation within groups:

where MSbetween is the mean square between groups and MSwithin is the mean square within groups. The F statistic is large when group means differ relative to within-group variation.

The assumptions are independent observations, approximately normal errors within groups, and reasonably similar variances. Levene’s test can assess variance equality, although graphical inspection and substantive knowledge are also important. When assumptions are seriously violated, analysts may consider transformations, Welch’s ANOVA, or nonparametric procedures.

A significant omnibus F-test says that at least two means differ, but not which ones. Post-hoc comparisons, such as Tukey’s method, identify differences while controlling familywise error. Conducting many unadjusted t-tests increases the probability of false positives.

ANOVA is a general linear-model framework. Factorial ANOVA can examine multiple categorical predictors and interactions. An interaction means that the effect of one factor depends on the level of another—for example, an advertisement may work better for new customers but not existing customers.

7. Correlation and Association

Pearson’s correlation measures the direction and strength of a linear relationship between two quantitative variables:

The coefficient ranges from −1 to +1. A positive value indicates that higher values of one variable tend to accompany higher values of the other; a negative value indicates an inverse linear tendency. A value near zero means little linear association, but nonlinear association may still exist.

Correlation is not causation. A third variable may cause both observed variables, reverse causality may operate, or the relationship may be produced by selection bias. Correlation is also sensitive to outliers and restricted range. Scatterplots should precede interpretation. Spearman’s rank correlation may be suitable for ordinal data or monotonic relationships that violate Pearson assumptions.

8. Linear Regression

Simple linear regression models a continuous outcome as a function of a predictor:

Here yᵢ is the outcome for observation i, xᵢ is the predictor, β₀ is the intercept, β₁ is the expected change in y for a one-unit increase in x, and εᵢ is the error term. The fitted equation replaces unknown parameters with estimates:

Multiple regression extends the model:

A coefficient is interpreted while holding the other included predictors constant. This phrase is crucial: regression adjustment does not automatically establish causality. Causal interpretation requires an appropriate design, credible temporal ordering, adequate confounder control, and defensible assumptions.

The coefficient of determination is:

where SSE is the sum of squared residuals and SST is total sum of squares. R² is the proportion of sample variation explained by the fitted model relative to a baseline mean model. It is not the proportion of individual behavior determined by the predictors, nor a guarantee of predictive accuracy on new data. Adjusted R² penalizes unnecessary predictors, while cross-validation evaluates out-of-sample performance.

Important diagnostics include residual-versus-fitted plots, Q–Q plots, leverage, influential observations, nonlinearity, heteroscedasticity, and multicollinearity. Heteroscedasticity affects standard errors; robust standard errors may help with inference, but they do not repair a badly specified mean relationship. Multicollinearity makes individual coefficients unstable even when the model predicts well. Variance inflation factors are diagnostic aids, not universal pass–fail rules.

Categorical predictors can be represented with dummy variables. If a variable has k categories, k−1 indicators are typically entered and one category is the reference group. The coefficient compares a category with that reference, conditional on other predictors.

9. Logistic Regression

Linear regression is unsuitable when the outcome is binary, such as purchase versus no purchase. Logistic regression models the log odds of the event:

where p is the probability of the event. Exponentiating a coefficient gives an odds ratio:

An odds ratio above 1 indicates higher odds as the predictor increases; below 1 indicates lower odds. An odds ratio of 1 indicates no change in odds. Odds are not probabilities: odds equal p/(1−p). A one-unit coefficient effect may produce different probability changes at different baseline probabilities.

Logistic regression assumes independent observations, a correctly specified relationship between continuous predictors and the logit, and adequate information for the number of predictors. Separation can produce unstable or infinite estimates. Analysts should report confidence intervals, predicted probabilities, discrimination measures such as the area under the ROC curve, and calibration—not merely classification accuracy. A classification threshold reflects costs of false positives and false negatives and should be chosen in relation to the marketing decision.

10. Statistical Significance, Effect Size, and Managerial Significance

Statistical significance depends partly on sample size. A tiny effect can be significant in a very large sample, while a strategically important effect may be nonsignificant in a small sample. Therefore, responsible reporting combines:

1. The estimated effect.
2. A confidence interval.
3. The p-value or compatible test result.
4. The sample size and design.
5. Practical consequences and uncertainty.

Suppose an email experiment estimates a 0.4 percentage-point increase in conversion, with p < .05. The result may be statistically credible, but its value depends on list size, contribution margin, implementation cost, and whether the increase persists. Conversely, a 3-point increase with p = .08 may merit further study if the sample was small and the potential payoff substantial.

Power analysis should occur before data collection. A planning calculation requires a meaningful minimum effect, α, desired power, outcome variability or baseline rate, and design information. Post-hoc power calculations based only on an observed p-value add little; confidence intervals and prospective planning are more informative.

11. Common Analytical Errors

Confusing association with causation. Observational relationships may reflect confounding or selection.

Treating p > .05 as proof of no effect. The correct conclusion is usually that the evidence was insufficient to reject H₀.

Ignoring effect size. Significance without magnitude is poor managerial guidance.

Running many tests without adjustment. Searching across outcomes, segments, and specifications increases false discoveries.

Using the wrong unit of analysis. Repeated observations from one customer are not independent observations.

Dichotomizing continuous variables unnecessarily. Converting income or satisfaction into high/low categories loses information and power.

Extrapolating beyond the data. A relationship observed within the sampled age range may not apply outside it.

Overfitting. A model with too many predictors can describe noise and perform poorly on new data.

Deleting outliers mechanically. Investigate whether an observation is an error, a valid extreme customer, or evidence of model misspecification.

Equating statistical adjustment with fairness. Sensitive attributes may be relevant for auditing unequal outcomes, but their use in targeting or pricing can create legal and ethical risks.

12. Applied Case: Evaluating a Loyalty-App Campaign

Background

This illustrative case concerns a grocery chain testing whether a personalized loyalty-app campaign increases monthly spending. Customers were randomly assigned to a campaign group or a control group. The primary outcome was spending during the following month; a secondary outcome was whether a customer made at least one purchase.

Problem

Managers wanted to know whether the campaign produced a meaningful financial return, not merely whether a statistical test produced a small p-value.

Analysis and Approach

The analyst first checked balance between groups using descriptive statistics and confidence intervals. Because spending was continuous and assignment was independent, the primary comparison used Welch’s t-test. The binary purchase outcome was examined using a difference in proportions and logistic regression adjusted for pre-campaign purchase frequency. The analyst inspected distributions, missingness, and extreme spending values before modeling.

The estimated mean spending difference was $2.80 per customer, with a 95% confidence interval from $0.90 to $4.70 and p = .004. The result supports a positive average effect under the experimental design. However, the business case depends on campaign cost. If the incremental contribution margin is $1.20 per dollar spent and campaign cost is $2.00 per customer, the estimated incremental contribution is:

This is positive at the point estimate, but uncertainty in the effect should be propagated into the financial decision. The lower confidence-limit scenario gives:

Thus, although the average estimate is attractive, the campaign may lose money under a conservative scenario. The team should consider a larger replication, customer-segment heterogeneity, and persistence of the effect.

Outcome

Management approved a controlled rollout rather than an unrestricted launch. The second phase prespecified the primary outcome, monitored incremental profit, and evaluated whether effects differed by customer tenure without conducting uncontrolled subgroup fishing.

Learning Points

The randomized design supports a stronger causal interpretation than an observational correlation. The confidence interval is essential for financial planning. Statistical significance did not by itself establish profitability, and a managerial decision required costs, margins, scalability, and uncertainty.

13. Ethics, Governance, and Contemporary Practice

Marketing analysts have responsibilities beyond technical correctness. They should document data provenance, sampling procedures, exclusions, transformations, model specifications, and deviations from the analysis plan. Personal data must be processed lawfully, minimally, and transparently under applicable privacy and consumer-protection requirements. Sensitive characteristics require particular care; models can reproduce historical discrimination even when sensitive variables are omitted through proxy variables.

AI-assisted analytics can accelerate coding, model comparison, and report drafting, but generated analyses require human verification. Analysts should not upload confidential customer records to unauthorized systems. Automated model selection can encourage p-hacking or overfitting unless validation data, preregistration, and reproducible workflows are used. Explainability, audit trails, access controls, and monitoring for drift are important when predictions influence offers, credit, prices, or customer treatment.

Best practice is to combine statistical expertise with domain knowledge: define the decision first, select the estimand and design, inspect data before modeling, test assumptions, report uncertainty, validate predictions, and communicate limitations plainly.

Chapter Summary

Statistical inference uses sample data to learn about populations while recognizing uncertainty. Hypothesis tests compare evidence with a null model, but p-values do not measure effect size or managerial value. Confidence intervals communicate estimation precision. Chi-square tests address categorical association; t-tests compare one or two means; ANOVA compares several means and requires appropriate post-hoc analysis; correlation describes linear association; linear regression predicts continuous outcomes; and logistic regression models binary outcomes through log odds and odds ratios.

Every method depends on assumptions concerning independence, measurement, distributional form, variance, model specification, and sample size. Diagnostics, effect sizes, power, validation, and careful design are essential. Statistical association does not automatically imply causation, and statistical significance does not guarantee profitability or strategic importance. Ethical analysis requires privacy protection, transparency, fairness assessment, reproducibility, and responsible use of AI.

Key Terms

Population: Complete set of units relevant to a study.

Sample: Subset of a population that is observed.

Parameter: Numerical characteristic of a population.

Statistic: Numerical characteristic calculated from a sample.

Standard error: Estimated sampling variability of a statistic.

Confidence interval: Range produced by a procedure intended to capture a population parameter at a stated long-run rate.

Null hypothesis: Default claim tested by a statistical procedure.

p-value: Probability, under H₀ and the model, of a result at least as extreme as observed.

Type I error: Rejecting a true null hypothesis.

Type II error: Failing to reject a false null hypothesis.

Statistical power: Probability of detecting a specified effect under stated conditions.

Chi-square test: Test of association between categorical variables.

t-test: Test involving one or two means.

ANOVA: Analysis that compares multiple group means through variance components.

Correlation: Standardized measure of linear association.

Regression: Model relating an outcome to one or more predictors.

Odds ratio: Multiplicative change in odds associated with a predictor change.

Effect size: Quantitative magnitude of a difference or association.

Multicollinearity: Strong dependence among predictors in a regression model.

Managerial significance: Practical importance of a result for a decision.

Review Questions

Short Answer Questions

1. What is the difference between a parameter and a statistic?
2. What does a p-value represent, and what does it not represent?
3. When is a paired-samples t-test preferable to an independent-samples t-test?
4. Why does a significant ANOVA require post-hoc comparisons?
5. What is the interpretation of an odds ratio below 1?

Long Answer Questions

1. Explain the relationship among statistical significance, confidence intervals, effect size, and managerial significance.
2. Compare the assumptions and applications of chi-square tests, t-tests, and ANOVA.
3. Explain why correlation and regression do not automatically establish causation.
4. Discuss the main assumptions and diagnostic procedures for multiple linear regression.
5. Explain how statistical power is affected by sample size, effect size, variability, and significance level.

Analytical / Application Questions

1. A retailer finds that customers exposed to a campaign spend more than unexposed customers in an observational dataset. Design an analysis that addresses confounding and explain what evidence would support a causal claim.
2. A logistic model reports an odds ratio of 1.40 for app engagement. Explain how you would communicate this result to a manager and why it should not be described as a 40% increase in probability.
3. An email experiment produces a statistically significant 0.2 percentage-point conversion increase. Identify the additional information required before recommending a full rollout.

Multiple Choice Questions with Answers

Question 1

Which statement best describes a p-value?

A. The probability that the null hypothesis is true

B. The probability of observing evidence at least as extreme as the sample result, assuming the null hypothesis and model are appropriate

C. The probability that the alternative hypothesis is true

D. The proportion of variation explained by a regression model

Correct Answer: B. The probability of observing evidence at least as extreme as the sample result, assuming the null hypothesis and model are appropriate

Question 2

A researcher compares the same customers’ satisfaction scores before and after a campaign. Which test is most appropriate?

A. Independent-samples t-test

B. Chi-square test of independence

C. Paired-samples t-test

D. One-way ANOVA

Correct Answer: C. Paired-samples t-test

Question 3

In a chi-square test of independence, expected counts are calculated from:

A. The grand mean only

B. The row and column totals under independence

C. The standard deviations of the variables

D. The regression coefficients

Correct Answer: B. The row and column totals under independence

Question 4

A significant one-way ANOVA indicates that:

A. Every group mean differs from every other group mean

B. At least one population mean differs from another

C. The largest group has the highest mean

D. The variables are causally related

Correct Answer: B. At least one population mean differs from another

Question 5

Which statement about Pearson correlation is correct?

A. It proves that one variable causes the other

B. It measures the strength and direction of a linear relationship

C. It can only take positive values

D. It is appropriate for any categorical variable

Correct Answer: B. It measures the strength and direction of a linear relationship

Question 6

In a multiple linear regression, a slope coefficient is interpreted as the expected change in the outcome:

A. When all other included predictors are held constant

B. When the sample size is held constant

C. Only when the p-value equals zero

D. Without regard to the measurement units

Correct Answer: A. When all other included predictors are held constant

Question 7

Logistic regression is especially appropriate when the dependent variable is:

A. A binary outcome such as purchase versus no purchase

B. A continuous revenue amount

C. A nominal variable with no event definition

D. A standard deviation

Correct Answer: A. A binary outcome such as purchase versus no purchase

Question 8

If β = 0.693 in a logistic regression, the corresponding odds ratio is approximately:

A. 0.50

B. 0.693

C. 1.00

D. 2.00

Correct Answer: D. 2.00

Question 9

Which change generally increases statistical power, all else equal?

A. Reducing the sample size

B. Increasing measurement variability

C. Increasing the sample size

D. Reducing the true effect to zero

Correct Answer: C. Increasing the sample size

Question 10

Why should managerial significance be considered alongside statistical significance?

A. Because p-values always contain calculation errors

B. Because a statistically detectable effect may be too small to justify implementation costs

C. Because statistical tests cannot use samples

D. Because confidence intervals eliminate uncertainty

Correct Answer: B. Because a statistically detectable effect may be too small to justify implementation costs