Categorical Data Analysis and Analysis of Variance

Introduction

Business analysts frequently work with outcomes recorded as categories—such as retained/churned, paid/unpaid, or product type—and with numerical outcomes compared across several groups. Two core methods address these situations: chi-square procedures for categorical data and one-way analysis of variance (ANOVA) for comparing means across multiple groups.

These methods are not simply calculation techniques. They support decisions such as whether customer preferences match a planned distribution, whether purchase behaviour differs by region, and whether one marketing campaign produces a higher average conversion value than others. Sound analysis requires a suitable research question, correctly structured data, assumption checking, effect-size interpretation, and translation of statistical evidence into an action.

Learning Objectives

By the end of this chapter, you should be able to:

• distinguish categorical and numerical variables and select an appropriate test;
• conduct and interpret chi-square goodness-of-fit and independence tests;
• explain observed, expected, marginal, and conditional frequencies;
• conduct a one-way ANOVA and interpret its F statistic and p-value;
• check assumptions and identify suitable remedies;
• interpret effect sizes and post-hoc comparisons; and
• convert statistical findings into responsible business recommendations.

1. Data Structures and Statistical Questions

A categorical variable places observations into non-overlapping groups. Nominal categories have no natural order, such as payment method. Ordinal categories have an order, such as low, medium, and high satisfaction. A numerical variable measures an amount, such as revenue or delivery time.

The choice of method depends on both the outcome and the question:

A null hypothesis represents the reference position tested by the data. A p-value is the probability, assuming the null hypothesis is true and the model assumptions hold, of obtaining a result at least as inconsistent with that null as the observed result. It is not the probability that the null hypothesis is true, nor is it a measure of practical importance.

A significance level, commonly α = 0.05, is selected before analysis. Rejecting a null hypothesis means that the data provide sufficient evidence against it at the chosen level; it does not prove that every alternative explanation is false.

2. Chi-Square Goodness-of-Fit Test

Purpose and hypotheses

The goodness-of-fit test evaluates whether observed counts across one categorical variable are consistent with specified population proportions.

Suppose a retailer expects 50% of orders to use cards, 30% digital wallets, and 20% bank transfer. The null hypothesis is that the population proportions are p₁ = 0.50, p₂ = 0.30, and p₃ = 0.20. The alternative is that at least one population proportion differs from its specified value.

The test uses counts, not percentages alone. If the expected proportions are pᵢ and the total sample size is n, the expected count for category i is:

where Eᵢ is the expected count and pᵢ is the hypothesised proportion. The chi-square statistic is:

Here Oᵢ is the observed count, Eᵢ is the expected count, and k is the number of categories. Under the null hypothesis, with independent observations and adequate expected counts, the statistic is approximately chi-square distributed with:

degrees of freedom. The test is appropriate when categories are mutually exclusive and collectively exhaustive, observations are independent, and expected counts are sufficiently large. A common practical guideline is that all expected counts should be at least 5; when this is not satisfied, categories may need principled combining or an exact method may be considered.

Worked example: service-channel demand

A company records 200 support requests: 116 by web chat, 54 by telephone, and 30 by email. Management’s planning proportions are 50%, 30%, and 20% respectively.

Expected counts are 100, 60, and 40. Therefore:

There are 3 − 1 = 2 degrees of freedom. At α = 0.05, the critical value for 2 degrees of freedom is approximately 5.991. Since 5.66 < 5.991, the result is not statistically significant at the 5% level. The company should not conclude that demand differs from the planning distribution, although the observed pattern may still warrant monitoring. A conclusion should report the statistic, degrees of freedom, and p-value produced by the chosen software rather than relying only on a critical-value comparison.

Diagnostic interpretation

The total chi-square statistic is a sum of category contributions. A category with a large absolute difference between Oᵢ and Eᵢ, relative to Eᵢ, contributes more evidence against the null. Standardised residuals can help identify such categories, but they should be examined only after the overall test and interpreted with attention to multiple comparisons.

3. Chi-Square Test of Independence

Purpose and hypotheses

The independence test examines whether two categorical variables are associated in a population. For example, an analyst may investigate whether subscription status is related to customer age band.

The null hypothesis states that the variables are independent. The alternative states that they are associated. Independence means that the distribution of one variable is the same across the categories of the other; it does not mean that the variables have no business relevance.

Data are arranged in an r × c contingency table. If the observed count in row i and column j is Oᵢⱼ, the expected count under independence is:

The test statistic is:

The degrees of freedom are:

The observations should be independent, each observation should contribute to one cell only, and expected counts should be adequate. A chi-square test detects association, not causation. For example, a relationship between region and churn does not demonstrate that region causes churn.

Worked example: subscription and age band

A sample of 300 customers is classified as either subscribed or not subscribed and placed into three age bands:

For the 18–29 subscribed cell, the expected count is (100 × 150) / 300 = 50. The expected counts for subscribed customers are 50 in each row, and the same is true for non-subscribed customers. The statistic is:

The degrees of freedom are (3 − 1)(2 − 1) = 2. Since 13.92 exceeds 5.991, the association is statistically significant at the 5% level. Subscription rates are 36%, 60%, and 54% across the three age bands. Management may investigate why the youngest group subscribes less, but should consider confounding factors such as income, product awareness, and acquisition channel before changing strategy.

Strength and direction of association

Statistical significance depends partly on sample size. A large sample can make a small association significant, while a small sample can fail to detect a meaningful pattern. For an r × c table, Cramér’s V is a commonly used effect-size measure:

where n is the grand total. V ranges from 0 to 1, with larger values indicating stronger association. There is no universal meaningful cutoff: context, category structure, and decision consequences matter. For a 2 × 2 table, the phi coefficient is equivalent in magnitude to Cramér’s V.

4. One-Way Analysis of Variance

Why ANOVA is used

Suppose a firm compares average order value across four advertising channels. Conducting many separate two-sample tests increases the chance of at least one false positive. One-way ANOVA tests all group means in a single overall procedure.

The term “one-way” means that there is one categorical explanatory factor. If the factor has k groups, the hypotheses are:

• H₀: μ₁ = μ₂ = … = μₖ;
• H₁: at least one population mean differs.

ANOVA does not identify which means differ. If the overall test is significant, planned or post-hoc comparisons are needed.

Partitioning variation

ANOVA separates total variation in observations into variation between group means and variation within groups. Let xᵢⱼ be observation j in group i, x̄ᵢ be the mean of group i, x̄ be the grand mean, nᵢ be the group size, and N be the total sample size.

The identity is:

Degrees of freedom are dfᵦ = k − 1 for between groups, dfᵥ = N − k for within groups, and dfₜ = N − 1 in total. Mean squares divide sums of squares by their degrees of freedom:

The F statistic is:

If group means are equal and assumptions hold, MSB and MSW estimate the same error variance and F tends to be near 1. A large F indicates that between-group variation is large relative to within-group variation.

Worked example: delivery time by warehouse

An analyst records delivery times in days for three warehouses:

The group means are 5, 7, and 9, and the grand mean is 7. The between-group sum of squares is:

Within each group, deviations from its mean are −1, 0, and 1, so each group contributes 2 to SSW. Thus SSW = 6. With k = 3 and N = 9:

The F statistic has 2 and 6 degrees of freedom. The 5% critical value is approximately 5.14, so the overall result is significant at α = 0.05. The data indicate that at least one warehouse has a different mean delivery time. Because the group means are ordered and the sample is very small, the analyst should still check the data, process conditions, and assumptions before making an operational decision.

5. Assumptions and Robust Practice

ANOVA relies on:

1. Independence: observations are unrelated, or the sampling and experimental design appropriately supports independence.
2. Numerical outcome: the dependent variable is measured on a scale for which means are meaningful.
3. Approximate normality of residuals within groups: ANOVA is often reasonably robust with moderate, balanced samples, but severe skewness and outliers can distort results.
4. Homogeneity of variance: population variances are approximately equal across groups.

Independence is primarily a design issue and cannot be repaired by a statistical test after data collection. Review sampling procedures, repeated-customer records, time dependence, and clustering.

Use histograms, box plots, and residual plots to identify skewness and outliers. A formal normality test should not be used mechanically: with large samples it may flag trivial deviations, and with small samples it has low power. For variance equality, compare group spreads and consider Levene’s test. If variances differ substantially, Welch’s ANOVA is generally preferable to ordinary ANOVA, especially when group sizes are unequal.

Alternatives include:

• a transformation, such as analysing log revenue when values are strongly right-skewed;
• Welch’s ANOVA for unequal variances;
• the Kruskal–Wallis test for an ordinal or markedly non-normal outcome, recognising that it tests distributional differences rather than exactly the same mean hypothesis; and
• a regression or mixed-effects model when observations are clustered, repeated, or affected by additional predictors.

Do not remove outliers merely because they weaken a desired conclusion. Investigate whether they are errors, legitimate extreme cases, or evidence of a distinct customer segment. Document any correction or exclusion.

6. Post-Hoc Comparisons and Effect Sizes

A significant ANOVA tells us only that at least one mean differs. Testing every pair with ordinary unadjusted t-tests inflates the familywise Type I error rate. Appropriate procedures include Tukey’s honestly significant difference for all pairwise comparisons under suitable variance assumptions, and Games–Howell when variances and group sizes differ. Bonferroni or Holm adjustments can be used for a specified set of comparisons.

A post-hoc result should include the estimated mean difference, confidence interval, adjusted p-value, and direction. “Group A differs from Group B” is less informative than “average order value in A exceeded B by £12, 95% confidence interval £4 to £20, adjusted p < 0.05.”

A common omnibus effect size is eta squared:

Eta squared is the proportion of observed total variation attributable to the factor in this sample. Omega squared is often a less biased population-oriented estimate:

These measures are meaningful only with the design and measurement context. A statistically significant difference may be too small to justify changing prices, staffing, or policy; conversely, a practically important effect may be uncertain in a small sample.

7. From Statistical Evidence to Business Action

A disciplined workflow is:

1. Define the business decision and the population of interest.
2. Identify variable types and formulate hypotheses before inspecting outcomes.
3. Check sampling, coding, independence, missing data, and possible duplication.
4. Produce descriptive tables, percentages, group means, spreads, and visualisations.
5. Select the test and verify assumptions.
6. Report the statistic, degrees of freedom, p-value, confidence intervals, and effect size.
7. Conduct adjusted post-hoc comparisons where appropriate.
8. Assess costs, benefits, uncertainty, fairness, and operational feasibility.
9. Recommend an action, pilot, or further data collection.
10. Monitor outcomes after implementation.

Statistical results should be reported with practical language. For example, “The campaign groups differed significantly” is incomplete. A stronger statement identifies the population, outcome, magnitude, uncertainty, and decision relevance.

Illustrative Case: Regional Promotion and Customer Response

Background and problem

A fictional online retailer tests three promotional messages—value, convenience, and sustainability. It wants to know whether response category differs by region and whether average basket value differs by message.

Analysis

For response category (responded, did not respond), the analyst constructs a region × response contingency table and applies a chi-square independence test. Expected counts are calculated from row and column totals, not from the observed percentages alone. If the test is significant, regional response rates are compared using cell residuals and adjusted follow-up comparisons.

For basket value, the analyst uses one-way ANOVA with message as the factor. Before analysis, box plots reveal two unusually large business orders. The records are verified as legitimate rather than errors. Because the sustainability group has a larger variance, the analyst uses Welch’s ANOVA and Games–Howell comparisons rather than ordinary ANOVA and unadjusted pairwise tests.

Outcome and action

Assume the categorical test finds a significant association: the convenience message performs relatively better in urban regions, while the value message performs relatively better in rural regions. Assume Welch’s analysis finds that the value message produces a mean basket value £9 higher than the convenience message, with a confidence interval excluding zero, but no clear difference from sustainability.

The recommended action is not to deploy one message universally. The retailer should run a controlled, region-specific pilot, estimate incremental profit rather than response alone, and monitor whether differential targeting creates unfair exclusion or misleading claims. The result supports segmentation and further experimentation, not a causal claim about all future customers.

Learning points

The case illustrates that categorical and numerical outcomes require different analyses, that assumption checks influence method selection, and that statistical significance must be combined with effect size, cost, ethics, and implementation evidence.

Limitations, Ethics, and Governance

Chi-square and ANOVA are generally descriptive or inferential tools for specified designs; they do not automatically establish causality. Bias can arise from convenience sampling, selective non-response, inaccurate category definitions, multiple unplanned analyses, and data leakage from future information.

Analysts should protect personal data, minimise collection, control access, and report only the level of detail necessary for the decision. Small cells can make individuals identifiable, especially when tables are cross-tabulated. Suppress or aggregate sensitive cells where appropriate. Avoid using statistical associations to justify discriminatory treatment of protected groups. A significant difference in outcomes may reflect unequal access or historical bias rather than a legitimate basis for automated decisions.

Software can calculate expected counts, F statistics, p-values, residuals, and post-hoc tests efficiently, but it cannot determine whether the sampling design is defensible or whether a business action is ethical. Reproducible scripts, versioned data, documented exclusions, and review of assumptions improve auditability. AI-generated analyses require human verification of variable definitions, formulas, code, and interpretation.

Best-Practice Checklist

• Use counts for chi-square tests and preserve category definitions.
• Check expected frequencies and independence before interpreting p-values.
• Distinguish association from causation.
• For ANOVA, inspect residuals, outliers, balance, and variance equality.
• Use Welch’s ANOVA when unequal variances make ordinary ANOVA unreliable.
• Use multiplicity-adjusted post-hoc comparisons.
• Report effect sizes and confidence intervals, not p-values alone.
• Link conclusions to the population and decision under study.
• Document data preparation, assumptions, exclusions, and software settings.
• Treat statistical evidence as one input into an ethical, economically informed decision.

Chapter Summary

Chi-square goodness-of-fit tests compare observed categorical counts with a specified distribution. Chi-square independence tests assess whether two categorical variables are associated, using expected counts based on marginal totals. Both require independent observations and adequate expected frequencies, and neither establishes causation.

One-way ANOVA compares three or more population means by partitioning total variation into between-group and within-group components. Its F statistic compares the corresponding mean squares. Normality of residuals, equal variances, and independence should be considered, with Welch’s ANOVA, transformations, or non-parametric methods used when appropriate. A significant ANOVA requires adjusted post-hoc comparisons to identify differing groups. Effect sizes, confidence intervals, data quality, ethics, and business consequences are essential for responsible interpretation.

Key Terms

Categorical variable: A variable whose values are categories or labels.

Nominal variable: A categorical variable with no inherent order.

Ordinal variable: A categorical variable with a meaningful order.

Observed count: The number of observations actually recorded in a category or cell.

Expected count: The count predicted by a null hypothesis.

Goodness-of-fit test: A chi-square test comparing observed counts with specified proportions.

Independence: A condition in which the distribution of one categorical variable does not depend on another.

Contingency table: A table showing counts for combinations of categorical variables.

Marginal total: A row or column total in a contingency table.

Chi-square statistic: A weighted sum of squared differences between observed and expected counts.

Degrees of freedom: The number of independent values available to estimate variation under a model.

ANOVA: Analysis of variance for comparing group means.

Factor: The categorical explanatory variable in ANOVA.

Between-group variation: Variation attributable to differences among group means.

Within-group variation: Variation among observations inside groups.

F statistic: The ratio of between-group mean square to within-group mean square.

Post-hoc comparison: An adjusted follow-up comparison identifying which groups differ.

Effect size: A measure of the magnitude of an association or group difference.

Welch’s ANOVA: An ANOVA method that is more robust to unequal variances.

Familywise error rate: The probability of making one or more Type I errors across a set of tests.

Review Questions

Short Answer Questions

1. What is the purpose of a chi-square goodness-of-fit test?
2. How is an expected count calculated in a test of independence?
3. State the null hypothesis of a one-way ANOVA.
4. Why are post-hoc comparisons needed after a significant ANOVA?
5. Name two assumptions that should be considered before using ordinary ANOVA.

Long Answer Questions

1. Explain the logic, calculation, and interpretation of a chi-square test of independence.
2. Describe how ANOVA partitions total variation and forms the F statistic.
3. Compare statistical significance with practical significance in a business analysis.
4. Explain how unequal variances and outliers can affect ANOVA and describe suitable responses.
5. Discuss ethical and governance issues when using categorical associations for business decisions.

Analytical / Application Questions

1. A retailer’s observed channel counts differ from its planned proportions. Design a goodness-of-fit analysis, including hypotheses, assumption checks, reporting, and a possible managerial action.
2. A company finds a significant association between customer segment and cancellation status. Explain what additional analyses are needed before recommending segment-specific retention offers.
3. Three advertising groups have different mean revenues, unequal sample sizes, and visibly unequal variances. Recommend an analysis and reporting strategy, including post-hoc comparisons and effect-size interpretation.

Multiple Choice Questions with Answers

Question 1

A chi-square goodness-of-fit test compares observed counts with:

A. Specified or hypothesised counts

B. The sample mean

C. A regression slope

D. A continuous probability density only

Correct Answer: A. Specified or hypothesised counts

Question 2

For a contingency table, the expected count in a cell is calculated as:

A. Grand total divided by the number of cells

B. Row total × column total ÷ grand total

C. Observed count ÷ grand total

D. Row total + column total

Correct Answer: B. Row total × column total ÷ grand total

Question 3

A chi-square independence test primarily assesses whether two categorical variables are:

A. Normally distributed

B. Equal in variance

C. Associated

D. Measured on an interval scale

Correct Answer: C. Associated

Question 4

For an r × c contingency table, the degrees of freedom are:

A. r + c

B. rc − 1

C. r − c

D. (r − 1)(c − 1)

Correct Answer: D. (r − 1)(c − 1)

Question 5

In one-way ANOVA, the null hypothesis states that:

A. All population group means are equal

B. All observations are identical

C. All group sample sizes are equal

D. All variances are zero

Correct Answer: A. All population group means are equal

Question 6

The ANOVA F statistic is:

A. MSW divided by MSB

B. MSB divided by MSW

C. SSB divided by SST

D. SST divided by SSW

Correct Answer: B. MSB divided by MSW

Question 7

A significant overall ANOVA result establishes that:

A. Every group differs from every other group

B. The largest group mean is the only different mean

C. At least one population mean differs

D. The factor causes the outcome

Correct Answer: C. At least one population mean differs

Question 8

Which method is generally suitable for pairwise comparisons when variances and group sizes differ?

A. Unadjusted repeated t-tests

B. Games–Howell

C. A goodness-of-fit test

D. A test of one proportion

Correct Answer: B. Games–Howell

Question 9

If a chi-square test is statistically significant, the result demonstrates:

A. Causation

B. Perfect prediction

C. An association or departure from the specified distribution

D. That the effect is practically large

Correct Answer: C. An association or departure from the specified distribution

Question 10

Eta squared in one-way ANOVA is calculated as:

A. SSB/SST

B. SSW/SSB

C. MSW/MSB

D. SST/SSB

Correct Answer: A. SSB/SST