Statistical Reasoning and Inference

Introduction

Data science is not only concerned with describing observed data. It also seeks to learn from a sample and make defensible statements about a wider population. Statistical reasoning provides the framework for doing this while explicitly recognising uncertainty. It helps a data scientist answer questions such as: How likely is an observed result? How precise is an estimate? Is an apparent difference compatible with random variation? Does one variable predict another, and can the relationship be interpreted causally?

This chapter develops the foundations of probability, sampling, confidence intervals, hypothesis testing, correlation and introductory regression. It also emphasises a central principle: association is not the same as causation. Statistical conclusions must be interpreted in relation to study design, measurement quality, assumptions and the practical context.

Learning Objectives

By the end of this chapter, you should be able to:

• distinguish populations, samples, parameters and statistics;
• use basic probability concepts and conditional probability;
• explain sampling variation and common sampling methods;
• construct and interpret confidence intervals;
• formulate and evaluate null and alternative hypotheses;
• interpret p-values without treating them as probabilities that hypotheses are true;
• explain correlation and calculate and interpret Pearson’s correlation coefficient;
• fit and interpret a simple linear regression model;
• distinguish association from causation;
• identify statistical, ethical and practical limitations in data analysis.

1. Statistical Reasoning: From Data to Evidence

A population is the complete set of units relevant to a question. A sample is a subset observed in practice. A parameter is a numerical characteristic of a population, such as the population mean μ. A statistic is a numerical characteristic calculated from a sample, such as the sample mean x̄.

Because a sample is usually not identical to its population, statistics vary from sample to sample. This variation is called sampling variability. Statistical inference uses probability to quantify that variability and to assess how strongly data support a claim.

Two kinds of uncertainty should be distinguished:

1. Sampling uncertainty arises because only part of a population is observed.
2. Measurement or process uncertainty arises from inaccurate instruments, inconsistent recording, changing conditions or imperfect definitions.

A narrow confidence interval does not correct for biased sampling or poor measurement. It describes precision under the assumed sampling process, not the total quality of an investigation.

Descriptive and Inferential Statistics

Descriptive statistics summarise observed data using tables, graphs, means, medians, proportions and measures of spread. Inferential statistics use sample data to estimate population quantities or evaluate claims about a population.

For example, the average delivery time in 200 observed orders is descriptive. Estimating the average delivery time for all future orders is inferential and requires assumptions about how the 200 orders were selected and whether future orders are comparable.

2. Probability Foundations

Probability is a numerical measure between 0 and 1 assigned to an event. An event with probability 0 is impossible under a stated model; an event with probability 1 is certain under that model.

For an event A, the complement is the event that A does not occur:

The addition rule for two events is:

Here, A ∪ B means A or B, while A ∩ B means both A and B. If A and B are mutually exclusive, they cannot occur together, so P(A ∩ B) = 0.

Conditional probability measures the probability of A given that B has occurred:

Events A and B are independent when knowing that B occurred does not change the probability of A:

Independence is stronger than merely being unrelated in a particular sample. It is an assumption about the probability structure.

Bayes’ Theorem

Bayes’ theorem updates a probability after new evidence is observed:

P(A) is the prior probability, P(B | A) is the likelihood of the evidence if A is true, and P(A | B) is the updated or posterior probability. Bayes’ reasoning is important in medical screening, fraud detection, classification and risk analysis.

Worked Example: Screening

Suppose 1% of transactions are fraudulent. A screening system flags 90% of fraudulent transactions and incorrectly flags 5% of legitimate transactions. Let F denote fraud and + denote a flag.

P(F) = 0.01, P(+ | F) = 0.90, and P(+ | Fᶜ) = 0.05. Since P(Fᶜ) = 0.99:

Therefore:

Only about 15.4% of flagged transactions are expected to be fraudulent. This result illustrates the importance of base rates: a highly sensitive system can still produce many false positives when the target condition is uncommon.

3. Random Variables and Probability Distributions

A random variable assigns numerical values to outcomes. A discrete random variable has countable possible values, whereas a continuous random variable can take values across an interval.

The expected value of a discrete random variable X is:

The variance is:

where μ = E(X). The standard deviation is σ = √Var(X), measured in the same units as X.

A common model for continuous measurements is the normal distribution, described by a mean μ and standard deviation σ. The standardised value, or z-score, is:

A z-score states how many standard deviations an observation lies above or below the mean. The normal model is useful in many settings, but data should not automatically be assumed to be normal. Skewness, outliers and bounded measurements can make it inappropriate.

4. Sampling and Sampling Distributions

A sampling method determines which units enter a sample. Common methods include:

A sampling distribution is the probability distribution of a statistic over repeated samples of the same size and design. It is not the distribution of individual observations. The standard error measures the spread of a statistic’s sampling distribution.

For a random sample of n independent observations with population standard deviation σ, the standard error of the sample mean is:

When σ is unknown, it is commonly estimated by:

where s is the sample standard deviation. The formula assumes independent observations and a sampling design for which the simple random-sample model is reasonable. Increasing n reduces standard error at a rate proportional to 1 ÷ √n; quadrupling the sample size approximately halves it.

The central limit theorem states, under broad conditions, that the sampling distribution of the mean becomes approximately normal as sample size increases, even when individual observations are not normally distributed. The theorem does not remove problems caused by dependence, extreme outliers or a biased sample.

5. Estimation and Confidence Intervals

A point estimate is a single value used to estimate a parameter. An interval estimate gives a range of plausible values, together with a confidence level.

For a population mean with known σ, a two-sided 95% confidence interval is:

When σ is unknown and the population is reasonably normal, or the sample is sufficiently large, a t-based interval is commonly used:

Here, t* is the critical value from the t distribution with n − 1 degrees of freedom. The t distribution has heavier tails than the standard normal distribution, especially for small samples.

For a population proportion, with sample proportion p̂ = x ÷ n, a large-sample approximate interval is:

This approximation is most reliable when the expected counts of successes and failures are sufficiently large. For small samples or extreme proportions, exact or improved interval methods may be preferable.

Correct Interpretation

A 95% confidence procedure produces intervals that capture the fixed population parameter in approximately 95% of repeated samples, assuming the procedure’s conditions hold. It is not strictly correct to say that there is a 95% probability that the already-calculated fixed parameter lies in this particular interval. In practical communication, one may say that the interval provides a plausible range for the parameter under the stated assumptions.

Worked Example: Mean Delivery Time

A random sample of 64 deliveries has x̄ = 42 minutes and s = 8 minutes. Using a normal critical value as an approximation, the estimated standard error is:

A 95% approximate interval is:

The estimate is 42 minutes, and the margin of error is 1.96 minutes. This conclusion assumes that the sample is representative, observations are sufficiently independent and the mean is an appropriate summary.

The margin of error depends on confidence level, variability and sample size. Higher confidence produces a wider interval; greater variability produces a wider interval; larger samples generally produce a narrower interval.

6. Hypothesis Testing

Hypothesis testing evaluates data against a null model. The null hypothesis, H₀, represents a benchmark such as no difference or no association. The alternative hypothesis, H₁ or Hₐ, represents the research claim being assessed.

A test may be:

• two-sided: the parameter differs from a specified value;
• one-sided: the parameter is greater than or less than a specified value.

The direction must be chosen before examining the results. A one-sided test should not be selected merely because it gives a smaller p-value.

A test statistic measures how far the sample result is from the null value relative to its standard error. For a mean with known σ:

where μ₀ is the mean specified by H₀. With unknown σ, a t statistic is generally used:

A p-value is the probability, assuming H₀ is true and the analysis assumptions hold, of obtaining a result at least as extreme as the observed result in the direction defined by the test. It is not the probability that H₀ is true, nor is it the probability that the result occurred by chance in an unrestricted sense.

A significance level α is selected in advance, commonly 0.05. If p ≤ α, the result is described as statistically significant and H₀ is rejected. If p > α, H₀ is not rejected. Failure to reject H₀ does not prove that H₀ is true.

Errors and Power

A Type I error rejects a true H₀. Its probability is controlled approximately by α under the test assumptions. A Type II error fails to reject a false H₀. The power of a test is:

Power tends to increase with larger samples, larger true effects, lower variability and a higher α. Statistical significance should be considered alongside effect size, confidence intervals, costs and practical importance.

Multiple Testing and Practical Significance

Testing many hypotheses increases the chance of at least one false positive. Analysts should pre-specify primary outcomes, control false discovery where appropriate and distinguish exploratory findings from confirmatory findings. A very small effect can be statistically significant in a very large sample, while a meaningful effect can fail to reach significance in a small sample.

7. Correlation

Correlation measures the direction and strength of a linear association between two quantitative variables. Pearson’s sample correlation coefficient is:

The value lies between −1 and +1. Positive values indicate that larger values of one variable tend to accompany larger values of the other; negative values indicate an inverse linear association. A value near 0 indicates little linear association, but it does not rule out a nonlinear relationship.

Correlation is sensitive to outliers and can be misleading when data contain distinct subgroups. A scatterplot should be examined before interpreting r. Correlation is symmetric: r between X and Y is the same as r between Y and X. It does not identify a response variable and does not establish causation.

8. Simple Linear Regression

Simple linear regression models the mean of a response variable Y as a linear function of a predictor X:

The fitted sample equation is:

where ŷ is the predicted response, b₀ is the intercept and b₁ is the estimated slope. The least-squares slope is:

and the intercept is:

The slope represents the estimated change in the mean response for a one-unit increase in X. The intercept is the predicted response when X = 0; it may have no practical meaning if zero lies outside the observed range.

A residual is:

A useful summary is R², the proportion of variation in Y explained by the fitted linear model in the sample. In simple linear regression with an intercept, R² = r². R² does not prove causation and does not guarantee useful predictions outside the observed range.

Important assumptions include linearity of the conditional mean, independent errors, approximately constant error variance and, for conventional small-sample inference, approximately normal errors. Extrapolation beyond the data range is especially risky.

Worked Example: Training and Assessment Score

Suppose a fitted model predicts an assessment score from hours of training:

For 6 hours of training:

The slope means that the model estimates an average increase of 4.5 score points per additional hour in the observed setting. It does not establish that forcing a learner to complete one more hour will cause a 4.5-point increase. Learners who train more may differ in motivation, prior knowledge or access to support.

9. Association, Causation and Study Design

An association is a pattern in which variables vary together. A causal effect concerns what would happen to the same units under different interventions or exposures. Three common explanations for an association are:

1. X causes Y.
2. Y causes X.
3. A third variable, Z, causes or influences both.

Confounding occurs when a third variable is related to both the predictor and outcome and is not adequately controlled. For example, ice-cream sales and drowning incidents may rise together because hot weather increases both. The association does not mean ice cream causes drowning.

Randomised experiments provide the strongest basis for causal conclusions because random assignment tends to balance measured and unmeasured confounders. In observational studies, researchers can adjust for measured variables using stratification, matching or regression, but unmeasured confounding may remain.

Causal interpretation also requires attention to temporality, interference, selection bias, missing data, measurement error and the precise definition of the intervention. Statistical adjustment is not automatically valid: controlling for a mediator or collider can introduce bias.

10. Applied Case: Evaluating a Customer-Support Intervention

Background

A software company introduces an AI-assisted response tool and wants to know whether it reduces average customer-resolution time.

Problem

Before implementation, the average resolution time was 18 hours from a historical sample of 1,000 tickets. After implementation, a sample of 250 tickets has a mean of 16.8 hours and a standard deviation of 6 hours. The company asks whether the reduction demonstrates that the tool caused improvement.

Analysis

The observed difference is 16.8 − 18 = −1.2 hours. An approximate standard error for the post-implementation mean is 6 ÷ √250 ≈ 0.379 hours, but comparing this mean with a historical benchmark also requires uncertainty in the benchmark and consideration of changes over time. A one-sample test against 18 hours would be:

This would provide evidence against a null mean of 18 hours under the one-sample assumptions. However, it does not by itself prove that the AI tool caused the reduction. Ticket complexity, staffing, seasonal demand, policy changes or selective use of the tool may have changed.

Approach and Solution

A stronger design would randomly assign eligible tickets, or support agents, to use the tool or a control process during the same period. The analysis should predefine the primary outcome, record ticket complexity, monitor missing and excluded cases, and report the estimated difference with a confidence interval. If randomisation is impossible, a matched comparison group and a difference-in-differences design may improve the analysis, although causal assumptions must be stated and assessed.

The company should also examine secondary outcomes: customer satisfaction, repeat contacts, escalation rates, error rates and employee workload. A faster response that reduces quality would not necessarily be an improvement.

Outcome and Learning Points

The initial data suggest an association between implementation and shorter resolution time, but the evidence is insufficient for a strong causal claim. A controlled evaluation would provide more credible evidence. The case illustrates that statistical significance, operational significance and causal validity are separate questions.

11. Ethical and Professional Considerations

Statistical work affects people and organisational decisions. Responsible practice includes:

• explaining uncertainty rather than presenting estimates as facts;
• reporting all important outcomes and relevant analyses;
• avoiding misleading graphs, truncated axes and selective comparisons;
• protecting privacy and limiting access to identifiable data;
• checking whether sampling or measurement disadvantages particular groups;
• documenting assumptions, exclusions, transformations and software versions;
• distinguishing exploratory analysis from pre-specified testing;
• avoiding automated decisions that cannot be justified or audited.

AI systems can accelerate modelling and generate plausible statistical explanations, but they may reproduce sampling bias, fabricate interpretations or conceal assumptions. Human review, reproducible code, data provenance and domain expertise remain essential. In regulated settings, analysts must also follow applicable privacy, data-protection and sector-specific requirements.

12. Best Practices and Common Misinterpretations

A sound inferential workflow is:

1. Define the population, estimand and decision problem.
2. Inspect data quality, missingness, measurement and sampling.
3. Visualise distributions and relationships.
4. Choose a method whose assumptions match the design and variables.
5. Estimate effects and report confidence intervals.
6. Use hypothesis tests as supporting evidence, not as the entire conclusion.
7. Check sensitivity to reasonable alternative assumptions.
8. Interpret practical importance and causal validity.
9. Communicate limitations and preserve reproducibility.

Common errors include saying “the p-value is the probability the null hypothesis is true,” treating a confidence level as a probability about one fixed interval, interpreting correlation as causation, ignoring multiple testing, reporting only significant outcomes, and using regression to extrapolate far beyond observed data.

Chapter Summary

Statistical inference uses probability and sampling theory to learn about populations from samples. Sampling variability explains why statistics differ across samples, while standard errors quantify the typical variation of estimates. Confidence intervals express precision under stated assumptions. Hypothesis tests compare observed evidence with a null model; p-values measure compatibility with that model and are not probabilities that hypotheses are true. Correlation describes linear association, while regression models a conditional mean and supports prediction within an appropriate range. Neither establishes causation by itself. Causal conclusions depend strongly on study design, randomisation, confounding control and temporal reasoning. Ethical analysis requires transparency, privacy protection, fair measurement, reproducibility and honest communication of uncertainty.

Key Terms

Population: The complete set of units relevant to a study.

Sample: A subset of a population that is observed.

Parameter: A numerical characteristic of a population.

Statistic: A numerical quantity calculated from a sample.

Sampling variability: Variation in a statistic across possible samples.

Standard error: The standard deviation of a statistic’s sampling distribution.

Random variable: A variable whose value is determined by a random outcome.

Conditional probability: The probability of an event given another event.

Independence: A condition in which occurrence of one event does not change the probability of another.

Confidence interval: An interval produced by a procedure designed to capture a parameter at a stated long-run rate.

Null hypothesis: A benchmark claim evaluated by a statistical test.

Alternative hypothesis: A claim supported when evidence is sufficiently inconsistent with the null model.

P-value: The probability, under H₀, of data at least as extreme as those observed.

Type I error: Rejecting a true null hypothesis.

Power: The probability of rejecting a false null hypothesis under specified conditions.

Correlation: A measure of the direction and strength of linear association.

Regression: A model describing the relationship between a response and one or more predictors.

Residual: The observed response minus its fitted or predicted value.

Confounding: Distortion of an association by a third variable related to the variables of interest.

Extrapolation: Prediction outside the range of observed predictor values.

Review Questions

Short Answer Questions

1. Distinguish a parameter from a statistic and give one example of each.
2. What does a standard error measure?
3. State the correct interpretation of a p-value.
4. Why can a narrow confidence interval still be misleading?
5. What does the slope in a simple linear regression model represent?

Long Answer Questions

1. Explain how sampling design influences the validity of statistical inference. Compare simple random, stratified, cluster and convenience sampling.
2. Explain confidence intervals, including the meaning of confidence level, margin of error and the effects of sample size and variability.
3. Describe the logic of hypothesis testing and distinguish Type I error, Type II error, statistical significance and practical significance.
4. Explain Pearson correlation and simple linear regression, including their interpretation, assumptions and limitations.
5. Discuss why association does not establish causation. Use confounding and randomised experiments in your answer.

Analytical / Application Questions

1. A fraud detector flags 8% of legitimate transactions and detects 95% of fraudulent transactions. Fraud occurs in 2% of transactions. Calculate the probability that a flagged transaction is fraudulent and explain why the result may be lower than expected.
2. A study reports a statistically significant correlation of r = −0.72 between screen time and sleep duration among students. Identify at least three reasons why this result cannot alone establish that screen time causes reduced sleep.
3. A company observes that average productivity increased after remote-work software was introduced. Design a stronger evaluation than a simple before-and-after comparison, identifying the treatment, comparison, outcome and major threats to validity.

Multiple Choice Questions with Answers

Question 1

A sample mean is best described as which of the following?

A. A population parameter

B. A sample statistic

C. A probability distribution

D. A causal effect

Correct Answer: B. A sample statistic

Question 2

If P(A) = 0.3 and P(B) = 0.4 and A and B are independent, what is P(A ∩ B)?

A. 0.12

B. 0.70

C. 0.10

D. 0.75

Correct Answer: A. 0.12

Question 3

What does a 95% confidence procedure mean in repeated sampling?

A. There is a 95% probability that the fixed parameter changes

B. 95% of the observations lie inside the interval

C. About 95% of intervals produced by the procedure capture the parameter

D. The null hypothesis is true with probability 0.95

Correct Answer: C. About 95% of intervals produced by the procedure capture the parameter

Question 4

A p-value is calculated under which assumption?

A. The alternative hypothesis is definitely true

B. The null hypothesis and test assumptions hold

C. The sample contains no variability

D. The observed result is impossible

Correct Answer: B. The null hypothesis and test assumptions hold

Question 5

If sample size increases from 100 to 400 while other conditions remain constant, the standard error of a mean approximately:

A. Doubles

B. Quadruples

C. Halves

D. Does not change

Correct Answer: C. Halves

Question 6

Which value indicates the strongest negative linear association?

A. −0.91

B. −0.08

C. +0.08

D. +0.91

Correct Answer: A. −0.91

Question 7

In the regression equation ŷ = 10 + 3x, what does 3 represent?

A. The predicted value when x is zero

B. The estimated change in the response for a one-unit increase in x

C. The residual standard deviation

D. The correlation coefficient

Correct Answer: B. The estimated change in the response for a one-unit increase in x

Question 8

Which situation most directly supports a causal interpretation?

A. A strong correlation in a convenience sample

B. A before-and-after comparison with no control group

C. A randomised experiment with well-measured outcomes

D. A regression model with a high R² only

Correct Answer: C. A randomised experiment with well-measured outcomes

Question 9

A Type I error occurs when a researcher:

A. Fails to reject a false null hypothesis

B. Rejects a true null hypothesis

C. Accepts a true alternative hypothesis

D. Uses a sample larger than necessary

Correct Answer: B. Rejects a true null hypothesis

Question 10

Why is extrapolation in regression potentially dangerous?

A. It always makes the slope equal to zero

B. Predictions outside the observed range rely on untested behaviour of the relationship

C. It eliminates all residuals

D. It guarantees a statistically significant result

Correct Answer: B. Predictions outside the observed range rely on untested behaviour of the relationship