Regression, Econometrics and Causal Analysis

Introduction

Managerial decisions frequently involve questions of prediction and explanation. A sales director may ask how advertising expenditure affects revenue; a human-resources manager may examine whether training improves productivity; and a policy leader may want to know whether an intervention caused an observed change. Regression analysis provides a disciplined way to describe relationships among variables, while econometrics combines regression with economic theory, data, and statistical reasoning. Causal analysis goes further by asking whether changing one variable would change another, rather than merely whether the variables are associated.

Regression is therefore both powerful and easily misused. A high correlation does not establish causation, a statistically significant coefficient may have little managerial importance, and a well-fitting model may still violate important assumptions. This chapter develops the foundations of simple and multiple regression, interpretation and diagnostics, dummy variables, panel-data concepts, experiments, and modern causal-inference methods.

Learning Objectives

By the end of this chapter, you should be able to:

1. Explain the purpose and structure of regression and econometric models.
2. Estimate and interpret simple and multiple regression models.
3. Distinguish statistical significance, practical significance, prediction, and causation.
4. Diagnose nonlinearity, omitted variables, multicollinearity, heteroscedasticity, and influential observations.
5. Use dummy variables and interaction terms appropriately.
6. Explain panel data, randomized experiments, difference-in-differences, instrumental variables, and regression discontinuity.
7. Communicate results responsibly, including limitations and ethical implications.

1. Regression as a Model of Conditional Relationships

A regression model describes the expected value of an outcome conditional on explanatory variables. In a simple linear population model:

Here, Y_i is the outcome for observation i, X_i is the explanatory variable, β₀ is the intercept, β₁ is the slope, and u_i contains unobserved influences on Y_i. The observation index may represent a customer, employee, firm, region, or time period.

The conditional mean is:

This equation does not claim that every observation lies on the line. It states the average outcome associated with a given value of X_i. The disturbance term includes omitted determinants, measurement error, random shocks, and model imperfections.

A central distinction is between association and causation. Regression can estimate an association even when X_i is not manipulable or when confounding variables affect both X_i and Y_i. A causal interpretation requires stronger assumptions or a credible research design.

Units and interpretation

If revenue is measured in thousands of dollars and advertising in thousands of dollars, β₁ = 2 means that an additional thousand dollars of advertising is associated with an average increase of two thousand dollars in revenue, holding other included variables constant. The units of the coefficient are outcome units divided by explanatory-variable units.

The intercept is the predicted outcome when all explanatory variables equal zero. It may have little substantive meaning when zero is outside the observed range, but it remains part of the fitted model.

2. Estimation by Ordinary Least Squares

Given data, the fitted simple regression is:

Ordinary least squares (OLS) selects b₀ and b₁ to minimize the sum of squared residuals:

The residual is e_i = Y_i − Ŷ_i. The OLS slope and intercept are:

The denominator in the slope formula is positive when the explanatory variable varies. OLS is appropriate for estimating a linear conditional mean when observations and measurement processes support the model assumptions. It does not require that the outcome itself be normally distributed for coefficients to be calculated.

Worked example: advertising and sales

Suppose four monthly observations record advertising expenditure X and sales Y, both in thousands:

The means are X̄ = 2.5 and Ȳ = 12.5. The cross-deviation sum is 8 and the squared-deviation sum for X is 5, so b₁ = 8 ÷ 5 = 1.6. The intercept is b₀ = 12.5 − 1.6(2.5) = 8.5. Thus:

At advertising expenditure of 5 thousand, predicted sales are 16.5 thousand. This is an extrapolation because the observed advertising range ends at 4; extrapolation should be treated cautiously.

3. Multiple Regression

Business outcomes usually have several determinants. A multiple regression model is:

The coefficient β_j represents the expected change in Y associated with a one-unit increase in X_j, holding the other included explanatory variables constant. The phrase “holding constant” is a conditional comparison, not automatically a causal statement.

For example:

If β₁ = 1.2, advertising is associated with 1.2 thousand additional units of sales for each additional thousand monetary units of advertising, conditional on price and income.

Goodness of fit

The coefficient of determination is:

where SSE = ∑(Y_i − Ŷ_i)² is the residual sum of squares and SST = ∑(Y_i − Ȳ)² is the total sum of squares. With an intercept, R² is the share of sample variation in Y explained by the fitted values. It is not the probability that the model is correct and does not establish causality.

Because R² cannot decrease merely by adding regressors, adjusted R² is sometimes used:

where n is sample size and k is the number of explanatory variables excluding the intercept. Adjusted R² penalizes unnecessary regressors, but it is not a substitute for theory or valid specification.

Inference

A common test of a coefficient is:

Usually β_j,0 = 0. The standard error measures sampling uncertainty under the model and variance assumptions. A confidence interval is:

where t* is a critical value from the relevant t distribution. Statistical significance depends on effect size, variability, and sample size. A small p-value does not indicate that an effect is large or important.

4. Regression Assumptions and Diagnostics

OLS coefficients can be calculated without all classical assumptions, but interpretation and inference depend on them.

Linearity in parameters

The model must be linear in its unknown coefficients. It may include nonlinear variables, such as X² or ln(X), while remaining linear in parameters.

Exogeneity

For an unbiased conditional-mean interpretation, a central condition is:

This means that unobserved determinants in the error are not systematically related to included regressors. Omitted ability in a wage equation, reverse causality, and simultaneity are common threats.

Independent observations

Dependence among observations affects standard errors and sometimes identification. Repeated observations on the same firm, geographic clustering, or time-series autocorrelation require suitable methods such as clustered or robust standard errors.

Heteroscedasticity

Heteroscedasticity occurs when error variance varies across observations:

OLS coefficients may remain unbiased under exogeneity, but conventional standard errors can be wrong. Residual-versus-fitted plots, scale-location plots, and tests such as Breusch–Pagan can provide evidence, although formal tests should not replace substantive reasoning. Heteroscedasticity-consistent (robust) standard errors are often useful. Transformations, weighted least squares, or a better model may also be appropriate.

Normality

Normal errors are not required for OLS point estimates. They can support exact small-sample t and F inference under additional conditions. In larger samples, asymptotic methods often provide approximate inference, but dependence, clustering, and weak identification remain important.

Functional form

A scatterplot, residual plot, theory, and out-of-sample validation help assess whether a straight-line relationship is reasonable. Polynomial terms, logarithms, splines, or interactions can represent curvature, but complexity should be justified.

Outliers and influence

An outlier has an unusual outcome or predictor value. A high-leverage observation is unusual in predictor space; an influential observation materially changes estimates. Leverage, studentized residuals, and Cook’s distance are diagnostic tools. Observations should not be deleted solely because they are inconvenient. Investigate data quality, report sensitivity analyses, and distinguish genuine rare cases from errors.

5. Multicollinearity

Multicollinearity means that explanatory variables are strongly related. Perfect multicollinearity prevents estimation of separate coefficients. Near multicollinearity does not bias OLS under exogeneity, but it increases standard errors and makes individual coefficients unstable.

A common diagnostic is the variance inflation factor:

Here, R_j² is obtained by regressing X_j on all the other explanatory variables. Larger values indicate greater linear dependence; no universal threshold should be treated as a law. High VIF may be acceptable when variables represent theoretically distinct concepts, whereas even modest collinearity can be problematic in a small sample.

Responses include collecting more informative data, combining substantively similar variables, centering variables when appropriate, using theory to remove redundant predictors, or applying regularization for prediction. Dropping a variable solely to obtain significance can create omitted-variable bias.

6. Dummy Variables and Interactions

A dummy variable takes values 0 or 1. If D_i = 1 denotes a trained employee and 0 an untrained employee:

β₀ is the mean predicted productivity for the reference group, and β₁ is the difference between trained and untrained employees in this model.

For a categorical variable with g groups, include g − 1 dummies when an intercept is present. Including all g dummies plus an intercept creates perfect multicollinearity, known as the dummy-variable trap.

An interaction allows an effect to differ by group:

For the reference group, the slope on X is β₁. For the dummy group, it is β₁ + β₃. The interaction coefficient is the difference in slopes. Main effects must be interpreted at the zero value of the other variable; centering may improve interpretability but does not change the underlying fitted relationship.

7. Econometric Endogeneity and Causal Identification

Endogeneity arises when a regressor is correlated with the error term. Major sources include:

• omitted variables, such as motivation in a training-productivity model;
• reverse causality, such as profitable firms being more likely to advertise;
• measurement error, especially in explanatory variables;
• simultaneous determination, such as price and quantity in a market.

Adding controls is not automatically beneficial. A control that is caused by the treatment, or a collider influenced by both treatment and outcome, can introduce bias. Causal models should therefore be guided by institutional knowledge, economic theory, and explicit assumptions.

A useful conceptual framework is the potential-outcomes model. Let Y_i(1) be the outcome for unit i if treated and Y_i(0) the outcome if untreated. The individual causal effect is:

Only one potential outcome is observed for each unit. The average treatment effect is:

Randomization makes treatment assignment independent of potential outcomes in expectation, permitting an unbiased comparison of group means, subject to compliance, attrition, and implementation assumptions.

8. Experiments and Quasi-Experimental Methods

Randomized experiments

In a randomized controlled trial, eligible units are assigned by chance to treatment or control. The simple difference in mean outcomes estimates the intention-to-treat effect when assignment is properly implemented. Randomization balances observed and unobserved characteristics on average, though balance should be checked descriptively rather than used as a mechanical pass/fail rule.

Important threats include noncompliance, attrition, spillovers, poor measurement, inadequate sample size, and treatment effects that differ across populations. Ethical review, informed consent where applicable, privacy protection, and pre-specified outcomes are essential.

Difference-in-differences

Difference-in-differences compares changes over time for a treated group with changes for a comparison group:

The key identifying assumption is parallel trends: absent treatment, the treated and comparison groups would have experienced similar average changes. Pre-treatment trends, event-study graphs, placebo tests, and sensitivity analyses help assess credibility. With staggered treatment timing, conventional two-way fixed-effects estimators can be misleading when effects are heterogeneous; modern estimators should be considered.

Instrumental variables

An instrument Z can identify a causal effect of X on Y when it satisfies relevance and exclusion conditions. Relevance requires that Z affects X. Exclusion requires that Z affects Y only through X, conditional on included controls. A further independence condition requires no unobserved common cause of Z and the potential outcomes.

Two-stage least squares is commonly represented as:

followed by:

where W are controls and X̂ is the first-stage predicted value. IV estimates are often local: under monotonicity, they may describe a local average treatment effect for compliers. Weak instruments produce imprecise and potentially misleading results; first-stage strength and robust inference must be assessed.

Regression discontinuity

A regression-discontinuity design exploits a treatment rule based on a cutoff in a running variable. Units just above and below the threshold may be comparable if they cannot precisely manipulate the score. The estimated effect is local to the cutoff, and bandwidth, continuity, manipulation, and functional-form choices require careful examination.

9. Panel Data

Panel data follow units over time. A fixed-effects model is:

Here, α_i captures time-invariant unit characteristics, λ_t captures common time shocks, and β is identified from within-unit changes after accounting for these effects. Fixed effects do not remove time-varying omitted variables. Standard errors are commonly clustered at the unit level when errors may be serially correlated within units.

Random-effects models impose stronger assumptions about the relationship between unit-specific effects and regressors. The choice between fixed and random effects should be based on the research design and assumptions, not solely on a mechanical statistical test.

10. A Professional Workflow

A defensible regression analysis generally follows this sequence:

1. Define the managerial or research question and the target population.
2. Specify the outcome, treatment or predictors, time frame, and unit of analysis.
3. Draw on theory and institutional knowledge to identify plausible confounders.
4. Inspect data quality, missingness, coding, distributions, and measurement units.
5. Visualize relationships before fitting complex models.
6. Estimate a transparent baseline model and justified alternatives.
7. Examine residuals, leverage, heteroscedasticity, dependence, and collinearity.
8. Report estimates, uncertainty, sample size, specification choices, and missing-data procedures.
9. Conduct robustness and sensitivity analyses.
10. Validate predictions on data not used for estimation when prediction is the objective.
11. Translate results into decisions while clearly stating what the evidence cannot establish.

Prediction and causal explanation should not be conflated. Machine-learning methods may improve predictive accuracy through regularization, trees, ensembles, or neural networks, but predictive performance alone does not identify causal effects. AI systems also require attention to bias, data leakage, fairness, explainability, privacy, and human oversight.

Applied Case: Evaluating a Leadership-Training Programme

Illustrative case. A national retailer introduces a leadership-training programme in 20 stores and wants to know whether it improves employee retention. Store managers volunteer stores for the programme. After six months, trained stores report lower turnover than other stores.

Problem

A simple comparison may be biased because volunteering managers may be more capable, stores may have stronger local labour markets, or managers may introduce other changes at the same time.

Analysis

The analyst first defines turnover consistently and assembles monthly store-level panel data for two years. A baseline regression controls for store size, prior turnover, wages, seasonality, and local unemployment. Store and month fixed effects are included. Because participation was voluntary, fixed effects may remove stable differences but not time-varying factors related to adoption.

The analyst then examines whether adoption dates permit a difference-in-differences design. Pre-treatment trends are plotted. If treated and comparison stores had similar trends before adoption, the design becomes more credible, though not automatically valid. An instrumental-variable strategy might be considered if programme availability was determined by an external scheduling rule that affects retention only through training; the exclusion assumption would require strong institutional justification.

Outcome and decision

Suppose the estimated effect is a 3-percentage-point reduction in six-month turnover, with a confidence interval from 1 to 5 percentage points. Management should combine this estimate with training cost, replacement cost, implementation capacity, and possible spillovers. It should not claim that training caused the entire observed raw difference unless the identification assumptions are credible. Results should be reported separately for different employee groups if effects are heterogeneous, while avoiding unsupported subgroup conclusions from very small samples.

Learning points

The case illustrates that design precedes estimation. More controls cannot rescue a fundamentally weak comparison; pre-trends and institutional knowledge matter; fixed effects address only certain forms of confounding; and a statistically precise result still requires economic evaluation and ethical scrutiny.

Limitations, Governance and Ethics

Regression models compress complex social and economic processes. Measurement error, missing data, selective samples, changing definitions, and model dependence can produce misleading conclusions. Analysts should document data provenance, coding decisions, transformations, exclusions, and analysis versions. Reproducible code and appropriately protected data improve auditability.

Privacy is especially important when employee, customer, health, or location data are used. Data minimization, access controls, de-identification where feasible, lawful processing, and retention limits should be applied. Fairness assessment should examine whether model errors or treatment effects differ across protected or vulnerable groups. A model should not be used as the sole basis for high-stakes decisions when its assumptions, performance, or consequences are poorly understood.

Managers should distinguish three claims: “the variables move together,” “the model predicts the outcome,” and “changing the explanatory variable causes the outcome to change.” Each claim requires progressively stronger evidence.

Chapter Summary

Regression models describe conditional relationships and can support prediction, comparison, and explanation. OLS estimates coefficients by minimizing squared residuals; in multiple regression, each coefficient is interpreted conditional on the other included variables. R² describes in-sample variation explained by fitted values but does not prove causality. Reliable inference requires attention to exogeneity, functional form, dependence, heteroscedasticity, influential observations, and multicollinearity. Dummy variables represent categories, while interactions allow slopes to differ across groups. Endogeneity threatens causal interpretation, so experiments and quasi-experimental designs—difference-in-differences, instrumental variables, regression discontinuity, and panel fixed effects—must be evaluated through their identifying assumptions. Responsible analysis combines theory, data quality, diagnostics, transparent reporting, ethical governance, and sensitivity analysis.

Key Terms

Regression: A method for modelling the conditional relationship between an outcome and one or more predictors.

Econometrics: The application of economic theory, data, and statistical methods to estimate and evaluate relationships.

OLS: Ordinary least squares, which minimizes the sum of squared residuals.

Residual: The observed outcome minus its fitted value.

Coefficient: An estimated or population parameter describing a model relationship.

Exogeneity: A condition requiring explanatory variables to be unrelated to the conditional mean of the error.

Heteroscedasticity: Non-constant conditional variance of the error term.

Multicollinearity: Strong linear dependence among explanatory variables.

Dummy variable: A binary indicator representing category membership.

Interaction: A model term allowing the effect of one variable to depend on another.

Endogeneity: Correlation between an explanatory variable and the error term.

Confounder: A variable related to both a proposed cause and an outcome.

ATE: Average treatment effect across a defined population.

Panel data: Repeated observations on the same units over time.

Fixed effects: Unit-specific or time-specific terms controlling for invariant factors.

Difference-in-differences: A comparison of changes over time between treated and comparison groups.

Instrumental variable: A variable used to isolate exogenous variation in an endogenous predictor.

Regression discontinuity: A design using a treatment threshold to compare nearby observations.

Robust standard error: A standard-error estimator less dependent on constant error variance.

Causal identification: The assumptions and design features that permit a causal interpretation.

Review Questions

Short Answer Questions

1. What is the difference between a regression association and a causal effect?
2. Interpret a slope coefficient in a multiple regression model.
3. Why can multicollinearity increase standard errors without causing OLS bias?
4. What is the dummy-variable trap?
5. State the parallel-trends assumption in difference-in-differences analysis.

Long Answer Questions

1. Explain the main OLS assumptions and discuss the consequences of violating each.
2. Compare statistical significance, practical significance, predictive accuracy, and causal validity.
3. Explain how heteroscedasticity, serial correlation, and clustering affect inference.
4. Describe fixed-effects and random-effects approaches for panel data and their assumptions.
5. Explain instrumental variables, including relevance, exclusion, weak instruments, and local interpretation.

Analytical / Application Questions

1. A firm observes that stores using a new software system have higher sales. Design an analysis that addresses selection bias, time effects, and possible reverse causality.
2. A training coefficient is positive but statistically insignificant after controls are added. Explain at least four possible interpretations and propose diagnostic or data improvements.
3. Construct and interpret a difference-in-differences analysis for a policy introduced in one region but not another. State the assumptions, tests, threats, and managerial implications.

Multiple Choice Questions with Answers

Question 1

In a multiple regression, the coefficient on X₁ is interpreted as:

A. The unconditional correlation between X₁ and Y

B. The change in Y associated with X₁, holding included other variables constant

C. The probability that X₁ causes Y

D. The proportion of Y explained by X₁ alone

Correct Answer: B. The change in Y associated with X₁, holding included other variables constant

Question 2

What does heteroscedasticity mean?

A. The regressors are perfectly correlated

B. The conditional variance of the error is not constant

C. The outcome is normally distributed

D. The regression contains no intercept

Correct Answer: B. The conditional variance of the error is not constant

Question 3

If a model includes an intercept and three categories, how many category dummies are normally included?

A. One

B. Two

C. Three

D. Four

Correct Answer: B. Two

Question 4

Which condition is central to an unbiased OLS conditional-mean interpretation?

A. E(uᵢ | Xᵢ) = 0

B. R² = 1

C. All variables are binary

D. The outcome has zero variance

Correct Answer: A. E(uᵢ | Xᵢ) = 0

Question 5

An interaction term primarily allows:

A. The sample size to increase

B. The effect of one variable to depend on another variable

C. All omitted variables to be removed

D. The intercept to be deleted

Correct Answer: B. The effect of one variable to depend on another variable

Question 6

Which statement about R² is correct?

A. It proves that the model is causal

B. It is the probability that the regression is correct

C. With an intercept, it measures the share of sample outcome variation explained by fitted values

D. It must decrease when a regressor is added

Correct Answer: C. With an intercept, it measures the share of sample outcome variation explained by fitted values

Question 7

The key identifying assumption in a standard difference-in-differences design is:

A. The treated group has no observations before treatment

B. Treated and comparison groups would have followed parallel trends without treatment

C. Treatment is assigned randomly in every period

D. The outcome is constant over time

Correct Answer: B. Treated and comparison groups would have followed parallel trends without treatment

Question 8

For an instrumental variable, relevance means that the instrument:

A. Is unrelated to the endogenous predictor

B. Affects the endogenous predictor

C. Directly determines the outcome outside the predictor

D. Must be measured in monetary units

Correct Answer: B. Affects the endogenous predictor

Question 9

A fixed-effects model is especially useful for controlling for:

A. All time-varying omitted variables

B. Time-invariant unit characteristics

C. Measurement error in every variable

D. Reverse causality automatically

Correct Answer: B. Time-invariant unit characteristics

Question 10

If advertising is measured in thousands of dollars and the estimated sales coefficient is 1.6, the coefficient means:

A. One dollar of advertising causes 1.6 dollars of sales

B. One thousand dollars of advertising is associated with 1.6 thousand units of the outcome, conditional on the model

C. Advertising explains 160% of sales

D. Sales increase by 1.6% regardless of units

Correct Answer: B. One thousand dollars of advertising is associated with 1.6 thousand units of the outcome, conditional on the model