Attribution, Experimentation and Causal Measurement

Introduction

Media analytics often begins with an apparently simple question: which channel produced the sale, subscription, download or other desired outcome? In practice, answering this question is difficult because consumers encounter multiple messages, devices, platforms and non-media influences before acting. A platform report may associate an impression with a conversion, but association does not establish that the impression caused the conversion.

This chapter examines five major approaches to media effectiveness measurement: attribution models, marketing-mix modelling, incrementality testing, A/B testing and quasi-experimental methods. The central managerial issue is not merely measurement precision. It is whether evidence supports a causal decision such as increasing television investment, reducing paid search, changing campaign frequency or reallocating budget across audiences.

A causal media effect is the difference between the outcome observed when a unit receives a media intervention and the outcome that would have occurred for the same unit at the same time without that intervention. The latter is the counterfactual and cannot normally be observed directly. Research designs therefore attempt to construct a credible estimate of it.

Learning objectives

By the end of this chapter, students should be able to:

1. Distinguish association, attribution and causal effect.
2. Explain the assumptions and outputs of common attribution models.
3. Describe the logic, specification and interpretation of marketing-mix models.
4. Design and interpret A/B, holdout, geo-experiment and other incrementality studies.
5. Identify threats such as selection bias, confounding, interference, carryover and measurement error.
6. Select an appropriate method for a media-management decision and communicate uncertainty responsibly.

1. The causal problem in media measurement

Suppose a streaming service observes that subscribers exposed to a video advertisement are more likely to subscribe. This relationship may arise because the advertisement worked, but it may also arise because people already interested in the service were more likely to be targeted, search for the brand, or remain logged in across several devices. Exposure is therefore potentially endogenous: it is related to factors that also influence the outcome.

Let Y(1) denote the outcome for a unit if it receives treatment or media exposure, and Y(0) the outcome for that same unit if it does not. The individual causal effect is:

Here, i identifies a person, household, account, customer or geographic region. Only one potential outcome is observed for each unit. The average treatment effect is:

E denotes an average over the defined target population. The formula is applicable when treatment and outcome are clearly defined and the population and time period are specified. It does not itself solve the counterfactual problem; identification requires a credible research design.

For a treated group and a comparison group, a simple difference in observed outcomes is:

T = 1 indicates exposure and T = 0 indicates non-exposure. This difference equals a causal effect only under strong conditions, such as random assignment or sufficiently strong assumptions that treatment assignment is independent of potential outcomes after adjustment. Without those conditions, it is an association.

A useful distinction is:

• Association: two variables vary together.
• Attribution: a rule assigns credit for an outcome to one or more contacts.
• Incrementality: the additional outcome caused by marketing compared with a credible no-marketing counterfactual.
• Causal effect: a change in an outcome attributable to an intervention under a specified identification strategy.

Attribution can be operationally useful while still failing to estimate incrementality. For example, last-click attribution may assign a purchase to a search ad even when the consumer would have purchased without it.

2. Attribution models

Attribution models are rules for distributing observed conversions across recorded touchpoints. They are most common in digital environments where impressions, clicks and conversions can be linked through identifiers. They answer the question, “How should credit for recorded conversions be allocated?” They do not automatically answer, “What would have happened without the channel?”

2.1 Rule-based attribution

Common deterministic models include:

If a conversion has n recorded touchpoints, linear attribution assigns each touchpoint a share of 1/n. For example, four contacts receive 0.25 conversion credit each. These credits are accounting allocations, not causal estimates.

Rule-based approaches are transparent and inexpensive, making them useful for campaign reporting and operational optimisation. However, they are vulnerable to tracking loss, cross-device fragmentation, cookie restrictions, viewability problems and platform-specific definitions of exposure. They also tend to reward channels that are easy to observe rather than channels that create demand earlier in the journey.

2.2 Algorithmic and data-driven attribution

Data-driven attribution uses statistical or machine-learning methods to estimate how conversion probability changes across paths or contacts. Possible techniques include logistic regression, Markov chains, survival models and Shapley-value approaches.

A Markov approach models movement among states such as “no contact,” “display,” “search,” “site visit” and “conversion.” A removal effect can be calculated by comparing predicted conversions with and without a channel. Such models depend on the quality of path data and the assumption that observed transitions adequately represent the customer journey.

Shapley-value attribution originates in cooperative game theory. It distributes a total outcome across contributors according to their average marginal contribution across possible coalitions. In media applications, the result depends on how channels are defined, how interactions are represented and whether the data-generating process supports causal interpretation. A mathematically elegant allocation is not automatically an experimental estimate.

2.3 When attribution is useful

Attribution is appropriate for descriptive and operational questions, including:

• diagnosing campaign paths and creative sequences;
• monitoring delivery and conversion reporting;
• identifying duplicated or missing tracking;
• generating hypotheses about channel interactions; and
• supporting short-term bidding or trafficking decisions when limitations are understood.

It should not be the sole basis for long-term budget allocation, especially where media has delayed, offline, brand or spillover effects.

3. Marketing-mix modelling

Marketing-mix modelling (MMM) estimates relationships between aggregate outcomes and explanatory variables such as television spend, radio reach, digital impressions, price, distribution, seasonality, competitor activity and economic conditions. It generally uses weekly or daily time-series data at national, regional or market level.

A simplified model may be written as:

Yₜ is the outcome in period t, such as sales or subscriptions. Aₜ, Pₜ and Dₜ are media variables; β coefficients represent partial associations under the model; Xₜ is a vector of controls such as price, distribution and seasonality; γ is a vector of control coefficients; and εₜ is the unexplained error. Units must be consistent: if Yₜ is weekly units sold, media variables must be defined in measurable exposure or spend units and coefficients must be interpreted accordingly.

Media effects are often delayed and nonlinear. An adstock transformation represents carryover:

Aₜ is the transformed media stock, Aₜ is current media activity, Aₜ₋₁ is the previous stock and λ is the retention parameter. The restriction 0 ≤ λ < 1 ensures a nonnegative, geometrically decaying carryover under the usual formulation. λ = 0 means no carryover; values closer to 1 imply slower decay. The model is a simplifying assumption, not a universal law.

Diminishing returns can be represented by a saturation function, for example:

α is the maximum contribution under this functional form, k controls the speed of saturation, and e is the base of natural logarithms. The form assumes a nonnegative response that approaches α as media stock increases. Alternative functions, including Hill or logistic curves, may be more suitable depending on theory and data.

MMM can estimate return on advertising spend (ROAS):

If incremental revenue is measured in currency and media cost in the same currency, ROAS is dimensionless. A value of 3 means one unit of media cost is associated with three units of incremental revenue, before any stated treatment of margin, production cost or other expenses. For profit decisions, incremental profit or contribution margin is generally more appropriate than revenue.

Strengths and limitations of MMM

MMM covers offline and online media, captures broad market effects and does not require person-level tracking. It is valuable for strategic planning and scenario simulation. Its limitations include a relatively small number of observations, collinearity among channels, changing media quality, non-random investment decisions, aggregation bias, model specification risk and sensitivity to the time window. A coefficient can reflect omitted variables or reverse causality rather than a pure causal effect.

Good practice includes pre-specifying variables, controlling for price and distribution, testing alternative lag and saturation structures, using holdout periods where feasible, checking residuals, conducting sensitivity analysis and reporting prediction and parameter uncertainty. MMM should be calibrated against experiments or other credible causal evidence rather than treated as infallible because it uses sophisticated software.

4. Experiments and incrementality

An incrementality test compares outcomes under a marketing intervention with outcomes under a deliberately constructed control condition. The treatment effect is incremental if the design supports the claim that the difference would not have occurred without the intervention.

4.1 Randomised A/B testing

In an A/B test, eligible units are randomly assigned to treatment or control. Treatment may involve receiving an advertisement, a higher bid, a particular creative or a changed frequency. Randomisation balances observed and unobserved characteristics in expectation.

For a binary outcome, the estimated difference in conversion rates is:

pₜ is the treatment-group conversion rate and p꜀ is the control-group conversion rate. The relative lift is:

The denominator must be nonzero, and the interpretation is unstable when p꜀ is very small. If pₜ = 0.030 and p꜀ = 0.025, then Δ = 0.005, or 0.5 percentage points, and lift = 0.005 ÷ 0.025 = 0.20, or 20%.

A simple approximate standard error for the difference between two independent proportions is:

nₜ and n꜀ are the numbers of independent units in treatment and control. This approximation is most defensible with independent observations and sufficiently large samples; cluster-randomised media tests require methods that account for within-cluster correlation. Statistical significance does not establish business importance, and a non-significant result does not prove that the true effect is exactly zero.

A valid test requires a stable treatment definition, no contamination between groups, a pre-specified primary outcome and analysis window, adequate sample size, correct randomisation, and monitoring for technical failures. Multiple outcomes or repeated peeking can increase false-positive risk; these practices should be governed by a pre-analysis plan.

4.2 Holdout and geo experiments

A user-level holdout withholds media from a randomly selected eligible group. It is often effective for digital campaigns, but identity resolution and spillover can compromise the control group.

Geo experiments randomise or match geographic units such as postal areas, cities or media markets. They are useful when exposure cannot be assigned at person level, as in television, out-of-home or regional radio. Treatment and control regions should be comparable in pre-period outcomes, media availability, population and competitive conditions. Interference across borders, commuting and national campaigns are important threats.

A matched-market design may use a synthetic control: a weighted combination of untreated markets designed to reproduce the treated market’s pre-intervention trajectory. The post-period difference is informative only if the pre-period fit is strong and the no-intervention trajectory would have remained comparable.

4.3 Difference-in-differences

Difference-in-differences (DiD) compares changes over time in treated and control groups:

Yₜ,post and Yₜ,pre are treated-group outcomes after and before intervention; Y꜀,post and Y꜀,pre are corresponding control outcomes. The key identifying assumption is parallel trends: absent treatment, treated and control groups would have experienced similar average changes. Pre-period trend checks can support, but cannot prove, this assumption.

4.4 Regression discontinuity and instrumental variables

A regression-discontinuity design is possible when treatment changes at a known threshold, such as an audience score or eligibility rule. Units just above and below the cutoff may be comparable, but the estimated effect is local to the threshold and requires no manipulation around it.

An instrumental-variable design uses a variable Z that changes exposure but affects the outcome only through exposure, under the exclusion restriction. In media, examples might involve technical delivery variation, although credible instruments are difficult to find. Instrumental-variable estimates usually apply to compliers—units whose treatment changes because of the instrument—not necessarily to all consumers.

5. Comparing methods

No method is universally best. A company may use experiments for selected channels, MMM for total-budget planning, and attribution for operational diagnostics. Triangulation is stronger when methods have different error structures and their assumptions are made explicit.

6. Worked application: a regional streaming campaign

This illustrative case concerns StreamNow, a fictional subscription service. Management wants to know whether increasing connected-TV advertising creates incremental subscriptions. Last-click reporting credits 40% of new subscriptions to paid search, 25% to connected TV and the remainder to other contacts. The marketing director suspects that search captures demand generated by television and therefore requests a causal test.

Background and problem

Connected-TV delivery is purchased at household level, but subscriptions are recorded at account level. Some households contain existing subscribers, and paid search is active in every region. A simple exposed-versus-unexposed comparison would be biased because high-interest households are more likely to be served ads.

Approach

The team selects 24 comparable regional markets. Twelve are randomly assigned to a 30% increase in connected-TV impressions for six weeks; twelve continue at the existing level. The primary outcome is new paid subscriptions per 10,000 eligible households. The analysis is conducted at market level to avoid treating households within a market as independent observations. Search spending and price promotions are held constant where possible.

During the test, the treatment markets average 34 subscriptions per 10,000 households, while control markets average 30. The estimated absolute effect is:

The relative lift is:

If each treatment market contains 100,000 eligible households, the rough incremental subscriptions per treatment market are:

Across 12 treatment markets, the rough estimate is 480 incremental subscriptions, provided the reported averages and market sizes support this scaling. The calculation is illustrative; a formal analysis should weight markets appropriately, estimate uncertainty and account for serial correlation and any pre-period imbalance.

Interpretation

The experiment supports an incremental effect of the increased connected-TV treatment under the assumptions of valid randomisation, stable delivery, no material cross-market spillover and comparable measurement. It does not imply that connected TV caused 25% of all subscriptions, which is an attribution statement. It estimates the effect of the specified increase, population and six-week period. The effect of eliminating all connected TV, changing creative, or increasing spend by another amount may differ because of saturation and audience composition.

Managerial decision

Management should compare the incremental subscriptions with contribution margin and incremental media cost. It should also check delayed subscriptions, cancellation quality, brand effects and whether search conversions changed. If search conversions rise in treatment markets, this may indicate complementarity rather than search independently causing the whole outcome. A follow-up design could test coordinated television and search strategies.

Learning points

1. Randomisation addresses selection bias better than exposure-based attribution.
2. The estimand must be stated: treatment intensity, audience, outcome and time horizon matter.
3. Market-level experiments require market-level inference.
4. A positive incremental effect does not establish that every observed conversion was caused by the channel.
5. Decisions require economics, not only statistical significance.

7. Threats to validity

Selection bias occurs when exposed and unexposed units differ systematically. High-intent consumers may receive more ads, making exposure appear more effective than it is.

Confounding occurs when a third factor influences both exposure and outcome. Promotions may increase both media spend and sales.

Interference or spillover occurs when one unit’s treatment affects another unit, violating the assumption that units’ outcomes are unaffected by other assignments. Household sharing, social communication and geographic media overlap are common examples.

Carryover means treatment affects outcomes after the campaign or exposure period. Short tests can underestimate delayed effects; long tests can be contaminated by later changes.

Ad fatigue and saturation mean that the effect of one additional impression depends on previous exposure. Average effects should not be interpreted as constant effects for every consumer.

Outcome measurement error includes missing offline conversions, duplicate accounts, bot activity, attribution-window choices and changing platform definitions. Measurement error can bias results and reduce precision.

External validity concerns whether a result generalises across seasons, markets, creative executions, platforms and spending levels. An experiment is internally credible but still limited to its tested conditions.

8. Ethics, privacy and governance

Causal measurement requires responsible data governance. Organisations should collect only data necessary for a legitimate purpose, document consent and lawful basis where required, protect identifiers, restrict access and define retention periods. Sensitive attributes should not be used casually for targeting or evaluation.

Randomised experiments can expose people to different communications or offers. Designs should avoid foreseeable harm, discriminatory exclusion and unfair pricing. Control groups should not be denied essential services or legally required information. Researchers must distinguish de-identification from complete anonymity and assess re-identification risk.

Governance should include an estimand statement, pre-analysis plan, experiment owner, data dictionary, treatment definition, exclusion rules, stopping rule and audit trail. Automated or AI-based optimisation should not be allowed to convert uncertain correlations into unsupported causal claims. Platform-reported “lift” should be scrutinised for control contamination, attribution-window choices and conflicts of interest.

9. Contemporary developments and best practice

Privacy regulation and the decline of third-party identifiers are increasing interest in first-party measurement, clean rooms, aggregated reporting, consented panels and privacy-preserving computation. These technologies can reduce data leakage but do not remove identification problems. A clean room can securely join data while still producing a biased comparison.

Modern practice increasingly combines several evidence sources. A recommended measurement system is:

1. Use tracking and rule-based attribution for delivery diagnostics.
2. Run well-designed experiments for priority channels and major changes.
3. Use MMM for cross-channel, long-term and offline budget decisions.
4. Calibrate models with experimental results where definitions are compatible.
5. Report point estimates, uncertainty, assumptions and business value together.
6. Replicate important findings across periods, markets or creative conditions.

The most useful question is not “Which model gives the most credit?” but “What decision is being made, what counterfactual is relevant, and which design can estimate it credibly?”

Chapter Summary

Attribution allocates observed conversions among contacts, whereas causal measurement estimates what changed because of an intervention. Last-touch, first-touch, linear, time-decay and algorithmic attribution are useful for reporting and journey diagnostics but can reward demand capture and cannot, by themselves, establish incrementality. Marketing-mix modelling relates aggregate outcomes to media and market variables, incorporates carryover and saturation, and supports strategic planning, but depends heavily on data quality, specification and identification assumptions.

Randomised A/B and holdout tests provide strong causal evidence when treatment is correctly assigned, contamination is controlled and outcomes are measured appropriately. Geo experiments, synthetic controls and difference-in-differences extend experimentation to settings where individual randomisation is impractical, but rely on assumptions such as comparable pre-trends and limited spillover. Regression discontinuity and instrumental variables can be valuable in special settings but usually estimate local effects.

Effective media measurement combines methods rather than treating one as universally superior. Managers should specify the estimand, inspect assumptions, quantify uncertainty, consider privacy and fairness, and translate incremental outcomes into contribution, customer quality and long-term value.

Key Terms

Causal effect: The change in an outcome caused by an intervention relative to its counterfactual.

Counterfactual: The outcome that would have occurred for the same unit under an alternative treatment condition.

Attribution: A rule for allocating observed credit for an outcome across media contacts.

Incrementality: The additional outcome generated by marketing beyond what would otherwise have occurred.

Treatment: The media intervention or condition being evaluated.

Control: The comparison condition representing the relevant no-treatment or alternative-treatment state.

Average treatment effect (ATE): The average difference between potential outcomes under treatment and control.

Selection bias: Bias caused by systematic differences between treated and comparison units.

Confounding: Distortion caused by a variable related to both exposure and outcome.

Randomisation: Assignment by chance to balance potential confounders in expectation.

Holdout: A deliberately untreated group retained for comparison.

Geo experiment: An experiment that assigns geographic markets to treatment conditions.

Difference-in-differences: An estimator comparing changes over time between treated and control groups.

Marketing-mix modelling: Aggregate statistical modelling of outcomes against marketing and market variables.

Adstock: A transformation representing delayed or carryover media effects.

Saturation: Diminishing response as media exposure or spend increases.

Spillover: Treatment influence that crosses from treated units to control units.

External validity: The extent to which findings generalise beyond the tested conditions.

Estimand: The precisely defined causal quantity a study seeks to estimate.

ROAS: Incremental revenue divided by media cost.

Review Questions

Short Answer Questions

1. Why does a last-click conversion credit not necessarily represent a causal effect?
2. Define a counterfactual in the context of media measurement.
3. State the main assumption required for a basic difference-in-differences interpretation.
4. What does the adstock parameter λ represent?
5. Why can a platform-reported lift estimate be misleading?

Long Answer Questions

1. Compare rule-based attribution, data-driven attribution and incrementality testing in terms of purpose, data requirements and causal credibility.
2. Explain how marketing-mix modelling can incorporate carryover and diminishing returns, and discuss at least four limitations.
3. Describe the design of a valid user-level A/B test for an advertising campaign, including randomisation, outcomes, contamination and analysis.
4. Explain geo experiments, synthetic controls and difference-in-differences, highlighting the assumptions that support causal interpretation.
5. Discuss how privacy regulation and fragmented media measurement affect the design and governance of causal studies.

Analytical / Application Questions

1. A campaign produces a 4.0% conversion rate among exposed users and 3.2% among unexposed users. Identify two reasons why the 0.8 percentage-point difference may not be incremental, and propose a stronger design.
2. A television test increases subscriptions in treated markets, but paid-search conversions also rise. Develop two competing explanations and specify additional analysis that could distinguish them.
3. A manager wants to use last-click ROAS to move all budget from television to search. Evaluate this proposal using attribution, MMM and experimental reasoning, and recommend a measurement plan.

Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes attribution from incrementality?

A. Attribution allocates observed credit, while incrementality estimates the additional outcome caused by marketing.

B. Attribution always uses randomisation, while incrementality never does.

C. Attribution measures only offline media, while incrementality measures only digital media.

D. Attribution is causal by definition, while incrementality is descriptive.

Correct Answer: A. Attribution allocates observed credit, while incrementality estimates the additional outcome caused by marketing.

Question 2

In the potential-outcomes framework, what does Y(0) represent?

A. The average outcome across all treated units

B. The outcome for a unit under the control or no-treatment condition

C. The observed outcome after treatment

D. The treatment assignment probability

Correct Answer: B. The outcome for a unit under the control or no-treatment condition

Question 3

A treatment group converts at 3.0% and a control group at 2.5%. What is the relative lift?

A. 0.5%

B. 5%

C. 20%

D. 55%

Correct Answer: C. 20%

Question 4

What is the primary identifying assumption in a basic difference-in-differences analysis?

A. Treatment and control must have identical sample sizes

B. The groups would have followed parallel trends without treatment

C. The treatment effect must be zero before the intervention

D. All outcomes must be normally distributed

Correct Answer: B. The groups would have followed parallel trends without treatment

Question 5

In the adstock equation A*ₜ = Aₜ + λA*ₜ₋₁, what does λ describe?

A. The conversion rate

B. The media price

C. The retained carryover from the previous period

D. The number of treatment markets

Correct Answer: C. The retained carryover from the previous period

Question 6

Which method is generally best suited to estimating the incremental effect of a digital campaign when user-level randomisation is feasible?

A. Last-touch attribution

B. Randomised holdout test

C. Unadjusted exposed-versus-unexposed comparison

D. First-touch attribution

Correct Answer: B. Randomised holdout test

Question 7

Why can marketing-mix modelling be useful for television and out-of-home media?

A. It requires a deterministic user-level path for every conversion

B. It can analyse aggregate market and time-series data without person-level tracking

C. It eliminates all confounding automatically

D. It assigns every sale to the final impression

Correct Answer: B. It can analyse aggregate market and time-series data without person-level tracking

Question 8

What is spillover in an experiment?

A. A delayed outcome within the treatment group only

B. A measurement unit being counted twice

C. Treatment influence reaching comparison units

D. A model reaching its maximum saturation

Correct Answer: C. Treatment influence reaching comparison units

Question 9

A ROAS of 3 means, under the stated measurement definition, that:

A. Every customer purchased three times

B. Each unit of media cost generated three units of measured incremental revenue

C. The campaign generated a 3% conversion rate

D. Media cost was three times total revenue

Correct Answer: B. Each unit of media cost generated three units of measured incremental revenue

Question 10

Which practice most improves the credibility of an advanced media-measurement programme?

A. Using the most complex model without documenting assumptions

B. Reporting only statistically significant results

C. Combining experiments and modelling while stating estimands, assumptions and uncertainty

D. Replacing all controls with platform-reported attribution

Correct Answer: C. Combining experiments and modelling while stating estimands, assumptions and uncertainty