11. Causal Inference, Experimentation and Incrementality

Introduction and Context

Advertising analytics has traditionally relied on descriptive measures such as reach, impressions, clicks, conversions, cost per acquisition, and return on advertising spend. These measures are useful for monitoring activity, but they do not necessarily answer the most important managerial question: What outcomes were caused by advertising that would not otherwise have occurred?

A conversion observed after an advertisement is not automatically a conversion caused by that advertisement. The consumer may already have intended to purchase, may have encountered another marketing stimulus, or may have converted because of price, distribution, seasonality, word of mouth, or an unrelated event. Causal inference addresses this distinction between association and causation. Experimentation provides a disciplined way to estimate causal effects, while incrementality measures the additional outcomes generated by advertising beyond the baseline that would have existed without it.

This chapter develops the logic and practice of causal measurement in advertising. It examines randomized A/B tests, multivariate experiments, geo-tests, lift studies, quasi-experimental methods, confounding, selection bias, statistical power, and managerial interpretation. The central principle is simple but demanding: advertising effectiveness must be evaluated against a credible counterfactual—the outcome that would have occurred in the absence of the advertising intervention.

Learning Objectives

After studying this chapter, students should be able to:

1. Distinguish correlation, association, prediction, and causation in advertising analytics.

2. Explain counterfactual reasoning, treatment effects, average treatment effects, and incremental outcomes.

3. Design randomized A/B, multivariate, geo-experimental, and lift studies.

4. Identify confounding, selection bias, contamination, interference, and common experimental errors.

5. Apply difference-in-differences, regression discontinuity, matching, and synthetic-control approaches where randomization is unavailable.

6. Interpret lift, incremental conversions, incremental revenue, cost per incremental acquisition, and incremental return on advertising spend.

7. Evaluate the ethical, technical, organizational, and managerial implications of experimentation.

From Attribution to Causal Measurement

Attribution asks which touchpoints receive credit for an observed outcome. Causal inference asks whether changing exposure to advertising changes the probability of that outcome. These questions are related but not identical.

Suppose a user sees a retargeting advertisement and purchases a product. A last-touch attribution model may assign the purchase entirely to the retargeting ad. However, the user may have been highly likely to purchase already. The ad may have influenced the purchase, but it may also have merely appeared before an inevitable conversion. Attribution distributes observed credit; incrementality estimates additional impact.

The causal problem can be expressed through potential outcomes. For an individual, let Y(1) represent the outcome if the individual receives advertising and Y(0) represent the outcome if the same individual does not receive advertising. The individual causal effect is:

Y(1) − Y(0)

Only one of these outcomes can be observed for a particular individual. This is known as the fundamental problem of causal inference. If the individual receives advertising, the analyst observes Y(1) but not Y(0). If the individual does not receive advertising, the analyst observes Y(0) but not Y(1). Experiments estimate average effects by constructing comparable treatment and control groups.

The Average Treatment Effect (ATE) is:

ATE = E[Y(1) − Y(0)]

In advertising, the treatment may be exposure to a campaign, a particular creative, an increased bid, a promotional message, or a media channel. The outcome may be a purchase, lead, app installation, store visit, brand consideration, or revenue amount.

The incremental effect is the difference between the observed outcome under advertising and the estimated counterfactual outcome without advertising. If the treatment group generates 12,000 conversions and the comparable control group indicates that 9,000 would have occurred without advertising, incremental conversions equal 3,000.

Core Concepts and Terminology

Treatment, control, and assignment

The treatment is the advertising intervention being evaluated. The control condition is the alternative against which the treatment is compared. A control group may receive no advertising, ordinary advertising, a public-service message, or a different creative, depending on the research question.

Assignment describes how entities are allocated to treatment or control. In a randomized experiment, assignment is determined by chance. Exposure, by contrast, refers to what actually happens after assignment. A person assigned to treatment may not see the advertisement because of an ad blocker, inventory shortage, or platform delivery failure. Therefore, analysts must distinguish intention-to-treat effects, based on assignment, from treatment-on-the-treated effects, based on actual exposure.

Counterfactual and baseline

The counterfactual is the outcome that would have occurred in the absence of the treatment. It cannot normally be observed directly for the same unit at the same time. A control group, a pre-treatment trend, or a carefully constructed comparison group is used to estimate it.

The baseline is the expected level of outcomes without the intervention. Incrementality is measured relative to this baseline, not simply relative to zero.

Confounding

A confounder is a variable associated with both advertising exposure and the outcome. For example, high-intent consumers may be more likely to receive retargeting advertisements and more likely to purchase. If intent is not controlled, the campaign may appear more effective than it actually is.

Selection bias

Selection bias occurs when treatment and control groups differ systematically because participation or exposure is not random. Consumers who choose to click an advertisement are usually different from those who do not. Comparing clickers with non-clickers therefore produces a biased estimate of advertising impact.

Spillover and interference

Spillover occurs when treatment assigned to one unit affects another unit. A household-level campaign may influence untreated household members. A geo-test may expose residents in a control region through travel or digital media overlap. Such interference violates the assumption that each unit's outcome is independent of other units' treatment status.

Lift

Lift is the difference in outcomes between treatment and control, often expressed as an absolute or relative percentage:

Absolute lift = Treatment rate − Control rate

Relative lift = (Treatment rate − Control rate) / Control rate

If treatment conversion is 4.8% and control conversion is 4.0%, absolute lift is 0.8 percentage points and relative lift is 20%.

Designing a Credible Experiment

A strong experiment begins with a precise decision problem. “Does digital advertising work?” is too broad. A useful question might be: “Does increasing paid social exposure among new website visitors increase completed purchases within fourteen days, compared with the current media plan?” The treatment, eligible population, outcome, time horizon, and decision rule should be specified before launch.

Unit of randomization

The unit may be an individual, cookie, device, account, household, store, postcode, city, or media market. The choice affects statistical power and the risk of contamination. User-level randomization provides many observations but may permit cross-device exposure. Geo-level randomization reduces some forms of contamination but provides fewer units and may require larger budgets.

Randomization should generally occur at the level at which interference is likely. If members of a household share purchasing decisions, household-level assignment may be more appropriate than individual-level assignment.

Treatment and control conditions

The treatment should represent a realistic advertising intervention. It might be an additional campaign, a new creative, a higher frequency cap, or a revised audience strategy. The control condition must represent a credible alternative. A pure no-ad control is not always appropriate if the managerial decision concerns replacing one creative with another.

A holdout group is a deliberately protected control group that is withheld from the intervention. Holdouts are particularly valuable in platform advertising, where ordinary non-exposed users may differ from exposed users.

Randomized A/B testing

In an A/B test, eligible units are randomly assigned to version A or version B. In advertising, version A may be the existing creative or media plan, while version B is the proposed alternative. Randomization balances both observable and unobservable characteristics in expectation.

A basic workflow includes:

• defining the hypothesis and primary outcome;
• selecting the population and randomization unit;
• calculating the required sample size;
• implementing stable treatment assignment;
• monitoring delivery and data quality;
• analyzing results according to the pre-specified plan; and
• deciding whether the observed effect is practically and statistically meaningful.

The primary outcome should be selected carefully. If the business objective is profitable sales, clicks may be an intermediate metric rather than the final outcome. Secondary metrics such as reach, frequency, engagement, cost, complaints, and unsubscribes remain useful for diagnosing mechanisms and risks.

Multivariate experimentation

A multivariate experiment tests several factors or levels simultaneously. For example, a campaign may vary headline, visual, call to action, and offer. A full factorial design estimates main effects and interactions, but the number of combinations can grow rapidly. If four factors each have three levels, a full factorial design contains 81 combinations.

Fractional factorial designs reduce the number of combinations by accepting limitations in the estimation of some interactions. Multivariate tests require substantial traffic and disciplined interpretation. A creative that performs well in one combination may not be universally superior because interactions can be important: an offer may work only with a particular visual or audience.

Sample size, power, and statistical significance

An experiment should be planned around the smallest effect that would justify action, often called the minimum detectable effect. Required sample size depends on baseline conversion rate, desired power, significance level, treatment allocation, and the expected effect size.

Statistical significance indicates how incompatible the observed result is with a specified null hypothesis under the model assumptions. It does not indicate the economic importance of the effect. A very large sample may make a tiny lift statistically significant, while a commercially important effect may fail to reach significance in a small test.

Confidence intervals are more informative than a single point estimate. A treatment may produce a 5% relative lift with a confidence interval ranging from −2% to 12%. This result does not establish a reliable positive effect, although it may justify a larger test if the potential upside is substantial.

Repeatedly checking results and stopping when significance appears inflates false-positive risk. Sequential testing requires an appropriate statistical framework or a pre-specified stopping rule.

Measuring Incrementality

Suppose a treatment group contains 100,000 eligible users and records 5,000 purchases. A control group of equal size records 4,000 purchases. The treatment rate is 5%, the control rate is 4%, and absolute lift is 1 percentage point. Estimated incremental conversions are:

100,000 × (0.05 − 0.04) = 1,000

If the campaign cost is ₹600,000, cost per incremental conversion is ₹600. If average contribution margin per incremental conversion is ₹1,000, incremental contribution is ₹1,000,000 and incremental return on advertising spend, using contribution as the numerator, is 1.67.

Analysts must distinguish gross conversions from incremental conversions. Gross conversions include both conversions caused by the campaign and conversions that would have occurred without it. The latter are sometimes described as organic, baseline, or cannibalized conversions depending on the context.

Incrementality can also be estimated for revenue, profit, brand awareness, store traffic, or customer lifetime value. When outcomes mature slowly, a suitable attribution window and observation period are essential. Short windows may miss delayed effects; long windows may include unrelated influences.

Geo-Experiments and Lift Studies

Geo-tests assign treatment and control conditions to geographic units such as cities, regions, postal areas, or media markets. Treatment regions receive the campaign, while control regions do not, or receive a comparison intervention. Outcomes are then compared after accounting for pre-existing differences and trends.

Geo-tests are useful when user-level randomization is unavailable, when advertising is purchased through broadcast or out-of-home media, or when the outcome is naturally measured at store or market level. Their weaknesses include a small number of units, geographic heterogeneity, cross-market media spillover, and difficulty finding comparable regions.

Pre-treatment data should be used to assess whether candidate markets have similar trends. Matching, clustering, or synthetic-control construction can improve comparability. A synthetic control is a weighted combination of untreated markets designed to reproduce the treated market's pre-intervention trajectory.

A brand lift study measures changes in attitudes, awareness, consideration, preference, or purchase intent. Respondents may be surveyed after exposure and compared with a control group, or surveyed before and after the campaign. Randomized survey invitations are preferable because self-selected respondents can differ from the target population. Brand lift should not be treated as a direct substitute for sales incrementality; it measures a different stage of the consumer response process.

Quasi-Experimental Approaches

Randomized experiments are the strongest practical design for many advertising questions, but they are not always feasible. Quasi-experimental methods attempt to approximate experimental logic using observational data.

Difference-in-differences

Difference-in-differences compares the change over time in a treated group with the change over time in a comparison group:

DiD = (Post-treatment − Pre-treatment in treatment group) − (Post-treatment − Pre-treatment in control group)

The key assumption is parallel trends: absent the intervention, treatment and control groups would have followed similar trends. This assumption should be examined using multiple pre-treatment periods. A one-period pre/post comparison is weak evidence because other events may coincide with the campaign.

Matching and propensity scores

Matching pairs treated units with similar untreated units based on observed characteristics such as prior purchases, geography, device, demographics, or browsing behavior. Propensity scores estimate the probability of treatment given observed covariates. Matching can reduce imbalance but cannot eliminate bias from unmeasured variables. It also becomes unreliable when treated units have no comparable controls, a problem known as lack of common support.

Regression discontinuity

Regression discontinuity can be used when treatment is assigned at a known threshold. For example, users above a score may receive a premium offer while those just below it do not. Comparing units close to the threshold can estimate a local causal effect, provided individuals cannot precisely manipulate the assignment variable and other determinants change smoothly at the cutoff.

Instrumental variables

An instrumental variable affects advertising exposure but influences the outcome only through that exposure. Valid instruments are difficult to find. A weak or invalid instrument can produce misleading estimates. Instrumental-variable estimates often apply to a local subgroup whose treatment status is changed by the instrument rather than to the entire population.

Regression adjustment

Regression models can control for observed differences and estimate treatment effects. However, regression adjustment is not automatically causal. Correct model specification, adequate overlap, treatment timing, and control of post-treatment variables are essential. Controlling for a variable caused by advertising can block part of the true effect or create bias.

Confounding, Selection, and Experiment Integrity

The most common analytical error is to compare exposed and unexposed users without considering how exposure was assigned. High-value customers may receive greater advertising intensity because an algorithm predicts that they are likely to purchase. A simple exposed-versus-unexposed comparison will then attribute pre-existing purchase propensity to advertising.

Other threats include:

• Contamination: control units receive the treatment through another channel.
• Noncompliance: assigned users do not receive the intended treatment.
• Attrition: outcome data are missing differentially across groups.
• Novelty effects: an initially strong response fades after repeated exposure.
• Seasonality: treatment and control periods differ in demand patterns.
• Concurrent interventions: pricing, promotions, distribution, or public events affect the outcome.
• Metric manipulation: optimization toward an easy-to-improve proxy harms the actual business objective.
• Interference: one unit's treatment affects another unit's outcome.

Randomization protects against many confounders only when implementation is correct. Analysts should validate assignment balance, inspect delivery rates, document exclusions, monitor exposure frequency, and maintain an auditable experiment log.

Applied Case: An Illustrative Streaming-Service Experiment

Background

Consider StreamNow, a fictional subscription video service seeking to increase paid subscriptions. Its marketing team uses paid social advertising to target visitors who have watched at least one free episode. Last-touch reporting attributes many subscriptions to retargeting ads, but finance questions whether the ads create incremental demand or simply capture users who would subscribe anyway.

Situation and problem

The team proposes a four-week randomized holdout experiment. Eligible visitors are randomly assigned at the household level to either continued retargeting advertising or a control condition in which they receive unrelated public-service content. The primary outcome is paid subscription within fourteen days. Secondary outcomes include trial starts, completed viewing sessions, cancellations, and advertising complaints.

Analysis and design

The unit of randomization is the household because several users may share a subscription decision. Assignment occurs before campaign bidding, and the assignment ID is passed to the advertising platform. The team excludes existing subscribers but does not exclude high-intent visitors, since the objective is to estimate the effect across the eligible audience.

Before launch, the team specifies a minimum detectable relative lift of 8%, a 95% confidence level, and 80% power. It also defines an analysis based on assignment rather than only delivered impressions. This intention-to-treat approach preserves the benefit of randomization and reflects the effect of making the campaign available under normal delivery conditions.

During the test, some treatment households do not receive an impression because of inventory constraints. The analysts do not simply remove them from the treatment group. Instead, they report delivery rate separately and estimate the intention-to-treat effect. They also examine whether control households were accidentally reached through other campaigns.

Results and interpretation

The treatment group produces a subscription rate of 6.0%, while the control group produces 5.5%. The absolute lift is 0.5 percentage points and the relative lift is approximately 9.1%. If each group contains 200,000 households, the estimated incremental subscriptions are 1,000.

The campaign generated 12,000 observed subscriptions in the treatment group, but the control rate implies that approximately 11,000 would have occurred without retargeting. Therefore, reported attributed subscriptions substantially exceed incremental subscriptions. The campaign cost is ₹450,000, producing a cost per incremental subscription of ₹450.

A further analysis finds that the effect is concentrated among visitors who watched only one episode, while frequent viewers show little incremental response. The company therefore decides not to scale the campaign uniformly. It reduces retargeting among high-intent frequent viewers, reallocates budget to lower-intent eligible visitors, and repeats the test with a longer observation window to study delayed subscriptions.

Learning points

The case illustrates five principles. First, attribution can overstate causal impact. Second, a holdout group creates a counterfactual. Third, intention-to-treat analysis is safer than analyzing only people who actually saw the advertisement. Fourth, heterogeneous effects can improve targeting and budget allocation. Finally, a statistically positive result does not imply that every audience segment or every additional impression is profitable.

Managerial and Professional Applications

Causal measurement influences decisions at several levels. Media planners can compare channels using incremental cost per acquisition rather than platform-reported conversions. Brand managers can evaluate whether a creative changes consideration, not merely engagement. Performance marketers can identify saturation points where additional frequency produces little incremental response. Finance teams can connect advertising lift to contribution margin and customer lifetime value.

At portfolio level, experiments can inform budget allocation across channels. However, channel tests should consider interaction effects. Removing television advertising may alter search behavior; reducing paid search may change direct traffic; increasing one channel may reduce the apparent effect of another. Sequential channel tests and geo-level designs may be needed to estimate combined effects.

Organizations also need an experimentation operating model. This includes a central measurement framework, standardized definitions, data governance, experiment review, pre-registration of hypotheses, technical support for randomization, and procedures for communicating uncertainty to executives.

Ethical, Privacy, and Governance Considerations

Experimentation involves manipulating communication, attention, and sometimes access to offers. Participants should not be exposed to unreasonable risk, discriminatory treatment, or deceptive practices. Sensitive attributes should not be used casually for heterogeneous-effect analysis. Data minimization, access controls, retention limits, and lawful processing are essential.

A control group may be denied a potentially beneficial offer, so the intervention should be proportionate and consistent with organizational policy. Advertising experiments should also be reviewed for disparate impact. An apparently effective campaign may perform differently across demographic or socioeconomic groups, and optimization can reinforce exclusion if fairness is ignored.

Governance requires a record of the research question, assignment method, eligibility rules, primary outcome, analysis plan, deviations, data sources, and decision. This documentation supports reproducibility and prevents selective reporting.

Technology, Automation, and Emerging Developments

Digital advertising platforms increasingly automate bidding, targeting, creative selection, and budget allocation. Automation can improve operational efficiency but complicates causal measurement because the algorithm may change exposure in response to user behavior. Stable holdouts, platform conversion-lift studies, randomized geo experiments, and independent measurement can reduce this problem.

Machine learning can support uplift modeling, which predicts how an individual's outcome changes under treatment rather than merely predicting purchase probability. This distinction is important: the best target for advertising is not necessarily the person most likely to purchase, but the person whose purchase probability is most increased by advertising. Uplift models require careful validation and should not be interpreted causally unless their training design supports causal identification.

Privacy-enhancing technologies, clean rooms, aggregated reporting, and modeled conversions are changing measurement practice as identifiers become less available. These tools may improve privacy but can reduce transparency and make independent validation more difficult. The principle remains unchanged: measurement systems should preserve a credible comparison between treated and untreated conditions.

Common Mistakes and Best Practices

Frequent mistakes include defining success after seeing the data, using clicks as a substitute for business outcomes, stopping tests early, ignoring non-delivery, changing audience rules mid-test, comparing incomparable geographies, and reporting relative lift without the baseline rate. Another mistake is treating a statistically significant result as proof of durable profitability.

Best practice is to:

• define the causal question and decision rule before launch;
• randomize at the appropriate unit;
• select a meaningful primary outcome;
• estimate sample size and minimum detectable effect;
• protect the control group from contamination;
• analyze assignment-based effects first;
• report absolute lift, relative lift, uncertainty, and economic value;
• examine heterogeneous effects cautiously;
• test durability and saturation where relevant; and
• document limitations and deviations transparently.

Chapter Summary

Causal inference distinguishes advertising exposure associated with an outcome from advertising that caused an additional outcome. The essential tool is the counterfactual: an estimate of what would have happened without the intervention. Randomized experiments create credible counterfactuals through treatment and control assignment, while quasi-experimental methods approximate this logic when randomization is not possible.

A/B tests, multivariate designs, geo-tests, holdouts, and brand lift studies answer different questions and require appropriate units, outcomes, time horizons, and controls. Incremental conversions, revenue, profit, cost per incremental acquisition, and incremental return on advertising spend are more decision-relevant than attributed outcomes alone. Confounding, selection bias, contamination, interference, noncompliance, seasonality, and premature stopping can undermine conclusions. Sound experimentation therefore combines statistical design, technical implementation, business judgment, ethical governance, and transparent communication of uncertainty.

Key Terms

Causal inference: The estimation of how an outcome changes because of an intervention.

Counterfactual: The outcome that would have occurred for the same unit without treatment.

Treatment: The advertising intervention being evaluated.

Control group: A comparison group not receiving the focal intervention or receiving an alternative condition.

Randomization: Assignment to treatment conditions by chance.

Average Treatment Effect: The average difference between potential outcomes under treatment and control.

Incrementality: The additional outcome caused by advertising beyond the baseline.

Lift: The difference in outcome rates between treatment and control.

Confounding: Distortion caused by a variable related to both treatment exposure and outcome.

Selection bias: Systematic difference created when units enter treatment or control non-randomly.

Holdout group: Units deliberately withheld from an intervention to estimate a counterfactual.

Intention-to-treat effect: The effect of assignment to treatment regardless of actual exposure.

Treatment-on-the-treated effect: The effect among units that actually receive treatment.

Spillover: Treatment influence that reaches units assigned to control.

Difference-in-differences: A method comparing changes over time between treated and comparison groups.

Synthetic control: A weighted combination of untreated units used to approximate a treated unit's counterfactual.

Minimum detectable effect: The smallest effect an experiment is designed to identify with specified power.

Statistical power: The probability of detecting an effect of a specified size when it exists.

Uplift modeling: Predicting the change in outcome caused by treatment for different units.

Review Questions

Short Answer Questions

1. Why is an observed post-advertising conversion not necessarily an incremental conversion?

2. Distinguish intention-to-treat analysis from treatment-on-the-treated analysis.

3. What is the purpose of a holdout group?

4. Define confounding and provide an advertising example.

5. What is the difference between absolute lift and relative lift?

Long Answer Questions

1. Explain potential-outcome reasoning and its importance in advertising measurement.

2. Describe the stages involved in designing a credible randomized A/B experiment.

3. Compare user-level experiments, geo-tests, and brand lift studies in terms of uses and limitations.

4. Explain difference-in-differences, including its parallel-trends assumption.

5. Discuss how experimentation can improve advertising budget allocation and why attributed conversions may mislead managers.

Analytical / Application Questions

1. A campaign produces a 6% conversion rate among 50,000 treated users and 5% among 50,000 control users. Calculate absolute lift, relative lift, and incremental conversions.

2. A retargeting campaign is delivered mainly to users with high prior purchase intent. Explain why an exposed-versus-unexposed comparison may overestimate effectiveness and propose a better design.

3. A geo-test shows higher sales in treatment markets after advertising, but treatment markets were already growing faster before launch. Identify the threat to validity and recommend an analytical approach.

22. Multiple Choice Questions with Answers

Question 1

Which measure most directly represents the additional conversions caused by an advertising campaign?

A. Total attributed conversions

B. Incremental conversions relative to the control baseline

C. Total impressions divided by reach

D. Number of users who clicked the advertisement

Correct Answer: B. Incremental conversions relative to the control baseline

Question 2

Why is randomization valuable in an advertising experiment?

A. It guarantees that every user sees the advertisement

B. It removes the need to define an outcome

C. It balances observed and unobserved characteristics in expectation

D. It ensures that the treatment effect is economically profitable

Correct Answer: C. It balances observed and unobserved characteristics in expectation

Question 3

A treatment conversion rate is 5% and a control conversion rate is 4%. What is the relative lift?

A. 1%

B. 4%

C. 20%

D. 25%

Correct Answer: C. 20%

Question 4

Which situation is most likely to create selection bias?

A. Users are assigned to treatment by a random number generator

B. Users choose whether to click an advertisement and clickers are compared with non-clickers

C. Markets are assigned to treatment before the campaign begins

D. A pre-specified primary outcome is analyzed after the test

Correct Answer: B. Users choose whether to click an advertisement and clickers are compared with non-clickers

Question 5

In a difference-in-differences design, the parallel-trends assumption means that:

A. Treatment and control groups must have identical outcomes at all times

B. Both groups must receive exactly the same advertising exposure

C. Absent treatment, the groups would have followed similar trends

D. The treatment effect must be constant across all users

Correct Answer: C. Absent treatment, the groups would have followed similar trends

Question 6

A campaign generates 5,000 observed conversions in treatment, while the control group implies that 4,000 would have occurred without advertising. What is the estimated number of incremental conversions?

A. 1,000

B. 4,000

C. 5,000

D. 9,000

Correct Answer: A. 1,000

Question 7

Why should an intention-to-treat analysis retain treatment units that did not actually receive an impression?

A. It increases the reported conversion rate

B. It preserves the benefits of the original random assignment

C. It eliminates all delivery problems

D. It measures only the effect among highly engaged users

Correct Answer: B. It preserves the benefits of the original random assignment

Question 8

Which is a major risk in a geo-experiment?

A. Geographic units can never be compared statistically

B. Control markets may be exposed through media spillover

C. Sales cannot be measured at market level

D. Randomization is always impossible across markets

Correct Answer: B. Control markets may be exposed through media spillover

Question 9

A statistically significant result is not necessarily managerially valuable because:

A. Significance tests cannot be used in advertising

B. A significant effect may be too small to cover campaign costs

C. Control groups always exaggerate performance

D. Statistical significance proves that the result is random

Correct Answer: B. A significant effect may be too small to cover campaign costs

Question 10

Uplift modeling differs from ordinary purchase prediction because it attempts to predict:

A. The number of impressions purchased

B. The probability that a user will click any advertisement

C. How a user's outcome changes because of treatment

D. The average cost of media inventory

Correct Answer: C. How a user's outcome changes because of treatment