Measurement Planning and Research Design

Introduction

Campaign measurement is not an afterthought applied after media have been purchased. It is a system for converting strategic questions into credible evidence about whether communication produced meaningful outcomes. In advertising and media, outcomes may include reach, attention, brand awareness, consideration, preference, purchase, retention, revenue, profit, or longer-term customer value. Each outcome requires a different research design, data source, time horizon, and standard of evidence.

A measurement plan links five elements: the business objective, the causal hypothesis, the outcome metric, the data required, and the analytical design. Without this linkage, organisations often report abundant data but cannot answer the central managerial question: what changed because of the campaign?

Campaign measurement is especially difficult because advertising is rarely the only influence on behaviour. Prices, distribution, seasonality, competitors, public relations, product quality, macroeconomic conditions, and previous brand experiences may all affect outcomes. Good research design therefore seeks not merely to observe change, but to estimate the incremental effect attributable to the campaign.

Learning Objectives

By the end of this chapter, the reader should be able to:

1. Construct a campaign measurement plan from business objectives and hypotheses.
2. Distinguish descriptive, diagnostic, predictive, and causal measurement.
3. Select appropriate pre/post, control-group, experimental, and quasi-experimental designs.
4. Specify data requirements, sampling procedures, and survey instruments.
5. Design and interpret brand-lift studies.
6. Evaluate validity, reliability, bias, confounding, and statistical power.
7. Apply ethical and governance principles to campaign research.
8. Translate research findings into managerial decisions about optimisation and investment.

1. From Business Objective to Measurement Hypothesis

A measurement plan begins with a decision, not a metric. A business objective such as “increase profitable acquisition among young professionals” is more useful than “improve campaign performance,” because it identifies an audience, an intended outcome, and an economic purpose.

A practical planning chain is:

Business objective → communication objective → behavioural hypothesis → metric → data source → design → decision rule

For example:

• Business objective: increase profitable subscriptions.
• Communication objective: improve awareness and perceived relevance among non-subscribers.
• Hypothesis: exposed non-subscribers will show greater consideration and subscription intent than comparable unexposed non-subscribers.
• Primary metric: incremental subscription rate.
• Secondary metrics: aided awareness, message association, consideration, and cost per incremental subscriber.
• Data sources: exposure logs, survey responses, subscription records, and media costs.
• Design: randomized holdout or matched exposed/unexposed design, supported by pre/post measurement.
• Decision rule: scale activity only if the incremental return exceeds the organisation’s required threshold.

1.1 Types of measurement questions

A campaign may require all five forms, but they should not be confused. High reach demonstrates delivery, not persuasion. A correlation between exposure and sales does not by itself establish causation. A predictive model can forecast likely purchasers without proving that advertising caused their purchase.

1.2 Hypotheses

A hypothesis is a testable proposition about a relationship or difference. It should identify the population, treatment or exposure, outcome, direction where justified, and time frame.

A suitable hypothesis might be: “Among eligible adults in the target market, exposure to the campaign for at least three impressions during the flight increases aided brand awareness at the end of the campaign compared with no exposure.”

The corresponding null hypothesis states that there is no population-level difference in awareness between the conditions. In statistical notation, if μ₁ is the mean awareness score for the exposed group and μ₀ is the mean for the control group:

The alternative may be one-sided when theory and decision rules justify a directional claim, but two-sided testing is safer when effects could plausibly occur in either direction. Statistical significance does not establish practical importance; a very small effect can be statistically significant in a very large sample.

2. The Measurement Plan as a Research Architecture

A professional measurement plan documents what will be measured, how, when, by whom, and with what limitations. At minimum it should contain:

1. Decision context: the business decision the evidence will inform.
2. Objectives and hypotheses: expected changes and causal logic.
3. Population: the people, households, accounts, or transactions to which findings should generalise.
4. Unit of analysis: person, impression, household, store, region, customer, or time period.
5. Outcomes: primary, secondary, diagnostic, and guardrail metrics.
6. Data sources: first-party, platform, panel, survey, retail, CRM, and financial data.
7. Design: experimental, quasi-experimental, observational, or mixed-method.
8. Timing: baseline, campaign, immediate post-test, and delayed follow-up.
9. Sampling plan: frame, method, quotas, sample size, and weighting.
10. Analysis plan: estimand, models, covariates, missing-data rules, and subgroup tests.
11. Quality controls: validity, reliability, fraud detection, data reconciliation, and auditability.
12. Decision criteria: thresholds for continuation, optimisation, or discontinuation.

2.1 Metrics and estimands

An ordinary metric describes an observed quantity. An estimand specifies the quantity the study intends to estimate. For example, “conversion rate among exposed users” is a descriptive metric, whereas “the average incremental conversion rate caused by campaign exposure among eligible users” is a causal estimand.

For a binary outcome, the average treatment effect (ATE) is:

Here, Y(1) is the potential outcome if a person receives campaign exposure, Y(0) is the potential outcome without exposure, and E denotes the population average. The notation assumes that potential outcomes are conceptually defined even though each person is observed under only one condition.

The ATE requires assumptions such as consistent treatment definition, no interference between units, and credible assignment of exposure. In a randomized experiment, random assignment supports comparability. In observational research, additional assumptions and modelling are required.

For a binary outcome, the absolute lift is:

where p₁ is the outcome proportion in the exposed or treatment group and p₀ is the outcome proportion in the control group. Relative lift is:

Relative lift is undefined when p₀ is zero and can appear large when the baseline is small. Absolute lift is often more useful for forecasting incremental volume.

3. Research Designs for Campaign Evaluation

3.1 Pre/post designs

A pre/post study measures outcomes before and after a campaign. The change is:

This design is efficient for detecting movement in awareness, consideration, or sales, but it cannot reliably attribute all movement to advertising. A new competitor, price change, holiday period, or news event may have occurred between measurements.

A pre/post design is strongest when the same population is measured consistently, the baseline is close enough to the campaign to remain relevant, the campaign is the major intervention, and external conditions are stable. Repeated cross-sectional surveys may be preferable to a panel when repeated interviewing could itself alter responses. Panels, however, permit individual-level change analysis and can reveal transitions that aggregate cross-sections conceal.

3.2 Control groups

A control group provides a counterfactual: an estimate of what would have happened without campaign exposure. Controls may be created through geographic holdouts, audience suppression, randomized user assignment, or matched comparison areas.

A valid control should resemble the treatment group in all important respects except exposure. Contamination occurs when control members receive campaign exposure through another channel. Non-compliance occurs when assigned treatment is not actually delivered. Both problems weaken the contrast between groups.

3.3 Randomized controlled tests

Randomization assigns eligible units to treatment or control by chance. Under successful randomization, systematic pre-existing differences are expected to balance in the long run. The intent-to-treat effect compares outcomes according to assignment, even if some assigned individuals do not comply. This preserves the benefit of randomization and is usually the primary estimate for managerial decisions.

Randomized designs may be implemented through:

• User-level holdouts: selected users are deliberately excluded from campaign delivery.
• Geo experiments: regions are randomly assigned to treatment and control.
• Platform experiments: an advertising platform randomly withholds or varies exposure.
• Time-based tests: campaign intensity is varied across randomly selected time periods, although temporal interference can be substantial.

Randomization does not solve every problem. It may be expensive, ethically constrained, operationally difficult, or unsuitable when treatment spills across units. It also estimates the effect under the tested conditions, which may not generalise to all markets or future creative executions.

3.4 Difference-in-differences

When randomization is not possible, a quasi-experimental difference-in-differences design compares changes over time in treatment and comparison groups:

T denotes the treatment group and C the comparison group. The key assumption is parallel trends: absent the campaign, treatment and comparison groups would have followed similar average trends. This assumption should be examined using multiple pre-treatment periods whenever available. A simple pre/post difference in one group does not provide the same protection.

A regression representation is:

Y_it is the outcome for unit i at time t; Treatment_i indicates membership in the treated group; Post_t indicates the post-campaign period; ε_it is the error term. The interaction coefficient δ estimates the difference-in-differences effect under the design assumptions. Standard errors should account for the level at which treatment was assigned, especially in geo experiments.

3.5 Matched and observational designs

In observational measurement, exposure is not randomly assigned. People who receive more advertising may already be more interested, more digitally active, or more likely to buy. Matching, stratification, regression adjustment, inverse-probability weighting, and propensity scores can reduce imbalance on observed variables, but they cannot remove unmeasured confounding.

A propensity score is the probability of receiving treatment conditional on observed covariates. It can support comparison among units with similar measured characteristics. It should not be presented as a substitute for randomization, and balance diagnostics are essential.

4. Sampling and Survey Design

4.1 Defining the population and sampling frame

The target population is the group to which the study intends to generalise. The sampling frame is the operational list or system from which units are selected. These may differ. For example, a customer database excludes prospects, while an online access panel may underrepresent people with limited internet access.

Sampling error is uncertainty arising because a sample rather than the entire population is observed. Coverage error occurs when parts of the population have no or unequal chance of selection. Nonresponse error occurs when respondents differ systematically from nonrespondents. Weighting can adjust known imbalances but cannot guarantee correction for unmeasured differences.

4.2 Probability and non-probability sampling

Probability methods include simple random, systematic, stratified, and cluster sampling. They support defensible sampling-error estimation when selection probabilities are known. Non-probability methods include quota, convenience, purposive, and online opt-in sampling. They may be practical and inexpensive but require cautious claims about generalisation.

Stratification is useful when important subgroups require adequate representation. Weights can restore population composition. A weighted estimate of a mean is:

where y_i is the response for unit i and w_i is its analysis weight. Weights must be documented because they affect estimates and usually increase variance.

4.3 Sample size and statistical power

Statistical power is the probability of detecting an effect of a specified size under a chosen significance level, assuming the effect exists. It depends on effect size, sample size, outcome variability, study design, and the significance threshold.

For a simple comparison of two independent means with equal group sizes, an approximate per-group sample size is:

σ is the common standard deviation, ∆ is the minimum detectable difference, α is the Type I error rate, β is the Type II error rate, and z denotes the relevant standard-normal quantile. This approximation assumes independent observations, equal variances, a continuous outcome, and a sufficiently large sample.

For two independent proportions, planning should use the expected baseline proportion, the minimum meaningful lift, the allocation ratio, and the desired power. In practice, software or a validated statistical package is used because exact formulas vary by test, design effect, clustering, weighting, and finite population considerations.

If a survey expects 40% response but needs 1,000 completed interviews, the number invited is approximately:

This calculation assumes the response rate estimate is realistic and that all completed responses are usable. Oversampling may be necessary for small but strategically important segments.

Power should be determined before data collection, not increased after observing results. Multiple subgroup tests and repeated interim checking can inflate false-positive risk. Pre-specifying primary outcomes, adjustment procedures, and stopping rules improves credibility.

4.4 Survey questionnaire design

A campaign survey should measure only information relevant to the research objectives. Respondent burden increases satisficing, straight-lining, breakoff, and careless responses. Questions should use plain language, avoid double-barrelled wording, and specify a reference period where recall is required.

Poor question: “How appealing and trustworthy is the brand?”

Better questions:

• “How appealing is the brand?”
• “How trustworthy is the brand?”

Leading wording, assumptions, confusing scales, and forced answers can create measurement error. Randomising the order of unaided brand questions and answer options can reduce order effects. A “don’t know” or “not applicable” option is appropriate when respondents may genuinely lack information.

Brand awareness may be measured unaided (“Which brands come to mind?”) and aided (“Have you heard of any of the following brands?”). Consideration and purchase intent are useful intermediate outcomes, but they are not equivalent to behaviour. Survey measures should therefore be linked, where possible, to observed outcomes.

5. Brand-Lift Research

Brand-lift research estimates whether advertising changes perceptions or intentions. A typical study compares respondents exposed to the campaign with a control group that was not exposed, measuring awareness, ad recall, message association, favourability, consideration, preference, or intent.

A basic brand-lift estimate is:

For example, if 52% of exposed respondents consider the brand and 47% of controls do so, the absolute lift is 5 percentage points. The relative lift is approximately 5 ÷ 47 = 10.6%. The absolute result is generally clearer for planning incremental people or actions.

A credible brand-lift study requires:

1. A clearly defined target population.
2. Reliable exposure classification.
3. A control condition with minimal contamination.
4. Consistent questionnaire wording and timing.
5. Adequate sample size for the primary outcome.
6. Adjustment for important imbalances where appropriate.
7. Clear distinction between statistical and practical significance.

Self-reported ad recall is not the same as verified exposure. Recall can be influenced by prior brand interest and the survey itself. Platform-reported exposure may be technically precise but can omit cross-platform exposure. The strongest approach combines delivery logs, privacy-preserving identity resolution, and survey evidence while documenting linkage limitations.

6. Validity, Reliability, and Bias

6.1 Validity

Construct validity asks whether a measure represents the intended concept. A single question may not adequately measure brand trust, which is multidimensional.

Internal validity concerns whether the observed effect can reasonably be attributed to the campaign rather than confounding factors. Randomization, credible controls, temporal design, and manipulation checks strengthen internal validity.

External validity concerns generalisation across audiences, markets, devices, time periods, and creative contexts. A highly controlled experiment may have strong internal validity but limited generalisability.

Statistical conclusion validity concerns whether the analysis supports the stated inference. Low power, inappropriate tests, non-independence, missing data, and multiple comparisons can undermine it.

6.2 Reliability

Reliability is the consistency of measurement. Internal consistency may be assessed for multi-item scales; test–retest reliability examines stability over time; inter-rater reliability is relevant when human coders classify creative content. Reliability is necessary but not sufficient for validity: a consistently biased scale can be reliable but invalid.

A common internal-consistency coefficient is Cronbach’s alpha, but it should not be treated as proof of unidimensionality or validity. Researchers should examine item content, factor structure, and whether the scale is appropriate for the construct and population.

6.3 Sources of bias

Important biases include:

• Selection bias: treatment and control differ because of how units enter the study.
• Nonresponse bias: respondents differ from nonrespondents.
• Recall bias: respondents inaccurately remember exposure or behaviour.
• Social desirability bias: respondents provide socially acceptable answers.
• Survivorship bias: only continuing customers or visible outcomes are analysed.
• Platform bias: measurement excludes channels or users not observable by a platform.
• Attribution bias: credit is assigned to the easiest-to-measure touchpoint rather than the causal driver.
• Confirmation bias: analysts privilege results supporting the campaign narrative.
• Peeking and p-hacking: repeated analysis choices are made until a favourable result appears.

Bias reduction requires design controls, transparent documentation, preregistration where feasible, independent review, and reporting of null and adverse results.

7. Integrating Data Sources and Analytics

Campaign measurement commonly combines media delivery data, web or app analytics, CRM records, retail transactions, customer surveys, and cost data. Integration requires consistent identifiers, time zones, campaign definitions, exclusion rules, and privacy controls.

A measurement dictionary should define every field, unit, source, refresh frequency, owner, and quality test. Reconciliation checks should compare platform impressions with ad-server records, campaign dates with billing records, and conversions with transaction systems.

Artificial intelligence can assist with questionnaire coding, creative classification, anomaly detection, forecasting, and synthetic control construction. However, automated systems can reproduce sampling bias, misclassify language or sentiment, and obscure analytical assumptions. Human review, model validation, data lineage, and disclosure are necessary when AI affects consequential decisions.

Marketing mix modelling can estimate the relationship between aggregate media investment and business outcomes over time, while multi-touch attribution allocates observed digital conversions across touchpoints. Neither automatically identifies causal effects. Both are sensitive to omitted variables, measurement error, correlated media activity, adstock assumptions, and changing market conditions. Experimental calibration and triangulation are preferable to reliance on a single model.

8. Illustrative Case: The Northstar Subscription Campaign

The following is a fictional illustrative case.

Background

Northstar, a subscription-based financial education service, planned an eight-week campaign targeting adults aged 25–44 in three metropolitan regions. Its objective was to increase profitable annual subscriptions, not merely website traffic. The campaign used video, search, social, and audio placements.

Problem

The media agency proposed reporting reach, clicks, attributed conversions, and platform return on ad spend. The finance team argued that these metrics could overstate incremental subscriptions because high-intent users might have subscribed without advertising.

Analysis and design

The team defined the primary estimand as the incremental annual subscription rate among eligible non-subscribers. Secondary outcomes were aided awareness, consideration, trial starts, and cost per incremental subscriber.

Within each region, eligible users were randomly assigned to a treatment group eligible for campaign delivery or a holdout group suppressed from Northstar advertising. Assignment was conducted before the campaign, and exposure logs were retained. A baseline survey measured awareness and consideration. A post-campaign survey used the same core measures, with treatment and holdout respondents sampled from the eligible population.

The team also monitored contamination, unequal delivery, and pre-campaign balance. Because assignment occurred at user level, standard errors were adjusted for the platform’s assignment structure. The analysis plan specified subscription as the primary outcome and limited subgroup analysis to pre-specified age bands.

Results

Suppose the treatment group produced a subscription rate of 4.8% and the holdout group 4.0%. The absolute incremental rate was:

If 200,000 eligible users were treated, the estimated incremental subscriptions were:

If incremental contribution profit per subscription was £24 and campaign cost was £28,000, estimated incremental contribution was £38,400. The incremental return on campaign cost was:

This result is not the same as platform-attributed return on ad spend. It depends on the assumptions that the holdout represented the counterfactual, contamination was limited, the profit estimate was appropriate, and the measured effect generalises to the treated population.

The brand-lift survey found a 6-point increase in consideration among exposed respondents relative to controls, but the confidence interval was wide for a small subgroup. Management therefore treated the overall consideration result as directional and relied primarily on the randomized subscription estimate for budget allocation.

Outcome and learning points

Northstar continued the campaign but shifted budget toward placements with strong incremental effects rather than those with the highest last-click conversion rate. It retained a small holdout in the next quarter to test durability and possible frequency saturation.

The case demonstrates five principles: define the decision before selecting metrics; use a counterfactual where causal attribution matters; distinguish intermediate brand outcomes from commercial outcomes; estimate economic value using incremental rather than attributed results; and communicate uncertainty rather than presenting point estimates as facts.

9. Governance, Ethics, and Professional Practice

Campaign research uses personal, behavioural, and sometimes sensitive data. Researchers should apply purpose limitation, data minimisation, lawful processing, security, retention controls, and transparent participant communication. Consent should be meaningful where required, and respondents should not be deceived in ways that create material harm.

Identity resolution and cross-device measurement require particular care. Data should not be joined merely because technically possible. Access should be role-based, identifiers should be pseudonymised where feasible, and sensitive segments should not be used to target or exclude people unfairly.

Research reports should disclose sampling limitations, missing data, weighting, exposure definitions, multiple testing, confidence intervals, and deviations from the analysis plan. A result that is commercially inconvenient may still be methodologically valuable. Ethical measurement protects both participants and the long-term credibility of the advertising function.

10. Best-Practice Checklist

Before fieldwork, confirm that:

• the primary business decision and estimand are explicit;
• the target population and unit of analysis are defined;
• hypotheses and primary outcomes are pre-specified;
• the design includes a credible counterfactual where possible;
• the sample is large enough for the minimum meaningful effect;
• survey questions have been cognitively tested and piloted;
• exposure, contamination, and non-compliance rules are documented;
• privacy, consent, security, and retention requirements are satisfied;
• analysis procedures and subgroup tests are recorded in advance.

Before reporting, confirm that:

• data sources have been reconciled;
• balance and attrition have been assessed;
• confidence intervals accompany point estimates;
• statistical significance is separated from business significance;
• null findings and limitations are reported;
• incremental outcomes are distinguished from attributed outcomes;
• recommendations follow from the design’s actual evidence.

Chapter Summary

Measurement planning converts business objectives into hypotheses, metrics, data requirements, research designs, and decisions. Descriptive metrics show what happened, whereas causal designs seek to estimate what happened because of the campaign. Pre/post studies are useful but vulnerable to history and seasonality; control groups, randomization, and carefully justified quasi-experiments provide stronger counterfactuals.

A sound plan defines the target population, unit of analysis, estimand, primary outcome, timing, sampling frame, sample size, and analysis rules before data collection. Survey quality depends on clear constructs, neutral wording, appropriate scales, pilot testing, and attention to nonresponse and recall. Brand-lift studies can measure changes in awareness, consideration, and perceptions, but self-reported exposure and intermediate intentions should not be confused with verified behaviour.

Validity concerns whether conclusions are accurate and generalisable; reliability concerns measurement consistency. Selection, nonresponse, recall, attribution, platform, and confirmation biases must be actively managed. Statistical power should be planned in advance, and confidence intervals, practical significance, and economic value should accompany significance tests. Ethical governance requires privacy, transparency, proportionality, security, and honest reporting. The most credible campaign measurement systems triangulate experiments, surveys, platform data, commercial outcomes, and financial analysis rather than relying on a single dashboard metric.

Key Terms

Attribution: Allocation of observed outcomes to advertising touchpoints or channels.

Average treatment effect (ATE): Average difference between potential outcomes under treatment and no treatment.

Bias: Systematic error that shifts estimates away from the quantity of interest.

Brand lift: Difference in a brand outcome between exposed and comparison groups.

Control group: Units used to estimate outcomes in the absence of campaign treatment.

Counterfactual: The unobserved outcome that would have occurred under an alternative condition.

Difference-in-differences: Comparison of changes over time between treatment and comparison groups.

Estimand: Precisely defined quantity a study aims to estimate.

External validity: Extent to which findings generalise beyond the study conditions.

Internal validity: Credibility that the observed effect was caused by the intervention.

Nonresponse bias: Bias resulting when respondents differ systematically from nonrespondents.

Potential outcome: Outcome a unit would experience under a specified treatment condition.

Power: Probability of detecting a specified effect under stated assumptions.

Pre/post study: Design comparing outcomes before and after a campaign.

Randomization: Assignment to conditions by chance.

Reliability: Consistency or stability of a measurement.

Sampling frame: Operational list or system from which a sample is selected.

Statistical significance: Evidence, under a specified model, against a null hypothesis.

Validity: Degree to which a measure or inference represents what it claims to represent.

Weighting: Applying analysis factors to align a sample or estimate with a target population or selection probability.

Review Questions

Short Answer Questions

1. Why is a measurement plan stronger when it begins with a business decision rather than a media metric?
2. Distinguish absolute lift from relative lift.
3. What is the purpose of a control group in campaign research?
4. Explain the difference between reliability and validity.
5. Name two sources of bias in brand-lift research.

Long Answer Questions

1. Explain how a campaign objective can be translated into a hypothesis, estimand, metric, data source, and decision rule.
2. Compare pre/post, randomized controlled, matched observational, and difference-in-differences designs.
3. Discuss the assumptions and limitations of a difference-in-differences study.
4. Explain how sampling, questionnaire design, nonresponse, and weighting affect the credibility of a campaign survey.
5. Discuss why platform-attributed conversions should not automatically be interpreted as incremental conversions.

Analytical / Application Questions

1. A campaign produces a 7% conversion rate among exposed users and 5% among a holdout group. Calculate the absolute and relative lift, and explain which is more useful for estimating incremental volume.
2. Design a brand-lift study for a national campaign in which individual-level exposure data are unavailable. Specify the population, sampling approach, control strategy, outcomes, threats to validity, and mitigation steps.
3. A campaign is launched in four cities while four similar cities remain untreated. Historical data show that the treated cities had faster sales growth before launch. Evaluate whether a simple difference-in-differences estimate is credible and propose additional analyses or design changes.

Multiple Choice Questions with Answers

Question 1

Which statement best describes a causal estimand?

A. The number of impressions delivered by a campaign

B. The average outcome difference caused by treatment compared with no treatment

C. The percentage of respondents who completed a survey

D. The total media cost divided by reach

Correct Answer: B. The average outcome difference caused by treatment compared with no treatment

Question 2

A campaign produces a 6% conversion rate in treatment and 4% in control. What is the absolute lift?

A. 1 percentage point

B. 2 percentage points

C. 50 percentage points

D. 150 percentage points

Correct Answer: B. 2 percentage points

Question 3

What is the key identifying assumption in a standard difference-in-differences design?

A. Treatment and control must have identical sample sizes

B. All respondents must recall the advertisement

C. Treatment and comparison groups would have followed parallel trends without treatment

D. The post-period must be longer than the pre-period

Correct Answer: C. Treatment and comparison groups would have followed parallel trends without treatment

Question 4

Which design generally provides the strongest basis for causal attribution when implemented correctly?

A. Randomized controlled test

B. Convenience survey conducted after the campaign

C. Last-click attribution

D. Unaided recall analysis without a control group

Correct Answer: A. Randomized controlled test

Question 5

What does statistical power represent?

A. The probability of detecting a specified effect when it exists

B. The probability that the null hypothesis is true

C. The percentage of media budget spent on research

D. The number of variables in a regression model

Correct Answer: A. The probability of detecting a specified effect when it exists

Question 6

Which problem occurs when control-group members are exposed to the campaign through another channel?

A. Stratification

B. Contamination

C. Reliability

D. Oversampling

Correct Answer: B. Contamination

Question 7

Which question is double-barrelled and should normally be separated?

A. How often did you purchase the product last month?

B. Which brands come to mind?

C. How appealing and trustworthy is the brand?

D. Have you heard of this brand?

Correct Answer: C. How appealing and trustworthy is the brand?

Question 8

A survey needs 1,000 completed interviews and expects a 40% response rate. Approximately how many invitations are required?

A. 400

B. 1,400

C. 2,000

D. 2,500

Correct Answer: D. 2,500

Question 9

Which statement about reliability is correct?

A. Reliability guarantees causal validity

B. Reliability concerns consistency of measurement

C. Reliability eliminates sampling error

D. Reliability means a result is statistically significant

Correct Answer: B. Reliability concerns consistency of measurement

Question 10

Why can platform-attributed conversions overstate campaign impact?

A. Platforms never record conversions

B. All attributed users would otherwise have converted

C. Some users may have converted without advertising, creating a counterfactual problem

D. Conversion rates cannot be calculated from platform data

Correct Answer: C. Some users may have converted without advertising, creating a counterfactual problem