Quantitative Research, Sampling and Measurement

Introduction

Quantitative research converts observations, opinions, behaviours and market characteristics into numerical data that can be summarised and analysed. In marketing research and advertising, it is used to estimate market size, measure brand awareness, compare customer segments, test advertising effectiveness, forecast demand and evaluate product decisions.

A quantitative study is only as credible as its design. A large sample cannot rescue a poorly worded questionnaire, an unsuitable sampling frame or a measure that does not represent the construct being studied. This chapter therefore treats research design as an integrated process: define the decision problem, specify the population, select a defensible sample, operationalise concepts, construct the instrument, collect data ethically and assess both sampling and non-sampling error.

Learning Objectives

By the end of this chapter, you should be able to:

• distinguish exploratory, descriptive and causal quantitative research;
• formulate survey and experiment designs for product and advertising decisions;
• define target populations, sampling frames and units of analysis;
• compare probability and non-probability sampling methods;
• estimate practical sample requirements and explain the assumptions behind them;
• construct valid and reliable scales and questionnaires;
• identify and reduce sampling and non-sampling errors; and
• apply these principles to a professional marketing research problem.

1. Quantitative Research Design

1.1 From management problem to research question

A management problem is expressed in action terms, such as “sales of the new product are below expectations.” A research problem translates that concern into information requirements: “What is the level of awareness, perceived value and purchase intention among the intended segment, and which factors predict trial?” The second formulation is measurable and guides decisions about variables, respondents and analysis.

A useful research brief specifies:

1. the decision-maker and decision to be supported;
2. the target population;
3. the constructs or variables to measure;
4. the required comparisons or relationships;
5. the time period and geographical scope; and
6. the accuracy, budget and deadline constraints.

1.2 Descriptive and causal designs

Descriptive research measures the characteristics of a population or phenomenon. A cross-sectional survey can estimate the proportion of customers who recognise a brand, while a longitudinal study measures change over time. Descriptive claims should not be presented as causal claims.

Causal research examines whether changing one factor produces a change in an outcome. The strongest design is a randomised experiment. Participants may be randomly assigned to an advertisement or control condition, with purchase intention or click-through behaviour then compared. Random assignment balances, on average, both observed and unobserved characteristics across groups.

A basic experiment contains:

• an independent variable, such as advertisement version;
• a dependent variable, such as recall or conversion;
• treatment and comparison conditions;
• random assignment where feasible; and
• controls against alternative explanations.

Quasi-experiments are useful when random assignment is impractical. For example, one region may receive a campaign while another similar region does not. However, pre-existing regional differences may explain the outcome, so baseline measures and statistical controls are important.

1.3 Surveys and experiments compared

A survey may be cross-sectional, collecting data once, or longitudinal, collecting repeated observations. A panel follows the same respondents; a repeated cross-sectional design samples different respondents from the same population at different times. Panels measure individual change but may suffer from attrition and conditioning.

2. Populations, Sampling Frames and Samples

The target population is the complete set of units to which the researcher wishes to generalise. A unit may be a person, household, business, transaction, advertisement impression or store. The sampling unit is the element selected at a particular stage, and the unit of analysis is the entity about which conclusions are made. These may differ: stores may be sampled, but individual customers may be the units of analysis.

A sampling frame is the operational list or source from which the sample is drawn. Examples include a verified customer database, a business registry or a carefully constructed list of households. A frame may be incomplete, duplicated, outdated or contain units outside the target population. Frame coverage must therefore be assessed before sampling.

A population parameter is a numerical characteristic of the target population, such as the true proportion who prefer a product. A sample statistic is the corresponding value calculated from the sample. Sampling seeks to use the statistic to estimate the parameter.

2.1 Inclusion and exclusion criteria

Researchers should state who is eligible. For a survey of a streaming service, eligibility might require being at least 18 years old, having used the service in the past three months and residing in the market under study. Clear criteria reduce ambiguity and support reproducibility.

3. Sampling Methods

3.1 Probability sampling

In probability sampling, each eligible unit has a known, non-zero probability of selection. This makes sampling error estimable and supports statistical generalisation, provided execution is sound.

Simple random sampling gives every unit an equal selection probability. It is conceptually straightforward but requires a complete frame.

Systematic sampling selects every kth unit after a random starting point. If the frame contains N units and n are required, the interval is approximately:

This method is efficient, but periodic patterns in the list can bias selection.

Stratified sampling divides the population into mutually exclusive, collectively exhaustive strata, such as age groups or regions, and samples within each stratum. It ensures representation and can improve precision when members within strata are similar on the measured variable. Proportional allocation uses each stratum’s population share. Disproportionate allocation deliberately oversamples a small but strategically important group; weights are then normally needed for population estimates.

Cluster sampling selects naturally occurring groups, such as stores, schools or neighbourhoods, and studies units within selected clusters. It reduces travel and administration costs, but units within the same cluster tend to resemble one another, reducing the information gained from each additional respondent.

Multistage sampling combines stages, such as selecting regions, then stores, then customers. It is practical for dispersed populations but requires careful calculation of selection probabilities.

3.2 Non-probability sampling

In non-probability sampling, selection probabilities are unknown. It is often appropriate for exploratory work, rapid concept screening or populations without a usable frame, but estimates should not automatically be treated as representative.

• Convenience sampling recruits accessible participants.
• Purposive sampling selects people who meet a relevant expertise or experience criterion.
• Quota sampling fills specified subgroup counts without random selection within groups.
• Snowball sampling asks participants to refer others; it can reach hidden populations but may produce network bias.
• Opt-in online panels are efficient but may overrepresent people who frequently participate in research.

Quotas can improve demographic balance, but matching marginal demographics does not guarantee representativeness on attitudes, usage intensity or unobserved characteristics.

3.3 Choosing a method

The choice depends on the decision, population accessibility, required generalisability, budget, time and ethical considerations. For a national market estimate, a probability design is preferable. For early product discovery, purposive recruitment may be more useful because the goal is learning from relevant users rather than estimating a population percentage.

4. Sample Size and Precision

Sample size should be determined by the intended inference, variability, confidence, precision, design complexity, expected response rate and available resources. “More respondents” is not automatically better if the sample is systematically biased.

For a simple random sample estimating a population proportion, the approximate sample size for a large population is:

Here, n₀ is the required sample size before finite-population adjustment, z is the standard-normal critical value for the selected confidence level, p is the anticipated population proportion, and e is the desired maximum margin of error expressed as a proportion. The formula assumes a probability sample, independent observations, a proportion estimate and a simple random-sampling design.

When no prior estimate of p is available, p = 0.50 is conservative because p(1 − p) is then largest. For 95% confidence, z ≈ 1.96. If e = 0.05:

The result is rounded up to 385 respondents. This is a precision calculation, not a guarantee against coverage, nonresponse or measurement error.

If the population N is small relative to the sample, the finite-population correction is:

For N = 2,000 and n₀ = 384.16:

The minimum completed sample is therefore approximately 323.

If the design uses cluster sampling or another complex design, the simple-random-sample requirement is commonly multiplied by a design effect, D:

The design effect is dimensionless and depends on the sampling design and intracluster similarity. It should be estimated from prior evidence or a defensible planning assumption rather than invented after seeing results.

Finally, recruitment must allow for nonresponse. If the expected response rate is r, expressed as a proportion:

For 400 completed interviews and an expected response rate of 40%, the researcher should seek approximately 1,000 eligible contacts. This adjustment addresses expected completion, not systematic differences between responders and nonresponders.

For experiments, sample size should be based on the smallest practically meaningful effect, desired power, significance level, outcome variability and number of groups. A statistically significant but commercially trivial effect may not justify a campaign change; practical significance must be specified in advance.

5. Measurement and Operationalisation

Measurement assigns numbers or categories to characteristics according to explicit rules. A construct is an abstract concept, such as trust, perceived quality or advertising involvement. An indicator is an observable measure used to represent that construct. Operationalisation defines exactly how the construct will be measured.

A strong operational definition identifies the population, timeframe, response task, scale and scoring rule. “Customer loyalty” is too vague; “likelihood of renewing within the next six months, measured on a seven-point likelihood scale” is more actionable, although it represents intention rather than actual renewal.

5.1 Levels of measurement

A five-point satisfaction item is formally ordinal. Researchers often calculate means for multi-item scales when response categories are reasonably balanced, but they should report the measurement decision and consider distributions and robustness.

5.2 Reliability

Reliability concerns consistency of measurement. Test–retest reliability examines stability over time when the construct should be stable. Inter-rater reliability is relevant when observers code behaviour. Internal consistency examines whether items intended to measure one construct produce coherent responses.

Cronbach’s alpha is commonly reported for multi-item scales:

In this expression, k is the number of items, σᵢ² is the variance of item i, and σₜ² is the variance of the total score formed by summing the items. Alpha assumes items are used to measure a common construct and is influenced by the number of items; a high value does not prove validity or unidimensionality. Very high alpha may indicate redundant wording. Reliability should therefore be considered alongside theory, item content and factor evidence.

5.3 Validity

Content validity asks whether items adequately cover the construct domain. Construct validity concerns whether the measure behaves as theory predicts, including convergent and discriminant evidence. Criterion-related validity examines association with a relevant outcome, such as whether purchase-intention scores predict later trial.

Face validity—whether an item appears appropriate—is useful for comprehension but is weak evidence by itself. A measure can be reliable yet invalid: a badly calibrated scale may produce highly consistent but incorrect responses.

6. Questionnaire Construction

A questionnaire should follow a logical progression and impose minimal cognitive burden. A practical sequence is:

1. introduction, consent and eligibility screening;
2. relevant behaviour and experience;
3. evaluations, attitudes and intentions;
4. experimental stimulus or advertising questions, if applicable;
5. sensitive questions; and
6. demographics and classification variables.

Questions should be specific, concise and neutral. Avoid double-barrelled wording (“How satisfied are you with price and quality?”), leading wording (“How excellent is our service?”), undefined terms (“regularly”), double negatives and assumptions that all respondents have the same experience.

Specify a reference period: “How many times did you purchase the product in the past 30 days?” Use mutually exclusive and exhaustive response categories where appropriate. Include “not applicable,” “don’t know” or “prefer not to answer” when their absence would force inaccurate answers.

Use balanced scales with clearly labelled endpoints. A seven-point agreement item might range from “strongly disagree” to “strongly agree.” Do not rely excessively on reverse-coded items; they can create confusion and artificial factors. If reverse coding is necessary, define the scoring rule before analysis.

Pretesting should include expert review, cognitive interviews and a pilot survey. Cognitive interviewing asks respondents to explain how they interpret a question and select an answer. A pilot checks routing, timing, missing data, scale distributions and technical operation. Pilot data should not automatically be pooled with the main sample if the instrument changes materially.

7. Minimising Error

7.1 Sampling error

Sampling error is random discrepancy between a sample statistic and the population parameter because only part of the population is observed. It generally decreases as a probability sample grows, although the relationship has diminishing returns. The margin of error describes sampling uncertainty under stated assumptions; it does not cover all forms of bias.

Coverage error occurs when the frame excludes eligible units or includes ineligible ones. Sampling error can be reduced through an appropriate probability design, adequate sample size and stratification where justified.

7.2 Non-sampling error

Non-sampling error can occur in a census as well as a sample. Important forms include:

• Nonresponse error: selected units do not participate or omit particular questions.
• Measurement error: the response does not accurately represent the intended construct.
• Processing error: coding, weighting, transcription or software mistakes alter data.
• Interviewer error: interviewer behaviour influences answers.
• Recall error: respondents cannot accurately remember past behaviour.
• Social desirability bias: respondents report what appears acceptable rather than what they believe or did.
• Mode effects: answers differ between online, telephone, face-to-face and mobile formats.

Controls include multiple contact attempts, short and accessible surveys, appropriate incentives, neutral scripts, interviewer training, anonymous response options, validated scales, randomised item or stimulus order where appropriate, logic checks, range checks, audit trails and documented data cleaning. Weighting may correct known differences in selection probabilities or sample composition, but it cannot reliably repair unmeasured bias.

8. Ethics, Privacy and Governance

Participants should receive understandable information about the purpose, procedures, risks, benefits, data use, voluntary participation and withdrawal rights. Informed consent must be appropriate to the population and method. Minors and vulnerable groups require additional safeguards.

Collect only data necessary for the research purpose. Personal identifiers should be separated from analytical data where feasible, protected by access controls and retained only as long as justified. Researchers must comply with applicable privacy and data-protection law, organisational policy and professional codes. Sensitive attributes should not be collected merely because they are technically available.

Advertising experiments require special care when treatment could mislead, distress or disadvantage participants. Deception, if scientifically necessary, must be minimal, approved under applicable governance procedures and followed by debriefing where appropriate. Research results should not be manipulated to support a preferred campaign conclusion.

9. Technology and Contemporary Practice

Online panels, mobile surveys, behavioural logs, customer relationship management data and experiments on digital platforms have expanded quantitative research. Automated data collection improves speed but creates new risks: bots, duplicate respondents, fraudulent participation, device effects, algorithmic targeting and opaque panel recruitment.

AI can assist with questionnaire drafting, translation, coding of open responses and anomaly detection. It should not replace construct definition, ethical review or human validation. Researchers should document the system used, check hallucinated or biased outputs, protect confidential data and preserve a reproducible record of decisions. Platform experiments must also consider interference: one user’s treatment may affect another user through sharing or network effects.

10. Illustrative Case: Testing a New Sustainable Beverage Brand

Background

A beverage company is preparing a low-sugar drink in a competitive urban market. Management must decide whether to launch one of two advertisements: an environmental message or a taste-and-convenience message.

Problem

The company needs estimates of target-segment awareness and purchase intention, together with evidence about whether either advertisement improves intention.

Analysis and approach

The target population is adults aged 18–54 who purchase packaged beverages at least once per month in the launch cities. The team identifies a verified online panel and screens participants against these criteria. For the descriptive component, it uses stratified quotas by city and age to avoid severe imbalance, while acknowledging that the opt-in panel does not support strict probability generalisation.

For the causal component, eligible participants are randomly assigned to one of three conditions: environmental advertisement, taste-and-convenience advertisement or control content. The primary outcome is purchase intention on a seven-point scale; secondary outcomes are unaided ad recall, perceived credibility and willingness to pay. The questionnaire measures prior category usage before exposure, presents the stimulus once, randomises the order of several evaluation items and includes an attention check that does not reveal the preferred answer.

Before fieldwork, ten cognitive interviews identify that “sustainable” is interpreted inconsistently. The term is replaced with a specific description of recyclable packaging and reduced plastic use. A pilot reveals that the original survey takes 15 minutes, so redundant items are removed and the final median completion time falls to eight minutes.

Outcome

The taste-and-convenience advertisement produces higher mean purchase intention than the control, while the environmental advertisement produces a smaller but positive difference. However, the environmental message is more persuasive among respondents who already consider packaging important. The company therefore chooses the taste message for broad launch and tests a targeted environmental version for the relevant segment.

These findings should not be interpreted as proof of sales impact because intention is not purchase behaviour and the online environment may differ from retail exposure. A follow-up market test using geographically separated stores is recommended.

Learning points

The case illustrates the value of separating descriptive measurement from causal testing, defining constructs precisely, pretesting wording, randomising treatment assignment and reporting limitations. It also shows why a statistically detectable effect must be evaluated against commercial relevance, segment differences and implementation conditions.

Best-Practice Checklist

Before launching a quantitative study, confirm that:

• the decision and population are explicitly defined;
• the frame or recruitment source is assessed for coverage;
• the sampling method matches the required generalisation;
• sample size reflects precision, design, power and expected response;
• every construct has a defensible operational definition;
• questions are neutral, specific and tested with intended respondents;
• experiment participants are assigned and outcomes measured consistently;
• consent, privacy, retention and access procedures are documented;
• quality checks address bots, duplicates, missingness and routing errors; and
• limitations and possible sources of bias are reported honestly.

Chapter Summary

Quantitative marketing research provides numerical evidence for product and advertising decisions, but its quality depends on the alignment of problem definition, design, population, sample, measurement and analysis. Surveys are effective for describing characteristics and associations; experiments are better suited to causal claims. Probability sampling supports generalisation when coverage and execution are sound, whereas non-probability methods can be valuable for exploratory and practical purposes but require cautious interpretation.

Sample-size formulas describe precision under assumptions and do not eliminate coverage, nonresponse or measurement bias. Constructs must be operationalised through clear indicators, and measures should be assessed for reliability and validity. Well-designed questionnaires use neutral wording, appropriate response options, pretesting and logical routing. Finally, ethical consent, privacy protection, transparent reporting and controls for sampling and non-sampling error are essential features of professional research.

Key Terms

Target population: The complete set of units to which the researcher intends to generalise.

Sampling frame: The operational list or source from which a sample is selected.

Sampling unit: The unit selected at a particular stage of sampling.

Unit of analysis: The entity about which conclusions are drawn.

Parameter: A numerical characteristic of a population.

Statistic: A numerical characteristic calculated from a sample.

Probability sampling: Sampling in which each eligible unit has a known, non-zero selection probability.

Non-probability sampling: Sampling in which selection probabilities are unknown.

Stratified sampling: Sampling separately within defined population subgroups.

Cluster sampling: Sampling groups of naturally occurring units.

Sampling error: Random discrepancy caused by observing a sample rather than the entire population.

Coverage error: Bias caused by mismatch between the sampling frame and target population.

Nonresponse error: Bias arising when participation or completion differs systematically across units.

Construct: An abstract concept intended for measurement.

Operationalisation: Defining how a construct will be observed and scored.

Reliability: Consistency of a measurement.

Validity: The extent to which a measure supports the intended interpretation.

Design effect: The factor reflecting loss or gain of precision from a complex sampling design.

Margin of error: A stated bound describing sampling uncertainty under specified assumptions.

Random assignment: Allocation of experimental units to conditions by chance.

Review Questions

Short Answer Questions

1. What is the difference between a target population and a sampling frame?
2. Why does random assignment support causal inference?
3. Distinguish reliability from validity.
4. Name two forms of non-sampling error and one way to reduce each.
5. Why might a researcher use stratified sampling?

Long Answer Questions

1. Compare probability and non-probability sampling, including their uses, strengths and limitations in marketing research.
2. Explain how a researcher would design and pretest a questionnaire measuring brand trust.
3. Discuss the assumptions and limitations of the simple sample-size formula for estimating a population proportion.
4. Explain how coverage, nonresponse and measurement errors can jointly affect a campaign evaluation.
5. Compare a cross-sectional survey, a panel study and a randomised advertising experiment.

Analytical / Application Questions

1. A company has a customer population of 2,000 and wants a 95% confidence estimate with a 5% margin of error. Using the large-population proportion formula with p = 0.50 and the finite-population correction, determine the approximate completed sample and explain what the calculation does not control.
2. Design a sampling and measurement plan to assess satisfaction among users of a new mobile application, including the population, frame, sampling method, key constructs, scale choices and error controls.
3. An online advertisement increases average purchase intention, but the sample is an opt-in panel and the outcome is self-reported. Evaluate the strength of the conclusion and recommend a follow-up study.

Multiple Choice Questions with Answers

Question 1

Which feature most directly distinguishes an experiment from a descriptive survey?

A. Use of numerical data

B. Random assignment to treatment conditions

C. Collection of demographic information

D. Use of a questionnaire

Correct Answer: B. Random assignment to treatment conditions

Question 2

A researcher selects every 20th name after choosing a random starting point. Which method is being used?

A. Cluster sampling

B. Quota sampling

C. Systematic sampling

D. Snowball sampling

Correct Answer: C. Systematic sampling

Question 3

For a population proportion with no prior estimate, which value of p is conventionally conservative in the sample-size formula?

A. 0.05

B. 0.10

C. 0.50

D. 0.95

Correct Answer: C. 0.50

Question 4

With 95% confidence, p = 0.50 and a 5% margin of error, the approximate large-population sample size is:

A. 96

B. 196

C. 323

D. 385

Correct Answer: D. 385

Question 5

Which statement about Cronbach’s alpha is correct?

A. It proves that a scale is valid

B. It measures internal consistency under relevant assumptions

C. It is used only for nominal variables

D. It directly measures sampling error

Correct Answer: B. It measures internal consistency under relevant assumptions

Question 6

“How satisfied are you with the product’s price and packaging?” is problematic primarily because it is:

A. Double-barrelled

B. Randomised

C. Nominal

D. A screening question

Correct Answer: A. Double-barrelled

Question 7

Which error occurs when eligible people are missing from the sampling list?

A. Coverage error

B. Recall error

C. Interviewer error

D. Processing error

Correct Answer: A. Coverage error

Question 8

If 500 completed responses are needed and the expected response rate is 25%, approximately how many eligible contacts are required?

A. 125

B. 625

C. 1,250

D. 2,000

Correct Answer: D. 2,000

Question 9

Which level of measurement has categories with order but not necessarily equal intervals?

A. Nominal

B. Ordinal

C. Interval

D. Ratio

Correct Answer: B. Ordinal

Question 10

What is the most defensible interpretation of a higher purchase-intention score after exposure to an advertisement in a randomised test?

A. The advertisement definitely increased sales

B. The advertisement caused the measured intention difference, subject to study validity

C. The sample is automatically representative of all consumers

D. The measure has no possible bias

Correct Answer: B. The advertisement caused the measured intention difference, subject to study validity