Market Research and Customer Insight

Introduction and context

Market research is the systematic process of defining a marketing problem, collecting evidence, analysing it, and communicating findings for decision-making. Customer insight is the managerial interpretation of that evidence: an explanation of what customers do, why they do it, which conditions influence their behaviour, and what the organisation should do next. Research produces evidence; insight connects evidence to action.

In marketing analytics, research is not limited to surveys. It combines qualitative methods, quantitative measurement, behavioural data, observation, social listening, and text analysis. A strong research programme begins with a decision rather than a dataset. For example, “Should the company redesign its onboarding journey?” is a useful decision question, whereas “Find out what customers think” is too vague to guide research design.

Learning objectives

By the end of this chapter, students should be able to:

1. Distinguish management problems, research problems, and research objectives.
2. Select appropriate qualitative, quantitative, observational, and digital research methods.
3. Design samples and questionnaires while recognising sources of bias and measurement error.
4. Apply descriptive statistics, hypothesis tests, and introductory multivariate methods.
5. Use social listening and text analysis responsibly.
6. Translate research findings into defensible customer and market insights.
7. Evaluate ethical, privacy, validity, and governance issues in marketing research.

From business problem to research design

A research design is the overall plan for obtaining evidence. It should align four elements:

Exploratory research is appropriate when the problem is poorly understood. Interviews, focus groups, ethnography, and open-ended text analysis can reveal language, motives, and unexpected issues. Descriptive research measures characteristics such as awareness, satisfaction, or purchase frequency. Causal research examines whether an intervention produces a change, usually through experiments or quasi-experimental designs.

A useful framework is the mixed-method sequence. Qualitative work can identify constructs and generate questionnaire items; quantitative research can estimate prevalence and test relationships; a second qualitative stage can explain surprising statistical patterns. This is stronger than treating qualitative and quantitative methods as competitors. They answer different kinds of questions.

Research quality

Reliability concerns consistency. A reliable measure tends to produce similar results under comparable conditions. Validity concerns whether the measure represents the intended concept. A scale may be reliable but invalid: for example, a consistently worded question may measure general mood rather than service satisfaction.

Researchers should distinguish:

• Construct validity: whether an operational measure represents the theoretical concept.
• Internal validity: whether a claimed cause plausibly produced the observed outcome.
• External validity: whether findings generalise to other people, settings, or times.
• Statistical conclusion validity: whether the analysis supports the stated relationship.

A research brief should define the target population, unit of analysis, variables, timing, method, decision criteria, and limitations before data collection begins.

Qualitative research and customer meaning

Qualitative research seeks depth, context, and meaning rather than population estimates. Common approaches include:

• Depth interviews: flexible one-to-one conversations useful for sensitive experiences and personal decision processes.
• Focus groups: moderated discussions that reveal shared language, disagreement, and social norms. Group conformity and dominant participants are risks.
• Observation and ethnography: study behaviour in natural settings, including workarounds that respondents may forget or fail to report.
• Diary studies: participants record experiences over time, reducing reliance on retrospective recall.
• Journey research: maps stages, touchpoints, emotions, effort, and breakdowns across an experience.

A qualitative interview guide should contain open questions, probes, and a logical progression from context to experience to meaning. “Tell me about the last time you used the service” is generally better than “Was the service convenient?” because it elicits an episode rather than imposing a judgment.

Analysis commonly involves familiarisation, coding, category development, comparison across cases, and theme interpretation. Codes should remain close to the data initially; later, related codes can be grouped into higher-level themes. Good qualitative analysis preserves negative cases and contradictory evidence. Quotations illustrate a theme but do not, by themselves, establish prevalence.

Observation requires a clear protocol. Researchers should specify the setting, behaviours of interest, observation period, recording method, and rules for researcher intervention. Covert observation may raise serious ethical concerns, particularly where people reasonably expect privacy.

Sampling and fieldwork

The target population is the full group about which conclusions are intended. A sampling frame is the operational list or process used to reach that population. The sample is the selected subset, and the sampling unit is the entity selected, such as a person, household, account, or transaction.

Probability and non-probability sampling

Probability sampling gives each eligible unit a known, non-zero probability of selection. Simple random, systematic, stratified, and cluster sampling are common forms. Stratification can improve representation of important subgroups, while cluster sampling can reduce fieldwork cost but often increases sampling variability because units within clusters resemble one another.

Non-probability methods include convenience, purposive, quota, and snowball sampling. They can be appropriate for exploratory research or hard-to-reach communities, but statistical generalisation requires caution because selection probabilities are unknown.

For a proportion estimated by a simple random sample, an approximate 95% confidence interval is:

Here, p̂ is the sample proportion, n is the sample size, and 1.96 is the standard-normal critical value for a two-sided 95% interval. This approximation assumes a probability sample, independent observations, and a sufficiently large sample; a common practical check is that both n p̂ and n(1 − p̂) are at least about 10. It does not account automatically for nonresponse, coverage error, weighting, or complex sample design.

For planning a sample when the anticipated proportion is unknown, p = 0.50 is conservative. A rough margin-of-error formula is:

where z is the chosen standard-normal critical value and e is the desired margin of error. For a 95% interval and e = 0.05, n ≈ 384 when p = 0.50. This is not a guarantee of accuracy: systematic nonresponse or biased coverage can dominate random sampling error.

Worked example: estimating satisfaction

Suppose 400 randomly selected customers respond, and 280 report satisfaction. Then p̂ = 280 ÷ 400 = 0.70. The estimated standard error is:

√(0.70 × 0.30 ÷ 400) ≈ 0.0229

The approximate 95% interval is 0.70 ± 1.96 × 0.0229, or approximately 0.655 to 0.745. The researcher may report that estimated satisfaction is 70%, with an approximate 95% confidence interval from 65.5% to 74.5%, subject to the sampling assumptions.

Questionnaire design and measurement

Questionnaire design is a measurement problem. Each item should measure one idea, use language understood by the target population, and have response options that are exhaustive and mutually exclusive where appropriate.

Poor questions often contain:

• Double-barrelled wording: “How satisfied are you with price and delivery?”
• Leading wording: “How much did you enjoy our excellent service?”
• Ambiguous time frames: “How often do you purchase?”
• Unsupported recall demands: asking for exact details over long periods.
• Unbalanced scales: offering more positive than negative options.

A typical structured instrument moves from screening and behavioural questions to evaluations, attitudes, demographics, and sensitive items. Randomising suitable item or option order can reduce order effects, but logical sequences and “none” or “not applicable” options must remain clear.

Likert-type items measure agreement or another ordered response. A multi-item scale should normally use consistent anchors and be evaluated for reliability. One common measure is Cronbach’s alpha:

Here, k is the number of items, sᵢ² is the variance of item i, and sₜ² is the variance of the total score formed by summing the items. Alpha assumes, approximately, that items reflect a common construct and have reasonably similar relationships with it. A high value does not prove unidimensionality or validity; very high values may indicate redundant items.

Pretesting should assess comprehension, recall, response burden, routing, mobile display, and the interpretation of scales. Cognitive interviewing—asking participants to explain how they understood and answered an item—is especially useful before launch.

Descriptive and inferential analysis

Descriptive statistics summarise observed data. For values x₁, x₂, …, xₙ, the sample mean is:

The sample variance is:

The mean is sensitive to extreme values, whereas the median is often more appropriate for skewed variables such as spending. Frequencies, proportions, cross-tabulations, distributions, and visualisations should be examined before modelling. Missing data, duplicate records, impossible values, and inconsistent coding must be documented and treated deliberately.

A hypothesis test compares an observed result with a null model. For a one-sample mean, a t statistic is:

where μ₀ is the hypothesised population mean, s is the sample standard deviation, and n is the sample size. The test assumes independent observations and, for small samples, an approximately normal population. A p-value is the probability, under the null model, of a result at least as extreme as the observed result. It is not the probability that the null hypothesis is true, nor does it measure managerial importance.

A confidence interval communicates both direction and precision. Researchers should report effect sizes and intervals rather than relying only on a threshold such as p < 0.05. Multiple tests increase the chance of false positives; preregistered hypotheses, correction procedures, and transparent reporting can reduce this risk.

For categorical variables, a chi-square test can assess whether observed counts differ from expected counts under independence:

Oᵢ is an observed cell count and Eᵢ is its expected count. Expected counts should generally be sufficiently large for the chi-square approximation; sparse tables may require exact or combined-category methods. Association does not establish causation.

Multivariate methods for customer insight

Marketing outcomes are usually influenced by several variables simultaneously. Multiple linear regression estimates a conditional relationship:

Here, yᵢ is the outcome for observation i, x variables are predictors, β coefficients represent expected changes holding other included predictors constant, β₀ is the intercept, and εᵢ is the unexplained error. Linear regression requires appropriate functional form, independent errors, limited problematic multicollinearity, and reasonably stable variance; interpretation should also consider selection and omitted variables.

Logistic regression is suitable for a binary outcome such as churn versus retention. It models the log odds:

The exponentiated coefficient, e^βⱼ, is an odds ratio for a one-unit increase in xⱼ, holding other predictors constant. Odds ratios are not risk ratios, and predictive accuracy does not demonstrate causal impact.

Other useful methods include factor analysis for identifying latent dimensions in correlated survey items, cluster analysis for exploratory segmentation, and conjoint analysis for estimating preferences over product attributes. These methods depend heavily on variable selection, scaling, sample quality, and validation. A segment is useful only if it is identifiable, substantial, reachable, stable enough to act upon, and meaningfully different in behaviour or needs.

Social listening and text analysis

Social listening collects and interprets publicly available or permissioned conversations across platforms, reviews, forums, and customer-service channels. It can detect emerging topics, service failures, competitor comparisons, and language used by customers. However, online authors are not necessarily representative of all customers. Volume may reflect platform algorithms, highly vocal minorities, or coordinated activity.

A text-analysis workflow normally includes defining the corpus and time period, removing duplicates, preserving relevant context, handling spelling and slang, coding themes or sentiment, validating automated classifications, and linking findings to business decisions. Sentiment tools may misread sarcasm, negation, multilingual text, and domain-specific language. Topic models identify patterns in word use; they do not automatically reveal motives or causal mechanisms.

Human review is essential for training samples, quality checks, sensitive categories, and ambiguous cases. Researchers should record sampling rules, model versions, confidence thresholds, exclusions, and error rates. “Sentiment” is not the same as satisfaction, and a positive mention may still contain a serious complaint.

Translating evidence into insight

An insight is more than a finding. A finding might be “38% of respondents abandoned checkout.” An insight interprets the significance: “Time-sensitive shoppers abandon checkout when delivery costs appear late in the process, suggesting that early cost transparency may protect conversion.” A useful insight typically contains:

1. Evidence: what was observed.
2. Interpretation: what it means for the customer.
3. Tension or opportunity: why the issue matters.
4. Action implication: what should be tested or changed.
5. Confidence and limitation: how strongly the evidence supports the claim.

Triangulation compares independent evidence sources. For example, interview frustration, observed checkout exits, and support-ticket language may converge on delivery-cost confusion. Divergence is also informative: customers may report satisfaction while behavioural data shows low repeat use, prompting investigation of measurement or hidden friction.

A customer journey map should connect stages, goals, touchpoints, emotions, effort, pain points, and evidence. Prioritisation can consider customer impact, commercial value, feasibility, and confidence. Recommendations should specify an owner, proposed intervention, success metric, test design, and review date.

Illustrative case: Northstar Fitness app

Background. Northstar Fitness, a fictional subscription app, has strong trial sign-ups but weak conversion to paid membership. Management initially assumes that price is the main problem.

Problem. The organisation needs to decide whether to reduce price, improve onboarding, or redesign the trial experience.

Analysis. Researchers conduct 18 depth interviews with recent trial users, analyse 12,000 anonymised event records, survey 600 eligible trial users, and review 4,000 support messages. Interviews reveal that users value personalised plans but do not understand how to start. Event data shows that users who complete the baseline assessment are more likely to reach a workout plan. The survey finds that 42% report difficulty understanding the first steps. Text analysis identifies “where do I begin?” as a recurring support theme.

The evidence is triangulated, but the team avoids claiming that onboarding alone causes conversion. Assessment completion may be associated with motivation or prior fitness experience. A logistic regression can adjust for observed factors such as device, acquisition channel, and prior activity, but unmeasured differences remain possible.

Approach. Northstar designs an experiment in which eligible new users are randomly assigned to the existing onboarding flow or a simplified flow with a progress explanation, immediate plan recommendation, and visible next action. The primary outcome is paid conversion within 30 days; secondary outcomes include assessment completion, first-week activity, support contacts, and cancellation. The team monitors adverse effects, such as users being pushed into unsuitable plans.

Outcome. In this illustrative scenario, the simplified flow increases assessment completion and paid conversion without changing price. The company retains the original price, rolls out the new flow gradually, and continues monitoring retention. The result supports an onboarding intervention, not a universal conclusion that price is irrelevant.

Learning points. The case demonstrates problem definition, mixed methods, behavioural validation, cautious causal reasoning, experimentation, and the difference between a descriptive finding and an actionable insight.

Ethics, governance, and contemporary practice

Marketing research should obtain informed consent where required, minimise data collection, explain intended uses, protect confidentiality, and provide appropriate withdrawal or access mechanisms. Personal data should be processed under applicable law and organisational policy. Sensitive attributes require heightened safeguards, and re-identification risk can persist after direct identifiers are removed.

Researchers must avoid deceptive recruitment, coercive incentives, discriminatory targeting, and manipulative question wording. Data quality does not excuse unethical collection. Access controls, retention limits, audit trails, documented data lineage, and secure deletion are practical governance controls.

AI can assist with transcription, coding, translation, survey routing, anomaly detection, and summarisation. It can also introduce hallucination, opaque classification, privacy leakage, automation bias, and unequal error rates across groups. Human review, representative validation sets, version control, protected data environments, and disclosure of material automation are therefore necessary. Generative systems should not be treated as respondents, neutral analysts, or substitutes for research judgment.

Best practice is to maintain a research repository containing instruments, sampling decisions, codebooks, exclusions, analysis scripts, model specifications, and limitations. Reproducibility and transparent uncertainty improve the credibility of customer insight.

Chapter Summary

Market research links a business decision to systematic evidence. Exploratory qualitative methods reveal meanings and mechanisms, while quantitative methods estimate prevalence and test relationships. Sampling quality depends not only on sample size but also on coverage, selection, nonresponse, and measurement. Well-designed questionnaires avoid ambiguity and support valid constructs. Descriptive statistics, confidence intervals, hypothesis tests, regression, segmentation, and text analysis provide complementary analytical tools, each with assumptions and limitations. Social listening expands the evidence base but is vulnerable to platform and participation bias. Customer insight emerges when findings are triangulated, interpreted in context, connected to action, and communicated with appropriate uncertainty. Ethical data practices, privacy protection, human oversight, and reproducible governance are essential to professional marketing analytics.

Key Terms

Target population: The complete group to which a study intends to generalise.

Sampling frame: The operational source used to identify or reach population units.

Probability sampling: Sampling in which each eligible unit has a known, non-zero selection probability.

Nonresponse bias: Systematic difference between respondents and eligible nonrespondents.

Qualitative research: Research focused on meanings, experiences, processes, and context.

Quantitative research: Research using numerical measurement and statistical analysis.

Construct validity: The degree to which a measure represents the intended theoretical concept.

Reliability: The consistency of a measurement procedure.

Likert-type item: An ordered response item measuring agreement or another evaluation.

Confidence interval: An interval procedure expressing sampling uncertainty around an estimate.

P-value: Probability, under a specified null model, of results at least as extreme as observed.

Effect size: A measure of the magnitude of a difference or relationship.

Regression: A model relating an outcome to one or more predictors.

Odds ratio: The multiplicative change in odds associated with a predictor change.

Segmentation: Dividing a market into groups with meaningful differences.

Social listening: Systematic monitoring and analysis of relevant online conversations.

Sentiment analysis: Computational or human classification of evaluative polarity in text.

Triangulation: Comparing multiple evidence sources or methods to strengthen interpretation.

Customer insight: An evidence-based explanation that supports a customer or marketing action.

Review Questions

5 Short Answer Questions

1. What is the difference between a research problem and a management decision?
2. Distinguish reliability from validity.
3. Why can a large convenience sample still produce biased results?
4. What does a p-value represent, and what does it not represent?
5. State two limitations of automated sentiment analysis.

5 Long Answer Questions

1. Compare depth interviews, focus groups, observation, and surveys for understanding customer journeys.
2. Explain how questionnaire wording, response scales, and pretesting affect measurement quality.
3. Describe the assumptions and managerial interpretation of a multiple linear regression model.
4. Explain how a researcher should design a mixed-method study for a declining customer-retention problem.
5. Discuss the ethical and governance requirements for using customer data and AI-assisted text analysis.

3 Analytical / Application Questions

1. A retailer finds that online satisfaction is high, but repeat purchase is falling. Design a triangulated research programme and explain how each method would contribute.
2. A survey reports that 70% of 400 randomly selected customers are satisfied. Calculate an approximate 95% confidence interval and identify at least three threats to interpreting it as a market-wide estimate.
3. An app company wants to claim that a new onboarding screen increases paid conversion. Propose an experiment, define primary and secondary outcomes, and explain why observational correlations would be insufficient.

Multiple Choice Questions with Answers

Question 1

Which research method is most appropriate for discovering unexpected language and motives among customers whose needs are poorly understood?

A. Depth interviews

B. A standardised satisfaction survey only

C. A financial audit

D. A fixed-price experiment

Correct Answer: A. Depth interviews

Question 2

A sampling frame is best defined as:

A. The complete theoretical population

B. The operational source used to reach eligible population units

C. The final statistical test

D. The questionnaire response scale

Correct Answer: B. The operational source used to reach eligible population units

Question 3

If 280 of 400 randomly selected customers report satisfaction, what is the sample proportion?

A. 0.28

B. 0.40

C. 0.70

D. 1.43

Correct Answer: C. 0.70

Question 4

Which statement about a p-value is correct?

A. It is the probability that the null hypothesis is true

B. It measures the practical importance of an effect

C. It is the probability, under the null model, of a result at least as extreme as observed

D. It proves that one variable causes another

Correct Answer: C. It is the probability, under the null model, of a result at least as extreme as observed

Question 5

What is a major risk of a double-barrelled questionnaire item?

A. It always increases reliability

B. It asks respondents to evaluate more than one idea at once

C. It guarantees representative sampling

D. It eliminates recall error

Correct Answer: B. It asks respondents to evaluate more than one idea at once

Question 6

In a logistic regression model, exp(βⱼ) is commonly interpreted as the:

A. Change in the arithmetic mean

B. Odds ratio for a one-unit predictor increase, holding other included predictors constant

C. Sample variance of the outcome

D. P-value of the entire model

Correct Answer: B. Odds ratio for a one-unit predictor increase, holding other included predictors constant

Question 7

Which statement best describes triangulation?

A. Deleting contradictory observations

B. Comparing multiple evidence sources or methods to strengthen interpretation

C. Using only the largest dataset

D. Replacing qualitative evidence with a p-value

Correct Answer: B. Comparing multiple evidence sources or methods to strengthen interpretation

Question 8

Why may social-listening data fail to represent all customers?

A. Online conversations are always experimentally controlled

B. Participants may be self-selected and highly vocal minorities may be overrepresented

C. Text cannot contain opinions

D. Public data has no sampling issues

Correct Answer: B. Participants may be self-selected and highly vocal minorities may be overrepresented

Question 9

For a 95% sample-size calculation for a proportion when the expected proportion is unknown, p = 0.50 is often used because it:

A. Minimises the required sample size

B. Produces the largest variance p(1 − p) and therefore a conservative sample size

C. Eliminates nonresponse bias

D. Makes the population normally distributed

Correct Answer: B. Produces the largest variance p(1 − p) and therefore a conservative sample size

Question 10

Which evidence would provide the strongest basis for claiming that a new onboarding screen caused higher conversion?

A. Anecdotal comments from one employee

B. A cross-sectional correlation between onboarding use and conversion

C. A properly randomised experiment with a defined conversion outcome

D. A larger collection of unverified online opinions

Correct Answer: C. A properly randomised experiment with a defined conversion outcome