Market Intelligence and Customer Insight

Introduction

Product positioning and launch strategy begin before a product is designed, priced, or promoted. They begin with an evidence-based understanding of the market: which customers experience a meaningful problem, how they currently solve it, what prevents them from adopting alternatives, and why a proposed product might be preferable. Market intelligence and customer insight provide this foundation.

Market intelligence is the systematic collection, analysis, and interpretation of information about customers, competitors, categories, channels, regulation, technology, and broader environmental forces. Customer insight is a validated explanation of customer behaviour, motivation, and unmet need that can guide a product or business decision. Data becomes insight only when it is connected to a decision and interpreted within a relevant context.

This chapter examines market research, voice-of-customer methods, Jobs-to-be-Done, personas, journey mapping, social listening, and market sizing. It also considers research quality, ethical responsibilities, artificial intelligence, and the translation of evidence into positioning and launch decisions.

Learning Objectives

By the end of this chapter, the learner should be able to:

1. Distinguish market data, market intelligence, and customer insight.
2. Design a mixed-method research programme for product decisions.
3. Apply voice-of-customer and Jobs-to-be-Done methods to identify customer problems.
4. Construct useful personas and customer journey maps.
5. Use social listening critically and ethically.
6. Estimate market size using TAM, SAM, and SOM and explain the assumptions behind each estimate.
7. Identify demand drivers, adoption barriers, and evidence gaps.
8. Convert research findings into positioning and launch implications.

1. From Market Data to Strategic Insight

Market research can be divided into four related activities:

• Descriptive research determines what is happening: market volume, customer characteristics, usage frequency, or brand awareness.
• Exploratory research investigates why something may be happening, often through interviews, observation, or open-ended analysis.
• Explanatory research tests relationships, such as whether perceived risk reduces purchase intention.
• Evaluative research assesses a concept, prototype, message, or launch activity.

Primary research is collected for the present decision, whereas secondary research uses existing sources such as government statistics, industry reports, financial filings, academic studies, reviews, and internal records. Neither is automatically superior. Secondary research is often efficient for category context, while primary research is usually needed to understand a specific problem or proposition.

A useful hierarchy is:

An insight should be more than a customer quotation. It should explain a meaningful behaviour, identify its context, and suggest an action that can be tested.

2. A Research Architecture for Product Decisions

Research should begin with a decision, not with a favourite method. A product team might need to decide whether to enter a segment, which problem to solve, how to differentiate, or whether a launch message is credible. Each decision requires different evidence.

A practical research sequence is:

1. Frame the decision. State the decision, alternatives, time horizon, and consequences of being wrong.
2. Form research questions. Convert the decision into questions about customers, competitors, behaviour, and constraints.
3. Audit existing evidence. Review analytics, support tickets, sales records, reviews, prior studies, and market reports.
4. Explore behaviour and language. Conduct interviews, observation, diary studies, or contextual inquiry.
5. Measure prevalence and prioritise. Use surveys, behavioural data, experiments, or market estimates.
6. Test the proposed response. Evaluate a concept, prototype, price, message, or channel.
7. Triangulate. Compare evidence from different methods and investigate contradictions.
8. Document uncertainty. Record assumptions, sample limitations, confidence, and unresolved questions.

A mixed-method design is especially useful. Qualitative research can reveal unknown motivations and customer vocabulary; quantitative research can estimate prevalence and compare segments. Behavioural data shows what people do, although it may not reveal why. The strongest decisions generally arise when these forms of evidence converge.

Research quality

Reliability concerns whether a method produces consistent results. Validity concerns whether it measures what it claims to measure. A large sample does not compensate for a biased question, an irrelevant population, or poor operationalisation. Common threats include leading questions, recall error, social desirability bias, survivorship bias, non-response bias, and convenience sampling.

For example, asking “How valuable would this innovative service be?” may create demand through its wording. A better approach is to ask about recent behaviour: “Tell me about the last time you encountered this problem. What did you do next?”

3. Voice of the Customer

Voice of the Customer (VoC) is a structured approach to collecting and interpreting customer statements, behaviours, expectations, complaints, and desired outcomes. Sources include interviews, usability sessions, call-centre records, reviews, support tickets, sales conversations, community forums, surveys, and product telemetry.

A disciplined VoC process has five stages:

1. Collect: gather evidence across relevant customer groups and touchpoints.
2. Clean: remove duplicates, identify source and date, and preserve original context.
3. Code: classify statements by need, situation, outcome, obstacle, emotion, or feature request.
4. Prioritise: assess importance, frequency, dissatisfaction, strategic fit, and ability to serve.
5. Translate: convert customer language into product requirements, positioning hypotheses, or research questions.

Customers often express solutions rather than needs. “I need a dashboard” may indicate a deeper need for visibility, control, accountability, or rapid exception detection. Product managers should record the request faithfully but investigate the underlying outcome before committing to a feature.

Interview practice

A strong interview is neutral, specific, and anchored in actual events. Useful prompts include:

• “Walk me through the last time this happened.”
• “What were you trying to accomplish?”
• “What alternatives did you consider?”
• “What made the process difficult?”
• “What happened after you chose that solution?”
• “What would have caused you not to proceed?”

Interviews are not sales demonstrations. The interviewer should avoid defending the concept, rewarding positive answers, or treating hypothetical enthusiasm as proof of demand. Behavioural evidence—time, money, workarounds, switching, and prior attempts—is generally stronger than abstract statements of interest.

4. Jobs-to-be-Done and Demand Drivers

Jobs-to-be-Done (JTBD) views a product as something customers “hire” to make progress in a particular situation. A job is not merely a demographic category or a product feature. It combines a context, desired progress, and criteria for success.

A job statement can be expressed as:

When [situation], I want to [motivation or progress], so that [desired outcome].

For example: “When preparing a client presentation under time pressure, I want to consolidate reliable evidence quickly, so that I can communicate a defensible recommendation.”

JTBD analysis commonly distinguishes:

• Functional jobs: practical tasks to accomplish.
• Emotional jobs: how the customer wants to feel.
• Social jobs: how the customer wants to be perceived.
• Related jobs: supporting activities surrounding the primary job.

The job is different from a need statement such as “customers need collaboration software.” The latter names a product category; the former identifies progress and context.

Demand drivers and adoption barriers

Demand drivers are forces that increase the likelihood, urgency, or value of adoption. They may include rising costs, regulation, time pressure, performance gaps, social influence, changing technology, or a new use case. Adoption barriers may include price, switching cost, risk, lack of trust, complexity, incompatible systems, procurement rules, habit, inadequate skills, or weak perceived value.

A useful analysis connects each driver or barrier to evidence:

5. Segmentation and Personas

Segmentation divides a broad market into groups with meaningful differences in needs, behaviour, value, or response to a marketing action. Useful bases include:

• Descriptive: age, location, firm size, industry, income.
• Behavioural: frequency, use case, channel, loyalty, adoption stage.
• Needs-based: desired outcomes, pain points, risk tolerance.
• Value-based: potential revenue, cost to serve, lifetime value.
• Contextual: situation, trigger, environment, or job being performed.

A segment is strategically useful when it is identifiable, sufficiently substantial, reachable, differentiated, and compatible with the organisation’s capabilities. Demographic groups alone are often weak predictors of product response. A needs-based segment can be more actionable—for example, “teams that require audit-ready reporting under regulatory deadlines.”

A persona is a research-based representation of a meaningful customer archetype. It should include evidence, context, goals, behaviours, constraints, triggers, alternatives, objections, and decision criteria. It should not become a fictional biography filled with unsupported details.

Personas are most useful when they answer: what job is being done, what creates urgency, what evidence builds trust, what blocks adoption, and how should the product and message respond? They should be updated when new evidence contradicts them.

6. Customer Journey Mapping

A customer journey map represents the sequence of stages and interactions through which a customer moves before, during, and after a product experience. Typical stages include awareness, consideration, evaluation, purchase, onboarding, use, support, renewal, and advocacy. The correct stages depend on the product and decision process.

Each stage can document:

• customer goal and job;
• actions and channels;
• questions and emotions;
• friction and failure points;
• organisation touchpoints;
• backstage processes;
• opportunities and measurable outcomes.

Journey mapping should be evidence-led. A visually attractive map based on assumptions can mislead a team. The map should distinguish observed behaviour from interpretation and should represent different journeys where segments or use cases differ.

Figure: An evidence-led customer journey map linking stages, friction, and opportunities.

The most valuable journey maps expose “moments of truth”: points where confidence, effort, or perceived value changes sharply. These moments often provide stronger positioning and launch opportunities than a general list of customer preferences.

7. Social Listening and Digital Behaviour

Social listening involves systematically monitoring public conversations, reviews, communities, search behaviour, and digital signals to identify themes, sentiment, language, emerging needs, and competitor perceptions. It differs from social monitoring, which usually focuses on responding to mentions of a particular brand or account.

A responsible social-listening workflow includes:

1. Define the decision and relevant keywords, synonyms, languages, and exclusions.
2. Select sources appropriate to the target audience and category.
3. Remove spam, duplicates, bots, and irrelevant uses of terms.
4. Code topics, use cases, emotions, complaints, desired outcomes, and competitor references.
5. Compare themes across time, segments, and events.
6. Validate important findings through direct research or behavioural data.

Sentiment analysis should not be treated as objective truth. Sarcasm, cultural context, mixed emotions, coordinated campaigns, and unequal participation can produce misleading classifications. Public online discussion is also not a representative sample of all customers. Social listening is especially useful for discovering language and emerging issues; it is less reliable as a sole basis for estimating market prevalence.

8. Market Sizing

Market sizing helps determine whether a problem is commercially significant and whether a launch ambition is realistic. Three common concepts are:

• TAM (Total Addressable Market): total theoretical demand if the relevant market were fully served.
• SAM (Serviceable Available Market): portion of TAM that fits the product’s geography, category, customer eligibility, and capabilities.
• SOM (Serviceable Obtainable Market): realistic share that the organisation can capture within a defined period, given competition and execution constraints.

A top-down estimate begins with an external market value and applies defensible filters. A bottom-up estimate starts with customers, usage, and price. Bottom-up estimates are often more transparent for a specific product, but they are sensitive to assumptions.

A basic revenue estimate is:

where N is the number of potential customers or units, f is annual purchase frequency or quantity per customer, and p is price per purchase. Units must be consistent: if N is customers, f must be purchases per customer per year and p must be revenue per purchase.

If the product serves only a proportion s of the population and the expected obtainable share is q, then an illustrative estimate is:

Here, s is the eligible/serviceable proportion and q is the realistically obtainable share. This is not a forecasting law; it is a scenario model. The assumptions must be stated and tested.

Worked example

Suppose a business identifies 120,000 eligible organisations. Research suggests that 40% have the relevant workflow, each would purchase 2 licences per year, the annual price per licence is $300, and a realistic first-period share is 3%.

The serviceable annual revenue opportunity is:

The obtainable first-period revenue scenario is:

The estimate assumes all eligible organisations have equal potential, the frequency and price are stable, and the 3% share is achievable. In practice, the team should model ranges—for example, conservative, base, and optimistic cases—and identify which assumptions most affect the result.

Market size is not the same as opportunity quality. A large market may have low urgency, high acquisition cost, weak access, or strong incumbents. A smaller market with severe pain, high willingness to pay, and concentrated distribution may be more attractive.

9. From Insight to Positioning and Launch Decisions

Research becomes strategically useful when it changes a decision. A synthesis should connect four elements:

1. Target customer: who experiences the problem most intensely?
2. Situation or job: when does the problem arise?
3. Value proposition: what progress or outcome does the product enable?
4. Reason to believe: what evidence supports the claim?

A positioning hypothesis can be written as:

For [target customer] who [situation or need], [product] is a [category or frame of reference] that [primary benefit], because [reason to believe]. Unlike [alternative], it [meaningful distinction].

This is a hypothesis, not a slogan. It should be tested for relevance, distinctiveness, credibility, and clarity. Launch implications may include which segment to prioritise, which problem to lead with, what proof to provide, what objection to address, and which channel reaches customers at the triggering moment.

A research-to-decision matrix is useful:

10. Applied Case: Launching a Shared Inventory Platform

Background

The following is an illustrative case. StockLink, a proposed platform for independent retailers, would allow stores to share inventory visibility and transfer products between locations. The team initially planned to position it as “a modern inventory dashboard.”

Problem

Early conversations produced enthusiastic feature requests but little evidence of willingness to pay. The team needed to identify the highest-value job, understand adoption barriers, estimate the opportunity, and develop a credible launch position.

Analysis

The team reviewed support records from existing inventory tools, interviewed twelve store managers, observed replenishment routines in six stores, surveyed 240 retailers, and analysed public reviews of competing systems. The evidence showed that the most costly problem was not lack of dashboards. It was lost sales when staff could not quickly locate stock across nearby stores. Managers used phone calls and spreadsheets as workarounds. The main barriers were inaccurate data, employee training, and fear that transfers would create disputes about ownership.

The core job was framed as: “When a customer wants an item that is unavailable in this store, help me locate and secure available stock quickly, so that I can complete the sale without damaging trust.” The primary segment was multi-location retailers with sufficient inventory fragmentation and frequent stock-outs—not all independent retailers.

Approach and solution

The team changed the positioning from a generic dashboard to a store-to-store availability and transfer service. The minimum launch proposition included near-real-time stock confidence indicators, a simple transfer workflow, audit trails, and onboarding support. The launch plan used a pilot with retailers that already had compatible point-of-sale systems. Demonstrations focused on a real stock-out scenario rather than a feature tour.

The team estimated 8,000 potentially eligible retailers, with 30% having multiple locations and relevant inventory complexity. At an average annual contract value of $1,200, the serviceable revenue scenario was:

A first-year obtainable share of 5% implied:

These figures were treated as scenarios, not facts. The team planned to update them after pilot conversion and retention data.

Outcome

The pilot produced stronger engagement when the message emphasised completed sales and reliable stock confidence. Retailers rejected the product when data accuracy was unclear, so the launch included a data-quality diagnostic and explicit confidence labels. The team also discovered that store managers and head-office buyers valued different outcomes; therefore, the product required separate messages for operational and economic stakeholders.

Learning points

The case demonstrates that customer language may begin with features while the economically important job concerns an outcome. It also shows why a market estimate must be tied to eligibility, evidence, and realistic capture assumptions. Finally, adoption barriers should shape the product and launch plan, not merely appear as risks in a presentation.

11. Ethics, Governance, and the Role of AI

Customer research involves personal information, opinions, recordings, behavioural traces, and sometimes sensitive data. Researchers should obtain informed consent where required, collect only necessary information, protect access, define retention periods, and avoid exposing identifiable quotations without permission. Regulatory obligations vary by jurisdiction, but privacy, fairness, transparency, and security are universal managerial responsibilities.

Research should not manipulate vulnerable participants, disguise sales activity as research, or use confidential customer information for unrelated purposes. Incentives should compensate participation without coercing it. When research involves employees, minors, health information, or financial data, stronger safeguards may be required.

AI can accelerate transcription, clustering, translation, survey coding, review summarisation, and hypothesis generation. However, AI-generated themes may reproduce sampling bias, misinterpret irony, erase minority views, or present uncertain inferences as facts. Teams should retain source traceability, review outputs with human researchers, protect confidential data, and label synthetic or inferred content. AI should support judgement, not replace direct engagement or accountability for decisions.

12. Best Practices and Common Failure Modes

Best practices include:

• Start with a decision and explicit research questions.
• Combine behavioural, qualitative, and quantitative evidence.
• Recruit participants who actually experience the target situation.
• Ask about recent events rather than hypothetical preference alone.
• Separate customer problem from proposed solution.
• Treat personas and journey maps as hypotheses grounded in evidence.
• Quantify market size with transparent assumptions and ranges.
• Record contradictions instead of forcing consensus.
• Link each insight to a product, positioning, or launch action.
• Revisit insights after launch using usage, retention, conversion, and support data.

Common failure modes include confirmation bias, over-reliance on enthusiastic interviewees, confusing social volume with demand, using average market statistics for a narrow segment, mistaking feature requests for jobs, and presenting an inflated TAM as a sales forecast. A professional product manager makes uncertainty visible and designs the next test accordingly.

Chapter Summary

Market intelligence combines information about customers, competitors, category conditions, channels, and the external environment. Customer insight explains behaviour in a way that supports a decision. High-quality product research begins with a decision, uses appropriate mixed methods, and triangulates evidence.

Voice-of-customer methods collect and structure customer language, behaviour, and expectations. Jobs-to-be-Done identifies the progress customers seek in a specific situation, while segmentation and personas help focus strategy on meaningful groups. Journey mapping reveals friction and moments of truth across the customer experience. Social listening helps discover language and emerging themes but should not be assumed to represent the whole market.

TAM, SAM, and SOM distinguish theoretical, serviceable, and realistically obtainable opportunity. Market-sizing formulas are scenario tools whose value depends on transparent assumptions. The final purpose of research is action: selecting a target, defining a job and value proposition, addressing adoption barriers, supplying credible proof, and designing a launch around customer triggers.

Key Terms

Market intelligence: Systematic analysis of market, customer, competitor, and environmental information.

Customer insight: An evidence-based explanation of customer behaviour that supports a decision.

Primary research: Information collected directly for a current research purpose.

Secondary research: Existing information originally collected for another purpose.

Voice of the Customer: A structured process for collecting and interpreting customer needs, language, and feedback.

Jobs-to-be-Done: A perspective that examines the progress a customer seeks in a particular situation.

Demand driver: A factor that increases urgency, value, or likelihood of adoption.

Adoption barrier: A factor that prevents or delays product acceptance or use.

Segmentation: Division of a market into groups with meaningful differences.

Persona: An evidence-based archetype representing a strategically relevant customer group.

Customer journey map: A representation of customer stages, actions, experiences, and organisational touchpoints.

Social listening: Analysis of public digital conversations and signals to identify themes and perceptions.

TAM: Total theoretical addressable market.

SAM: Portion of the total market that the product can serve.

SOM: Realistic market portion obtainable within a defined period.

Triangulation: Comparison of evidence from different sources or methods.

Validity: The extent to which research measures the intended concept.

Reliability: The consistency of a measurement or research procedure.

Positioning hypothesis: A testable statement about target customer, category, benefit, distinction, and proof.

Review Questions

Short Answer Questions

1. Distinguish market data from customer insight.
2. What is the difference between exploratory and evaluative research?
3. Define a Job-to-be-Done and give one example.
4. What are the three components of TAM, SAM, and SOM analysis?
5. Why should social listening not be treated as a representative sample automatically?

Long Answer Questions

1. Explain how a product team can design a mixed-method research programme from a strategic decision.
2. Discuss the strengths and limitations of voice-of-customer interviews.
3. Explain how personas and journey maps can influence positioning and launch planning.
4. Compare top-down and bottom-up market-sizing approaches, including their assumptions.
5. Discuss ethical and governance responsibilities when using customer data and AI in research.

Analytical / Application Questions

1. A team receives many requests for a mobile dashboard. Design an interview and analysis approach to determine the underlying customer job before prioritising the feature.
2. Using the market-sizing formula in this chapter, estimate obtainable annual revenue for 50,000 eligible customers, a 25% serviceable proportion, 1.5 purchases per year, a price of $400, and a realistic share of 4%. State the assumptions and limitations.
3. Construct a journey-map analysis for a subscription product with high trial sign-ups but low conversion. Identify three possible barriers, the evidence needed to investigate them, and a corresponding launch intervention.

Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes customer insight from raw market data?

A. Insight is always qualitative, while data is always quantitative.

B. Insight interprets evidence to explain behaviour and guide a decision.

C. Insight is any positive customer quotation.

D. Insight is a larger dataset than market data.

Correct Answer: B. Insight interprets evidence to explain behaviour and guide a decision.

Question 2

Which interview question is most likely to elicit behavioural evidence?

A. Would you use this product in the future?

B. How innovative does this concept seem?

C. Tell me about the last time you encountered this problem.

D. Would a dashboard solve your problem?

Correct Answer: C. Tell me about the last time you encountered this problem.

Question 3

In Jobs-to-be-Done analysis, a job primarily describes:

A. A product feature selected by a customer.

B. The progress a customer seeks in a particular situation.

C. A demographic description of a market.

D. A company’s internal delivery process.

Correct Answer: B. The progress a customer seeks in a particular situation.

Question 4

Which is an adoption barrier?

A. A regulatory deadline increasing urgency

B. A measurable improvement in performance

C. A switching cost associated with migration

D. A customer trigger creating demand

Correct Answer: C. A switching cost associated with migration

Question 5

What is the main purpose of a customer journey map?

A. To replace all quantitative research

B. To show customer stages, friction, touchpoints, and opportunities

C. To calculate total addressable market

D. To create fictional customer biographies

Correct Answer: B. To show customer stages, friction, touchpoints, and opportunities

Question 6

Which statement about social listening is most accurate?

A. Online discussion is always representative of the market.

B. Sentiment classification is free from cultural and contextual error.

C. Social listening can reveal language and themes but should be validated.

D. Social listening eliminates the need for customer interviews.

Correct Answer: C. Social listening can reveal language and themes but should be validated.

Question 7

A market has 10,000 eligible customers, a 30% serviceable proportion, one purchase per year, and a price of $500. What is the serviceable annual revenue scenario?

A. $150,000

B. $500,000

C. $1,500,000

D. $5,000,000

Correct Answer: A. $150,000

Question 8

In market sizing, SOM refers to:

A. The entire theoretical market

B. The portion compatible with product capabilities

C. The realistic portion obtainable within a defined period

D. The number of surveyed respondents

Correct Answer: C. The realistic portion obtainable within a defined period

Question 9

Which practice most improves research validity?

A. Using leading questions to obtain decisive answers

B. Recruiting only current brand advocates

C. Anchoring interviews in recent real experiences

D. Ignoring contradictory evidence

Correct Answer: C. Anchoring interviews in recent real experiences

Question 10

A feature request for a dashboard may conceal a deeper need for:

A. A larger colour palette only

B. Visibility, control, or rapid exception detection

C. More demographic segmentation

D. A higher total addressable market

Correct Answer: B. Visibility, control, or rapid exception detection