Product Discovery, Customer Research and Requirements

Introduction and Context

Product management begins before a product is designed or built. It begins with understanding a problem worth solving, the people who experience it, the circumstances in which it occurs, and the value that a viable solution could create. Product discovery is the disciplined process of reducing uncertainty about customers, problems, opportunities, solutions and business viability before substantial delivery investment is made.

In information technology projects, weak discovery is a major source of failure. Teams may deliver technically elegant systems that customers do not use, solve symptoms rather than causes, or optimize a low-value feature while ignoring a critical workflow. Requirements gathered only from senior stakeholders can also omit the needs of operational users, administrators, regulators and customers with accessibility requirements.

This chapter connects customer research with requirements engineering and product delivery. It examines design thinking, market research, personas, journey mapping, jobs-to-be-done, prototyping, experimentation, user stories, acceptance criteria and backlog prioritization. At postgraduate level, the central concern is not simply learning a set of tools, but understanding when each method is appropriate, how evidence should be evaluated, and how product decisions can be governed under uncertainty.

Learning Objectives

By the end of this chapter, students should be able to:

1. Explain the purpose, scope and limitations of product discovery.
2. Select and conduct appropriate qualitative and quantitative customer research.
3. Construct evidence-based personas, journey maps and jobs-to-be-done statements.
4. Convert customer insight into outcomes, requirements, user stories and acceptance criteria.
5. Design prototypes and experiments that test important assumptions economically.
6. Prioritize and manage a product backlog using transparent decision criteria.
7. Evaluate ethical, accessibility, privacy and governance issues in discovery.
8. Critically assess the relationship between discovery, delivery and product strategy.

1. Product Discovery as Uncertainty Reduction

A product idea contains several forms of uncertainty:

• Desirability: Do identifiable users experience a meaningful problem and want a solution?
• Usability: Can users understand and successfully use the proposed solution?
• Feasibility: Can the organization build, integrate, secure and operate it?
• Viability: Can it support sustainable financial, strategic or public-value outcomes?
• Compliance and responsibility: Is it lawful, safe, accessible and ethically acceptable?

Discovery does not eliminate uncertainty. Rather, it makes assumptions explicit and tests the assumptions that matter most. A useful discovery question is therefore not “What features should we build?” but “What must be true for this product to create value, and what is the cheapest credible way to learn whether it is true?”

Discovery and delivery are related but distinct. Discovery investigates the problem and possible responses; delivery implements and operates a chosen response. In iterative product organizations they form a continuous loop: evidence from usage informs new discovery, while discovery informs delivery priorities.

A product opportunity can be represented as a chain:

Figure: Product discovery links context and evidence to tested solutions and measurable value.

2. Design Thinking and Problem Framing

Design thinking is a human-centred approach commonly described through empathize, define, ideate, prototype and test. These labels should not be treated as a rigid sequence. Teams often return to research after testing a prototype, or redefine a problem when new evidence contradicts an initial assumption.

Empathize and observe

The purpose of empathy work is not to imitate users or merely collect opinions. It is to understand goals, practices, constraints, emotions, workarounds and social context. Observation is particularly valuable where users cannot easily articulate tacit practices. For example, a warehouse employee may say that inventory reconciliation is “simple,” while observation reveals repeated spreadsheet copying, informal codes and interruptions.

Define the opportunity

A strong problem statement identifies a population, situation, unmet need and consequence without prematurely prescribing a solution. A useful form is:

[User group] needs a way to [achieve outcome] when [context], because [evidence-based consequence].

“Users need a mobile app” is a solution statement. “Field technicians need to confirm the correct replacement part while working without reliable connectivity, because incorrect parts cause repeat visits” is a problem statement.

Ideate and converge

Ideation generates alternatives, including process changes, policy changes, service changes and technical solutions. Convergence requires explicit criteria such as user value, risk reduction, strategic fit, cost, accessibility and operational feasibility. Brainstorming alone is not evidence of desirability.

3. Customer and Market Research

Customer research investigates people, behaviour, needs and decision contexts. Market research additionally examines market structure, competitors, substitutes, regulation, purchasing processes and economic conditions. Both are important, but they answer different questions.

Qualitative research

Qualitative research is appropriate when the problem is poorly understood, concepts are ambiguous or the team needs explanations rather than prevalence estimates. Interview questions should focus on recent behaviour: “Tell me about the last time you…” is generally more informative than “Would you use…?” The researcher should probe sequence, decisions, workarounds, people involved and consequences.

A practical interview guide contains an introduction, consent statement, context questions, critical-incident questions, probes and closure. Leading questions, premature pitching and excessive jargon reduce validity. Interviews are not sales demonstrations.

Sampling should be purposeful. Include relevant variations in role, expertise, frequency of use, geography, accessibility needs and organizational context. A small study can be rigorous when participants are selected for information value and analysis is systematic; a large convenience sample can still be misleading.

Quantitative and market research

Surveys and product analytics help estimate scale and compare segments. Analysts must distinguish a sample statistic from a population parameter and report the sampling method. A survey of voluntary respondents cannot automatically represent all customers.

For a proportion estimate, an approximate 95% confidence interval under a simple random-sampling model is:

Here, p̂ is the observed sample proportion, n is the sample size, and 1.96 is the approximate standard-normal critical value for 95% confidence. The approximation assumes independent observations and is less reliable for very small samples or proportions close to 0 or 1. A confidence interval describes uncertainty under the sampling model; it does not correct coverage bias or a poorly designed question.

For example, if 120 of 200 respondents report a problem, p̂ = 0.60. The approximate standard error is √(0.60 × 0.40 / 200) ≈ 0.0346. The interval is approximately 0.60 ± 1.96 × 0.0346, or 0.532 to 0.668. This means the estimated proportion is 60%, with sampling uncertainty under the stated assumptions—not that 95% of individual users fall in that range.

Research synthesis

Synthesis converts raw data into patterns and implications. Teams may code interview notes, cluster observations, compare segments and record supporting quotations or incidents. A research repository should preserve source, date, participant characteristics, method and confidence. Separate:

• Observation: what was seen or reported.
• Interpretation: what the team believes it means.
• Implication: a decision or question resulting from it.

This separation reduces the risk that an attractive hypothesis will be mistaken for evidence.

4. Personas, Segmentation and Journey Mapping

A persona is a research-based archetype representing a meaningful user group. It normally includes goals, behaviours, context, capabilities, frustrations, motivations and relevant environmental constraints. It should not be a fictional biography created from stereotypes. A persona is useful when it helps a team make a decision; it is harmful when treated as a permanent description of every person in a segment.

Personas should distinguish roles. The buyer, administrator, daily user, approver and beneficiary may be different people. In a university learning platform, for example, a student uses the system, a lecturer creates content, an administrator configures it and a procurement department selects it.

Segmentation can use needs, behaviour, context, organization size, risk profile or value potential. Demographic categories should be used only when relevant and ethically justified. Segmentation by average behaviour can hide people with disabilities or low digital access; inclusive discovery deliberately seeks variation.

A journey map represents stages in an experience and the user’s actions, questions, emotions, pain points, channels and backstage dependencies. It should include the complete service rather than only the interface. A digital appointment system, for example, may involve eligibility checking, booking, reminders, arrival, staff processing and follow-up.

Journey maps support opportunity identification. A painful stage is not automatically a product opportunity: the cause may be a policy, staffing issue, data dependency or third-party process. Teams should therefore map service dependencies and ownership, not merely add interface features.

5. Jobs-to-be-Done and Outcome Thinking

Jobs-to-be-done (JTBD) focuses on the progress a person is trying to make in a situation. A job is not identical to a feature or demographic category. A customer may “hire” different products to accomplish the same job, and one product may serve several jobs.

A job statement often follows this structure:

When [situation], I want to [motivation or progress], so I can [desired outcome].

For a commuter: “When a train is delayed while I am travelling to work, I want to know credible alternatives quickly, so I can arrive with minimal disruption.” The job exposes functional, emotional and social dimensions. JTBD complements personas: a persona describes a recurring user perspective, whereas a job emphasizes the circumstance and desired progress.

Outcome statements should be testable. “Make reporting better” is vague. “Reduce the time required for a supervisor to identify unresolved incidents” is more actionable. Outcome-driven thinking discourages teams from equating delivery of features with creation of value.

6. Prototyping and Experimentation

A prototype is an external representation of a proposed experience or system. Its fidelity should match the question being tested. Paper sketches can test information structure; clickable wireframes can test navigation; a service role-play can test operational coordination; a technical spike can test integration feasibility.

A useful experiment specifies:

1. The assumption or hypothesis.
2. The target population and context.
3. The prototype or intervention.
4. The behavioural or attitudinal measure.
5. A threshold for learning or action.
6. The decision that will follow.

A hypothesis might be: “For existing small-business users, a guided invoice workflow will enable first-time completion without staff assistance.” The test should define “enable,” such as task completion, error rate, time, assistance requests or comprehension. “Users liked it” is weaker than observable evidence.

A minimum viable product (MVP) is not simply the smallest feature set. It is the smallest coherent product or experiment capable of testing a value proposition with real users and producing meaningful learning. An MVP that is too incomplete may measure confusion caused by poor execution rather than the underlying proposition.

A/B testing can compare alternatives when traffic, randomization and a measurable outcome are available. It is inappropriate when exposure is too small, ethical risk is high, or the change affects a complex service whose effects take a long time to appear. Product teams should avoid optimizing a proxy metric that conflicts with user welfare, such as maximizing clicks while increasing complaints.

7. From Insight to Requirements

A requirement states a needed capability, condition or constraint. Requirements engineering involves elicitation, analysis, specification, validation, prioritization, management and traceability. Discovery evidence does not automatically become a requirement; it must be interpreted, negotiated and checked.

Requirement types

• Business or product requirements: desired outcomes and strategic goals.
• User requirements: capabilities needed by a user or stakeholder.
• Functional requirements: behaviours or services the system must provide.
• Non-functional requirements: qualities or constraints, such as performance, security, availability, accessibility, auditability and maintainability.
• Regulatory requirements: obligations arising from law, standards, contracts or institutional policy.
• Transition requirements: data migration, training, rollout and operational change needs.

Non-functional requirements are frequently under-specified. “The system must be fast” is not verifiable. A stronger requirement identifies a measurable condition, population and context, such as “For 95% of authenticated search requests under the defined normal load, the service shall return results within two seconds.” The exact threshold must come from user expectations, technical analysis, risk and business commitments rather than being invented for appearance.

Requirements should be clear, necessary, feasible, unambiguous, testable, traceable and prioritized. Conflicts should be made visible. For example, stronger identity verification may improve fraud resistance but increase completion time and exclusion risk.

User stories

A common user-story form is:

As a [role], I want [capability], so that [value or outcome].

Example: “As a clinic receptionist, I want to see a patient’s preferred communication channel, so that I can send appointment information appropriately.” The format encourages discussion but is not a complete specification. A story should be accompanied by context, dependencies, design notes and acceptance criteria.

The INVEST mnemonic is often used as a heuristic: Independent, Negotiable, Valuable, Estimable, Small and Testable. It is not an absolute law. Technical dependencies and architectural work can make complete independence impossible; the aim is to expose dependencies and preserve value-oriented planning.

Acceptance criteria

Acceptance criteria define conditions under which a story is accepted. They should describe observable behaviour and important exceptions. Given–When–Then syntax is useful:

• Given a registered patient has a preferred email address,
• When the receptionist selects “send appointment details,”
• Then the system sends the approved template to that address and records the event in the audit log.

Criteria should include validation, permissions, failure conditions, accessibility and relevant non-functional constraints. They are not a substitute for exploratory testing or professional judgement. The definition of done may additionally require code review, automated tests, security review, documentation and deployment readiness.

8. Prioritization and Backlog Management

A product backlog is an ordered, evolving set of work items. Ordering is not merely ranking features by stakeholder seniority. It should reflect value, urgency, risk, learning potential, dependencies, compliance and effort.

A simple scoring approach is weighted scoring. For item i:

Here, S_i is the item’s score; V_i, R_i, L_i and E_i are normalized ratings for value, risk reduction, learning value and effort; and w_v, w_r, w_l and w_e are explicit weights. Ratings should use a documented scale, such as 0–10, and the weights should reflect strategy. This is a decision aid, not an objective truth. Scores are not meaningful if the scales are inconsistent or evidence is weak.

Cost of delay divides the economic or strategic impact of waiting by duration or effort. It can be useful for time-sensitive work, but monetary estimates may be uncertain and should not conceal ethical or regulatory obligations. Mandatory security or legal work should not be demoted simply because its estimated direct revenue is low.

A balanced backlog may include:

• customer-facing value;
• technical enablers and architecture;
• defects and reliability work;
• discovery experiments;
• security, privacy and compliance obligations;
• accessibility and inclusive-design improvements;
• operational and data-quality work.

Backlog refinement clarifies items, splits oversized stories, identifies dependencies, updates estimates and removes obsolete work. A backlog is not a contract that prevents learning. New evidence should change priorities.

9. Traceability and Governance

Traceability connects strategic outcomes to opportunities, research evidence, requirements, design decisions, implementation, tests and measures. It enables impact analysis when a requirement changes and helps auditors determine why a high-risk feature was released.

A lightweight traceability record may include:

Governance should define decision rights. Product managers may own prioritization, but security, legal, risk, architecture, data protection and affected business units require appropriate involvement. Governance that occurs only at the end of delivery is expensive and encourages late rejection.

10. Ethics, Privacy and Inclusion

Customer research involves power relationships and personal information. Participants should understand the purpose, voluntary nature, recording practices, retention period and intended use of research. Sensitive data should be minimized, protected and de-identified where possible. Consent is not a universal solution: organizations must also assess fairness, necessity, proportionality and potential harm.

Researchers should avoid recruiting only the easiest or most enthusiastic participants. Excluding people with disabilities, limited connectivity, language differences or low digital confidence can produce a product that works for a privileged minority. Accessibility should be investigated during discovery, not added as cosmetic compliance at the end.

Algorithmic and AI-enabled products require additional scrutiny. Teams should identify who may be harmed by errors, whether training data represents relevant populations, how decisions can be explained, and whether human review is required. Synthetic or automatically summarized research can accelerate analysis, but it can also erase minority views, introduce fabricated details or expose confidential information. Human researchers remain accountable for interpretation.

11. Contemporary Practice: Product Analytics and AI-Assisted Discovery

Modern product organizations combine qualitative evidence with event analytics, session recordings, experimentation platforms and feedback systems. Instrumentation should be designed around meaningful user outcomes rather than collecting every possible event. Metrics need clear definitions, ownership, data-quality checks and privacy controls.

AI tools can assist with interview transcription, clustering, draft story generation, anomaly detection and prototype content. They should be treated as assistants, not authoritative researchers. Teams must verify summaries against source material, protect confidential data, identify model bias and preserve the participant’s meaning. Automated prioritization can amplify historical usage patterns and undervalue new or underserved groups. Human review and an auditable rationale remain essential.

12. Illustrative Case: Campus Equipment Booking Service

Background

A university operates laboratories across several buildings. Students and staff currently reserve specialist equipment through email and separate spreadsheets. Double bookings are common, and technicians spend considerable time resolving conflicts.

Problem and discovery

The initial proposal is “build a booking app.” A discovery team interviews students, lecturers, technicians and laboratory managers, observes booking and handover processes, reviews incident records and maps the journey from identifying equipment to returning it.

The research reveals three distinct jobs:

• Students need confidence that equipment will be available when completing assessed work.
• Technicians need to protect equipment capacity and record maintenance restrictions.
• Lecturers need to coordinate class-wide allocations and enforce training requirements.

The major problem is not simply calendar entry. Availability depends on training status, maintenance windows, loan duration, location and handover capacity. A persona based on a frequent student user would not represent the technician’s administrative and safety responsibilities.

Experiment and requirements

The team prototypes a searchable availability view and tests it with representative users using realistic scenarios. It discovers that users interpret “available” differently: some mean physically unused, while others mean bookable by a person with the required training. The design therefore separates physical status from eligibility and explains the reason for restrictions.

An outcome-oriented requirement is: “Authorized users shall be able to identify equipment they are eligible to book for a specified time period.” Acceptance criteria include eligibility rules, maintenance blocks, timezone handling, conflict prevention, keyboard navigation and an audit record for changes.

A small pilot covers one laboratory and a limited equipment category. The backlog prioritizes conflict prevention, eligibility rules and auditability before visual customization. Analytics measure attempted double bookings, successful booking completion and technician intervention. Interviews continue during the pilot to explain unexpected behaviour.

Outcome and learning points

The pilot reduces spreadsheet reconciliation and exposes a previously hidden policy issue: some equipment has training that expires. The product team adds expiry notifications and a technician override process with reason capture. The case demonstrates that discovery changes not only interface design but also service rules, data requirements, governance and operational roles.

The main learning is that a feature request often represents a larger service problem. Evidence-based discovery prevents the team from automating an incomplete process.

Challenges and Limitations

Discovery is vulnerable to confirmation bias, stakeholder politics, survivorship bias, over-reliance on articulate participants and confusing stated preference with actual behaviour. Research can also become performative: teams conduct interviews but do not allow findings to influence scope.

There are practical trade-offs. A rigorous study may require time that a deadline does not allow. A quick experiment may produce directional learning but cannot establish general prevalence. Quantitative data may be abundant but poorly instrumented. Teams should state confidence, limitations and unresolved questions rather than presenting every conclusion as certain.

Another difficulty is organizational: discovery may reveal that the best intervention is a process change or that a proposed product is not viable. Mature governance allows teams to stop, pivot or narrow scope without treating these decisions as failure. The cost of learning is usually lower before large-scale delivery, but discovery itself must remain proportionate to risk and decision value.

Best-Practice Checklist

• Begin with outcomes, users and context rather than a predetermined feature.
• Make assumptions explicit and rank them by consequence and uncertainty.
• Use multiple evidence sources where feasible.
• Recruit participants who represent important variation and vulnerability.
• Distinguish observations, interpretations and decisions.
• Prototype the riskiest assumption at the lowest useful fidelity.
• Write requirements that are testable and include non-functional concerns.
• Use stories to support conversation, not to hide missing detail.
• Define acceptance criteria before implementation is considered complete.
• Order the backlog transparently and revisit it when evidence changes.
• Preserve traceability from evidence to outcome.
• Integrate privacy, accessibility, security and regulatory review early.
• Measure outcomes after release; delivery completion is not value realization.

Chapter Summary

Product discovery is a continuing, evidence-informed process for reducing uncertainty about desirability, usability, feasibility, viability and responsible operation. Design thinking provides a human-centred orientation, while research methods supply evidence about behaviour, needs and context. Interviews and observation reveal meaning and workflow; surveys, analytics and market analysis help estimate scale and compare alternatives.

Personas represent researched user groups, journey maps reveal experiences and service dependencies, and jobs-to-be-done focus attention on progress sought in a situation. Prototypes and experiments make assumptions testable before costly delivery. Insights must then be translated into outcome statements, functional and non-functional requirements, user stories and observable acceptance criteria.

Backlog ordering should consider value, risk, learning, effort, urgency, dependencies and obligations. Scoring models assist judgement but do not replace it. Traceability, ethical research, privacy protection, accessibility and responsible use of AI are integral to professional product management. The strongest product teams treat discovery and delivery as a learning loop and remain willing to change or stop work when evidence warrants it.

Key Terms

Product discovery: Structured learning about users, problems, solutions and value before and during delivery.

Desirability: The extent to which a solution addresses a meaningful user need.

Feasibility: The extent to which a solution can be built and operated with available constraints.

Viability: The extent to which a solution can create sustainable organizational or societal value.

Design thinking: A human-centred approach involving problem understanding, ideation, prototyping and testing.

Contextual inquiry: Research conducted while observing users in their real setting.

Persona: A research-based archetype representing a meaningful user group.

Journey map: A visual or structured representation of stages, actions, emotions and dependencies in an experience.

Jobs-to-be-done: A perspective that examines the progress a person seeks in a particular situation.

Prototype: An early representation used to learn about a proposed product or service.

Experiment: A planned test of an explicit assumption with defined evidence and a decision rule.

Minimum viable product: The smallest coherent product or experiment capable of generating meaningful validated learning.

Requirement: A needed capability, condition, quality or constraint.

Non-functional requirement: A quality or constraint such as security, performance or accessibility.

User story: A concise statement of a user capability and its intended value.

Acceptance criteria: Observable conditions used to determine whether a requirement is satisfied.

Product backlog: An ordered, evolving set of product work items.

Traceability: The ability to connect outcomes, evidence, requirements, implementation and tests.

Cost of delay: The impact of postponing an item, considered in relation to time or effort.

Outcome: A change in user, organizational or societal condition produced by an intervention.

Review Questions

Short Answer Questions

1. What are the five major forms of uncertainty addressed by product discovery?
2. How does a persona differ from a demographic profile?
3. Why should an experiment specify a decision threshold?
4. What is the purpose of acceptance criteria?
5. State two limitations of survey research.

Long Answer Questions

1. Explain how design thinking and requirements engineering complement one another in an IT product lifecycle.
2. Compare interviews, contextual inquiry, surveys and analytics as customer-research methods. Include appropriate use cases and limitations.
3. Discuss how personas, journey maps and jobs-to-be-done can be combined without confusing their purposes.
4. Explain the difference between functional and non-functional requirements, using examples from a digital service.
5. Critically evaluate weighted scoring as a product-backlog prioritization technique.

Analytical / Application Questions

1. A bank proposes a mobile feature allowing customers to freeze a payment card. Design a discovery plan that identifies key assumptions, participant groups, research methods, prototype form and success evidence.
2. Convert the following problem into an outcome statement, user story and three acceptance criteria: field engineers often arrive without the correct replacement part because inventory information is fragmented.
3. A product experiment shows increased clicks but also increased customer complaints. Analyze why the metric may be misleading and recommend a revised measurement and decision approach.

Multiple Choice Questions with Answers

Question 1

Which statement best describes product discovery?

A. Writing detailed technical specifications before speaking to users

B. Reducing uncertainty about problems, users, solutions and value through evidence

C. Delivering the maximum number of features in a release

D. Replacing project governance with informal brainstorming

Correct Answer: B. Reducing uncertainty about problems, users, solutions and value through evidence

Question 2

Which research method is generally most appropriate for revealing tacit practices in a workplace?

A. Contextual inquiry

B. A purely demographic survey

C. A financial forecast

D. A feature voting contest

Correct Answer: A. Contextual inquiry

Question 3

A job-to-be-done primarily focuses on:

A. The programming language used by a product

B. The organizational hierarchy of the buyer

C. The progress a person seeks in a particular situation

D. The number of screens in an interface

Correct Answer: C. The progress a person seeks in a particular situation

Question 4

Which is the strongest requirement?

A. The system should be user-friendly

B. The application should be modern

C. The system must be fast enough

D. For normal load, 95% of authenticated searches shall return results within two seconds

Correct Answer: D. For normal load, 95% of authenticated searches shall return results within two seconds

Question 5

What is the main purpose of acceptance criteria?

A. To define observable conditions for accepting a work item

B. To identify the product team's job titles

C. To replace all exploratory testing

D. To guarantee that a feature will be profitable

Correct Answer: A. To define observable conditions for accepting a work item

Question 6

In an experiment, why is a decision threshold useful?

A. It makes every result statistically significant

B. It specifies what evidence will lead to continuing, changing or stopping an approach

C. It eliminates the need to recruit participants

D. It ensures that users prefer the prototype

Correct Answer: B. It specifies what evidence will lead to continuing, changing or stopping an approach

Question 7

Which item should normally be included in a balanced product backlog?

A. Only customer-facing visual features

B. Only tasks requested by the most senior stakeholder

C. Security, accessibility, technical-enablement and operational work as well as customer value

D. Only work that can be completed in one day

Correct Answer: C. Security, accessibility, technical-enablement and operational work as well as customer value

Question 8

If 120 of 200 respondents report a problem, what is the observed sample proportion?

A. 0.20

B. 0.40

C. 0.60

D. 1.67

Correct Answer: C. 0.60

Question 9

Which practice best supports ethical customer research?

A. Collecting all available personal data in case it becomes useful

B. Recruiting only participants who agree with the proposed solution

C. Explaining purpose and data use, minimizing collection and protecting participants

D. Using automated summaries without checking source material

Correct Answer: C. Explaining purpose and data use, minimizing collection and protecting participants

Question 10

What does traceability connect in requirements governance?

A. Only source-code files and server locations

B. Outcomes, evidence, requirements, implementation, tests and measures

C. Employee salaries and office locations

D. Marketing slogans and brand colours

Correct Answer: B. Outcomes, evidence, requirements, implementation, tests and measures