3. Requirements Engineering and Product Discovery

Introduction and Context

Software systems succeed when they solve a real problem for identifiable users, operate within their organizational and technical context, and deliver sufficient value to justify their cost and risk. Requirements engineering is the disciplined process through which this understanding is developed, communicated, evaluated, documented, and maintained. Product discovery extends this work by investigating whether a proposed product or feature is desirable, feasible, usable, viable, and strategically valuable before substantial implementation begins.

Requirements are not merely a list of functions requested by a customer. They represent negotiated agreements about capabilities, qualities, constraints, assumptions, and outcomes. Poor requirements create ambiguity, encourage uncontrolled scope growth, and cause expensive rework. Conversely, carefully developed requirements help teams make architectural decisions, design meaningful user experiences, plan tests, estimate effort, and govern changes.

At postgraduate level, requirements engineering should be understood as a socio-technical activity. It involves analysis of stakeholders, organizational power, workflows, regulations, technologies, risks, and human behavior. A requirement may be technically precise but still fail if it addresses the wrong problem. Product discovery therefore complements specification with evidence: observation, experimentation, prototyping, analytics, and user feedback.

Learning Objectives

After studying this chapter, a learner should be able to:

1. Explain the purpose, scope, and activities of requirements engineering and product discovery.

2. Identify stakeholders, analyze their interests and influence, and manage conflicting expectations.

3. Apply suitable elicitation, modeling, specification, prioritization, validation, and change-management techniques.

4. Distinguish functional, non-functional, domain, business, user, and system requirements.

5. Develop testable requirements and maintain bidirectional traceability.

6. Evaluate product ideas through value, usability, feasibility, viability, and risk.

7. Use contemporary digital, analytical, and AI-supported techniques responsibly.

8. Recognize common failures, ethical concerns, governance obligations, and professional responsibilities.

The Nature of Requirements

A requirement is a documented or otherwise communicated need, capability, condition, or constraint that a system must satisfy. Requirements exist at several levels. Business requirements express organizational goals, such as reducing loan-processing time. Stakeholder requirements describe what a particular group needs, such as enabling a loan officer to review incomplete applications. System requirements state the behavior or quality expected from the software, such as automatically identifying missing documents within two seconds.

Functional requirements describe services, transformations, or responses provided by a system. For example, “The system shall allow an authorized analyst to export filtered transaction data in comma-separated value format.” Non-functional requirements describe qualities or constraints, including performance, security, accessibility, reliability, maintainability, legal compliance, and compatibility. A system that implements all required functions but exposes confidential data or becomes unusable under realistic load is not a successful system.

A useful distinction is between a requirement and a solution assumption. “Users need to compare alternative insurance plans” is a need; “the system shall provide a three-column comparison table” is one possible solution. Prematurely fixing the solution can prevent discovery of better alternatives. Requirements should therefore be sufficiently specific for agreement and verification, while leaving design freedom where design has not yet been decided.

Requirements also differ in certainty. Some are confirmed constraints; others are hypotheses requiring investigation. Treating every early statement as fact creates false confidence. Product discovery makes uncertainty explicit and ranks assumptions according to their potential impact.

Stakeholders, Context, and Product Value

A stakeholder is any individual, group, organization, or external entity that affects, is affected by, or has an interest in the system. Stakeholders may include end users, customers, sponsors, administrators, developers, operations staff, regulators, suppliers, auditors, accessibility advocates, and people indirectly affected by automated decisions. The person who commissions a system is not necessarily the person who uses it, and the most influential stakeholder is not always the most vulnerable one.

Stakeholder analysis normally begins with identification and proceeds to classification. A power–interest matrix distinguishes stakeholders who have high power and high interest from those with low formal authority but high operational knowledge. A salience perspective considers power, legitimacy, and urgency. Personas, role maps, customer-journey maps, and ecosystem maps help reveal goals, pain points, dependencies, and exclusions.

Context analysis examines the environment in which a system operates: business processes, physical conditions, organizational culture, policies, legacy systems, data flows, market forces, and legal requirements. Requirements that appear reasonable in isolation may be inappropriate in context. For example, a biometric authentication requirement may improve security but create accessibility, privacy, or inclusion concerns.

Product discovery asks whether the proposed product creates meaningful value. A product hypothesis can be expressed as: for a defined user group, a particular problem is sufficiently important; a proposed intervention will improve a measurable outcome; and the organization can deliver and sustain it. Value is multidimensional. It may include revenue, cost reduction, safety, compliance, public benefit, user satisfaction, learning, or strategic positioning. Product teams should distinguish output metrics, such as number of released features, from outcome metrics, such as reduced abandonment or improved task completion.

A Requirements Engineering Lifecycle

Requirements work is iterative rather than strictly linear. Nevertheless, a structured lifecycle provides discipline:

• Inception and discovery: establish the problem, objectives, scope, stakeholders, assumptions, and success measures.
• Elicitation: obtain information through conversations, observation, document analysis, workshops, experiments, and other techniques.
• Analysis and negotiation: organize, model, prioritize, resolve ambiguity, and negotiate conflicts.
• Specification: record requirements in suitable forms, such as stories, scenarios, models, or formal statements.
• Validation: determine whether requirements are correct, complete enough, consistent, feasible, unambiguous, and verifiable.
• Management: control baselines, versions, dependencies, priorities, decisions, and changes throughout the product lifecycle.

Agile development does not eliminate requirements engineering. It changes the granularity and timing of requirements. A product goal and roadmap may be relatively stable, while detailed backlog items evolve through short feedback cycles. In plan-driven environments, a baseline specification may be established earlier, but it must still be reviewed and controlled. Hybrid approaches combine architectural and regulatory planning with iterative discovery and delivery.

Elicitation: Learning What Matters

Elicitation is an investigative activity, not a transcription exercise. Stakeholders often describe solutions rather than underlying needs, omit tacit knowledge, or disagree because they optimize for different objectives. Effective analysts probe for goals, exceptions, business rules, risks, and evidence.

Interviews are useful for individual perspectives and sensitive subjects. Open questions encourage explanation, while focused questions clarify rules and exceptions. Observation and contextual inquiry reveal workarounds and informal practices that users may not mention. Workshops bring multiple stakeholders together to map processes, define terms, and negotiate shared understanding. Document and interface analysis is important when replacing or integrating legacy systems.

Prototypes and storyboards make abstract ideas discussable. Low-fidelity prototypes are valuable when the problem or interaction is uncertain; high-fidelity prototypes are appropriate when visual behavior or usability requires detailed evaluation. Surveys can gather broad opinions, but they are weaker for understanding complex workflows. Experiments and minimum viable products test risky assumptions with limited investment.

A strong elicitation session records not only requested features but also the evidence behind them. An analyst should ask: Who experiences the problem? How frequently? What is the current workaround? What happens when the process fails? How is success measured? Which groups may be harmed or excluded? What constraints are non-negotiable?

Modeling Requirements and User Experience

Models reduce complexity by presenting selected aspects of a system. Context diagrams show system boundaries and external actors. Process models, such as activity or business-process diagrams, represent flow and responsibility. Use-case models describe interactions between actors and a system. Domain models identify important concepts and relationships. State models are useful when behavior changes according to events and conditions.

A user story is a concise expression of user intent, commonly written as “As a [role], I want [capability], so that [benefit].” A story is not a complete specification; its acceptance criteria, assumptions, dependencies, and examples provide necessary detail. Scenarios describe behavior in a particular context, including normal and exceptional paths. Behavior-driven specifications often use a given–when–then form to connect examples with tests.

User experience requirements concern effectiveness, efficiency, learnability, accessibility, trust, satisfaction, and error recovery. They should not be reduced to visual preferences. For example, “The first-time user shall complete account setup without assistance in no more than five minutes in usability testing” is more evaluable than “The interface shall be intuitive.” Accessibility requires consideration of diverse abilities, devices, language, cognition, and assistive technologies from the beginning rather than as a final inspection.

Specification Quality and Traceability

A high-quality requirement is clear, necessary, feasible, consistent with other requirements, singular in purpose, prioritized, and verifiable. Avoid vague terms such as “fast,” “user-friendly,” “appropriate,” or “secure” unless they are operationally defined. A performance statement might specify that 95 percent of search requests shall return results within two seconds under a defined concurrent-user and data-volume condition.

A requirements specification may include scope, glossary, stakeholders, assumptions, functional requirements, quality attributes, interfaces, business rules, data requirements, constraints, acceptance criteria, and open issues. The appropriate level of formality depends on risk. Safety-critical and regulated systems usually require more structured documentation and evidence than a low-risk internal prototype.

Traceability links requirements to their sources, objectives, design elements, implementation items, tests, risks, and release decisions. A traceability matrix can expose untested requirements, redundant features, and change impacts. Bidirectional traceability is especially valuable: analysts can trace a test back to its requirement and trace a requirement forward to its validation evidence. Traceability should support reasoning, not become an administrative burden. Automated links in issue trackers, repositories, test systems, and modeling tools can reduce maintenance effort, but tool integration does not guarantee meaningful traceability.

Prioritization, Negotiation, and Product Decisions

Resources are limited, so requirements must be prioritized rather than simply accumulated. The MoSCoW method classifies requirements as Must have, Should have, Could have, or Won’t have for a particular timebox. Weighted scoring compares value, urgency, risk reduction, effort, strategic alignment, and dependency. The Kano model distinguishes basic expectations, performance attributes, and delighters. Cost of delay and economic prioritization emphasize the consequences of postponement.

Prioritization should be evidence-based and transparent. A feature requested by an influential stakeholder may have low user value, while a less visible accessibility improvement may be essential for legal compliance and equitable service. Dependencies and architectural risk must also be considered. A small feature may be strategically important if it validates a critical technical assumption.

Conflicts are normal. A security officer may require stronger authentication, while users seek rapid access; operations may demand stability, while marketing seeks frequent change. Negotiation should make interests, constraints, alternatives, and consequences visible. Decision records should identify the decision, options considered, rationale, participants, assumptions, and review conditions.

Validation and Verification

Validation asks whether the right problem and product have been specified; verification asks whether the specification has been expressed correctly and can be built and tested. Reviews, walkthroughs, inspections, prototypes, simulations, consistency checks, and acceptance-test design all contribute to both activities.

Validation examines questions such as:

• Does the requirement reflect a genuine stakeholder or business need?
• Is it within scope and consistent with policy?
• Can it be implemented with available technology and resources?
• Can an independent tester determine objectively whether it is satisfied?
• Does it address exceptions, misuse, accessibility, privacy, and operational conditions?

Requirements quality is not guaranteed by approval signatures. Stakeholders may approve a document they have not understood, and users may not recognize problems until they interact with a prototype. Continuous validation through demonstrations, usability evaluation, pilot deployment, operational monitoring, and outcome measurement is therefore necessary.

Managing Change and Uncertainty

Change arises from new regulations, market conditions, stakeholder learning, technical discoveries, defects, security threats, and organizational strategy. Change is not inherently a failure; uncontrolled and poorly understood change is the problem. A change-control process records the proposed change, rationale, urgency, affected requirements, dependencies, cost, risk, schedule impact, and decision authority.

In iterative delivery, a backlog provides a flexible change mechanism, but flexibility still requires governance. Product owners may reorder work, while significant changes to goals, architecture, compliance, or contractual scope may require formal review. Baselines establish a reference point against which changes can be assessed. Version control, decision logs, release criteria, and traceability provide historical accountability.

Uncertainty should be managed explicitly. Assumption logs, risk registers, spikes, prototypes, and experiments convert unknowns into learning activities. A requirement with high uncertainty and high impact deserves early investigation. Teams should avoid confusing stakeholder enthusiasm with evidence of product-market or organizational value.

Applied Case: A University Digital Assessment Platform

Background

The fictional Northbridge University plans a digital assessment platform for examinations, coursework submission, marking, feedback, and academic-integrity review. The proposed system will serve students, faculty, examination administrators, disability services, IT operations, and external moderators.

Situation and Problem

An initial proposal requests online examinations with webcam monitoring, automated plagiarism detection, real-time dashboards, and mobile access. Early meetings reveal conflicting priorities. Faculty want reliable marking and flexible feedback. Students want clear procedures and privacy. Disability services require accommodations. IT staff are concerned about network failure and support volume. Legal advisers identify obligations concerning personal data and retention.

Analysis

The discovery team maps the assessment journey from assessment design to result publication. Observation shows that the greatest current delay is not examination delivery but manual reconciliation of marks and late moderation. Interviews also reveal that some students have unreliable connectivity and that automated monitoring can produce false suspicion for students using shared or assistive environments.

The team separates outcomes from requested features. Important outcomes include reducing result-processing time, preserving assessment integrity, ensuring accessible participation, and improving the transparency of feedback. A risk register identifies privacy, service outage, algorithmic bias, and academic appeals as high-impact concerns.

Approach and Solution

The team creates personas and service blueprints, then prototypes two workflows: coursework submission and moderated marking. A pilot uses a limited set of low-stakes assessments rather than immediately deploying remote proctoring for high-stakes examinations. Requirements include an offline-resilient submission mechanism, explicit accessibility accommodations, role-based access, an audit trail for mark changes, defined retention periods, and human review of automated integrity alerts.

Prioritization places reliable submission, accessible participation, marking workflow, and auditability in the first release. Webcam monitoring is deferred pending evidence of necessity, proportionality, technical reliability, and legal acceptability. Acceptance criteria are linked to usability tests, load tests, security controls, and administrative procedures.

Outcome

The pilot reveals that clear submission confirmation and automatic reminders reduce support requests more effectively than several proposed dashboard features. Moderators value the audit trail, while students report greater trust when automated alerts are explained and subject to appeal. The university decides to expand the platform gradually and to review outcome measures after each release.

Learning Points

The case demonstrates that product discovery can challenge an attractive but weakly evidenced feature set. Stakeholder analysis must include vulnerable and indirectly affected groups. Non-functional requirements, governance, and human procedures are central to a successful product. Prototyping and pilot evidence can prevent premature investment, while traceability connects institutional objectives to system behavior and tests.

Risks, Limitations, and Professional Responsibilities

Common requirements failures include confusing requests with needs, interviewing only senior stakeholders, ignoring exceptions, writing compound or unverifiable statements, treating prototypes as finished products, underestimating non-functional requirements, and allowing the loudest stakeholder to determine priorities. Another failure is “requirements inflation,” in which every desirable idea enters the scope without evidence of value or capacity to deliver.

Requirements are also shaped by power. Users may agree to intrusive monitoring because they lack alternatives. Analysts should make consent, privacy, accessibility, fairness, safety, and appeal mechanisms visible. Data collected during discovery must be minimized, protected, and used for an explicit purpose. Automated recommendations or prioritization should be explainable enough for affected stakeholders and subject to human oversight where consequences are significant.

Professional responsibility includes accurate representation of uncertainty, honest communication of feasibility and risk, protection of confidential information, respect for intellectual property, and avoidance of discriminatory assumptions. Regulatory requirements should be treated as design constraints, not as paperwork added at the end. Governance should define who may approve requirements, who owns data, how decisions are audited, and how users can challenge harmful outcomes.

Digital Tools, Analytics, and AI-Supported Discovery

Requirements teams commonly use collaborative whiteboards, survey systems, issue trackers, modeling tools, repositories, test-management platforms, analytics dashboards, and product experimentation systems. Integration can provide a connected chain from objective to requirement, implementation, test, release, and outcome. However, a tool can record poor reasoning as efficiently as good reasoning.

Analytics can reveal funnel abandonment, task duration, error frequency, feature adoption, reliability, and differences among user groups. Quantitative evidence should be interpreted with qualitative investigation: a high abandonment rate may indicate confusing content, lack of trust, inaccessible interaction, or an inappropriate user need.

AI can support transcript summarization, clustering of feedback, duplicate detection, draft acceptance criteria, terminology analysis, and traceability suggestions. It should not be treated as an autonomous requirements authority. Generated summaries may omit minority perspectives, reproduce bias, or invent connections. Sensitive data must be handled according to organizational policy and applicable law. Human review, source verification, uncertainty labeling, and auditability are essential.

Current Directions and Best Practices

Contemporary practice increasingly combines continuous discovery with continuous delivery. Product teams test assumptions before committing engineering capacity and measure outcomes after release. Socio-technical systems thinking is gaining importance because software behavior is inseparable from organizational procedures, incentives, data, and human judgment. Privacy engineering, secure-by-design practice, responsible AI, accessibility, and sustainability are being incorporated earlier into requirements work.

Effective practice includes the following principles:

• Begin with outcomes, users, context, and evidence rather than a predetermined feature list.
• Involve diverse stakeholders, including people affected indirectly or adversely.
• Record assumptions, decisions, dependencies, and unresolved questions.
• Express quality attributes with measurable conditions and operational scenarios.
• Use prototypes and experiments to reduce high-impact uncertainty early.
• Prioritize transparently according to value, risk, effort, compliance, and equity.
• Maintain practical traceability from goals through tests and outcome measures.
• Validate continuously with users, technical specialists, and operational teams.
• Treat privacy, security, accessibility, and safety as integral requirements.
• Reassess requirements when evidence, context, or organizational objectives change.

Chapter Summary

Requirements engineering establishes a shared, testable understanding of what a software system must achieve and under what constraints. Product discovery extends this discipline by testing whether a proposed problem, user group, solution, and value proposition are sufficiently credible before large-scale delivery. The work includes stakeholder and context analysis, elicitation, modeling, specification, prioritization, negotiation, validation, traceability, and controlled change management.

Functional requirements describe system services, while non-functional requirements define qualities and constraints that often determine operational success. User experience, accessibility, privacy, security, governance, and ethical impact must be considered throughout the lifecycle. Agile and plan-driven methods use different rhythms and artifacts, but neither removes the need for disciplined requirements work. Prototypes, analytics, experiments, digital tools, and carefully governed AI can improve learning, provided that human judgment, evidence, inclusion, and accountability remain central.

Key Terms

Requirement: A need, capability, condition, or constraint that a system must satisfy.

Requirements engineering: The systematic elicitation, analysis, specification, validation, and management of requirements.

Product discovery: Investigation of user problems, value, feasibility, viability, usability, and risks before or during product development.

Stakeholder: A person, group, or organization that affects, is affected by, or has an interest in a system.

Functional requirement: A statement of behavior or service a system must provide.

Non-functional requirement: A quality, constraint, or condition governing the system or its operation.

Business requirement: An organizational objective or desired outcome motivating development.

Elicitation: The process of discovering requirements and relevant evidence from stakeholders and context.

Context analysis: Examination of the environment, processes, rules, systems, and conditions surrounding a product.

Acceptance criterion: An objectively assessable condition that must be met for a requirement or story to be accepted.

Traceability: The ability to link requirements to their sources, implementation, tests, risks, and outcomes.

Baseline: An agreed reference version of requirements or related artifacts controlled for change.

Prioritization: The systematic ordering of requirements according to value, risk, urgency, effort, or other criteria.

Prototype: A preliminary representation used to explore, communicate, or evaluate a product idea.

Assumption: A belief treated as true for planning or analysis but requiring evidence or validation.

Usability: The extent to which specified users achieve goals effectively, efficiently, and satisfactorily.

Product hypothesis: A testable proposition about users, problems, intervention, and expected outcomes.

Change control: The process of assessing, approving, recording, and implementing requirement changes.

Requirements validation: Evaluation of whether requirements represent the right needs and are feasible and verifiable.

Decision record: A documented account of a decision, alternatives, rationale, assumptions, and consequences.

Review Questions

Short Answer Questions

1. Distinguish a functional requirement from a non-functional requirement and provide one example of each.

2. What is the purpose of stakeholder analysis in requirements engineering?

3. Explain the difference between validation and verification of requirements.

4. Why are acceptance criteria important for user stories?

5. What does bidirectional traceability mean?

Long Answer Questions

1. Explain the requirements engineering lifecycle and discuss how its activities interact in an iterative development environment.

2. Describe major elicitation techniques and evaluate the situations in which interviews, observation, workshops, prototypes, and experiments are most useful.

3. Discuss how non-functional requirements should be specified, validated, and traced using measurable operational conditions.

4. Explain requirements prioritization and negotiation, including the influence of value, risk, effort, compliance, dependency, and stakeholder power.

5. Analyze the relationship between product discovery, user experience, accessibility, privacy, and responsible software development.

Analytical / Application Questions

1. A hospital wants an appointment system that is “fast, secure, and easy to use.” Convert these statements into measurable requirements and identify the evidence needed to validate them.

2. A government service team receives conflicting requests from citizens, administrators, security specialists, and policy officials. Design a stakeholder-analysis and prioritization approach that addresses power, inclusion, risk, and public value.

3. Using the university assessment-platform case, propose a traceability structure linking business outcomes, stakeholder requirements, system requirements, acceptance tests, risks, and post-release measures.

22. Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes product discovery from conventional feature collection?

A. Product discovery focuses only on documenting stakeholder requests.

B. Product discovery tests whether a problem, solution, and expected value are credible.

C. Product discovery begins only after implementation is complete.

D. Product discovery replaces all forms of requirements specification.

Correct Answer: B. Product discovery tests whether a problem, solution, and expected value are credible.

Question 2

Which is the strongest example of a measurable non-functional requirement?

A. The system shall be user-friendly.

B. The system shall provide modern functionality.

C. The system shall return 95 percent of searches within two seconds under defined load.

D. The system shall satisfy users.

Correct Answer: C. The system shall return 95 percent of searches within two seconds under defined load.

Question 3

What is the principal benefit of bidirectional requirements traceability?

A. It eliminates the need for stakeholder participation.

B. It links requirements both to their origins and to implementation and verification evidence.

C. It guarantees that requirements will never change.

D. It allows teams to avoid acceptance testing.

Correct Answer: B. It links requirements both to their origins and to implementation and verification evidence.

Question 4

During discovery, observation reveals that employees use an undocumented spreadsheet workaround. What does this most directly demonstrate?

A. The official process model is always complete.

B. Observation can reveal tacit practices omitted during interviews.

C. Workarounds should automatically become system requirements.

D. Stakeholder requirements are unnecessary.

Correct Answer: B. Observation can reveal tacit practices omitted during interviews.

Question 5

A team postpones webcam monitoring because its necessity, proportionality, privacy impact, and reliability are uncertain. Which principle is being applied?

A. Prioritize high-risk assumptions for investigation before commitment.

B. Give the most visible feature the highest priority.

C. Treat all stakeholder requests as mandatory.

D. Avoid validating non-functional requirements.

Correct Answer: A. Prioritize high-risk assumptions for investigation before commitment.

Question 6

Which artifact most directly records the rationale, alternatives, assumptions, and consequences of an important requirements decision?

A. Decision record

B. Context diagram

C. Persona

D. Defect log

Correct Answer: A. Decision record

Question 7

Why is “the interface shall be intuitive” a weak requirement?

A. It is too detailed for implementation.

B. It cannot be objectively interpreted or verified without operational criteria.

C. It describes a non-functional concern.

D. It refers to a user experience issue.

Correct Answer: B. It cannot be objectively interpreted or verified without operational criteria.

Question 8

In the university assessment case, why was human review of automated integrity alerts required?

A. Automated systems cannot process any assessment data.

B. Human review supports fairness, explanation, and appeal when automated judgments may be wrong.

C. Human review removes the need for privacy controls.

D. Automated alerts are always legally prohibited.

Correct Answer: B. Human review supports fairness, explanation, and appeal when automated judgments may be wrong.

Question 9

Which prioritization decision best reflects responsible requirements engineering?

A. Implement the request from the most senior stakeholder regardless of evidence.

B. Prioritize features solely by development convenience.

C. Consider value, risk, compliance, equity, dependencies, and effort transparently.

D. Include every requested feature in the first release.

Correct Answer: C. Consider value, risk, compliance, equity, dependencies, and effort transparently.

Question 10

What is the most appropriate role of AI in requirements work?

A. Autonomously approve requirements and resolve stakeholder conflicts.

B. Replace user research with generated summaries.

C. Assist with tasks such as clustering feedback while requiring human verification and governance.

D. Generate requirements without source data or uncertainty labels.

Correct Answer: C. Assist with tasks such as clustering feedback while requiring human verification and governance.