Problem Discovery and Requirements Engineering

Introduction

Successful web applications begin before interface design or programming. They begin with an accurate understanding of a problem, the people affected by it, the outcomes they need, and the constraints within which a solution must operate. Problem discovery investigates whether a meaningful problem exists and why it matters. Requirements engineering converts that understanding into an agreed, testable description of what a system must do and how well it must do it.

For full-stack developers, these activities are not merely project-management tasks. They directly influence database design, API contracts, user-interface behaviour, security controls, testing strategy, deployment choices, and project cost. A poorly understood requirement can produce technically elegant software that solves the wrong problem.

This chapter presents a practical requirements-engineering process for diploma-level web application projects. It covers stakeholder analysis, elicitation, problem statements, personas, user stories, use cases, acceptance criteria, functional and non-functional requirements, scope, minimum viable products (MVPs), and prioritized product backlogs.

Learning objectives

By the end of this chapter, you should be able to:

• identify and analyse stakeholders and their interests;
• plan and conduct suitable requirements-elicitation activities;
• distinguish problems, needs, requirements, constraints, and solutions;
• write a precise problem statement and useful personas;
• create user stories, use cases, and testable acceptance criteria;
• classify functional and non-functional requirements;
• define scope boundaries and an evidence-based MVP;
• prioritize and manage a product backlog; and
• recognize ethical, accessibility, privacy, and governance implications.

1. From a vague idea to an engineering problem

A project request often starts as a solution statement: “Build an online booking system.” Such a statement names a product but does not establish the problem, users, success conditions, or boundaries. Requirements work should first ask:

1. Who experiences difficulty?
2. What are they trying to accomplish?
3. What prevents them from accomplishing it effectively?
4. What evidence shows that the difficulty is important?
5. What outcomes would demonstrate improvement?

A problem is an undesirable situation or unmet need. A requirement is a documented capability, condition, or constraint that a system must satisfy. A feature is a product capability that may help satisfy one or more requirements. These terms must not be treated as synonyms. For example, “customers cannot see current appointment availability” is a problem; “the system shall display available appointment slots” is a functional requirement; “calendar view” is one possible feature.

A useful discovery sequence is:

Context → Stakeholders → Evidence → Problem definition → Desired outcomes
       → Requirements → Scope and MVP → Backlog → Validation

Discovery is iterative. New evidence may invalidate an assumption, reveal an overlooked stakeholder, or show that a proposed feature is unnecessary.

Problem statements

A strong problem statement describes the user, context, difficulty, consequence, and evidence without prematurely prescribing a solution. One practical structure is:

[User group] needs to [achieve a goal] in [context], but [barrier] causes [impact]. Evidence indicates that [evidence or measurable consequence].

Example:

Independent tutors need to coordinate lesson bookings across messaging applications and spreadsheets, but inconsistent updates cause double bookings and missed reminders. Interviews with six tutors and an inspection of three weeks of booking records indicate repeated scheduling corrections and avoidable cancellations.

This is stronger than “Tutors need a modern booking app” because it identifies the users, failure, impact, and evidence.

A problem statement should be tested against five quality criteria: it is user-centred, specific, evidence-based, solution-neutral, and significant enough to justify development.

2. Stakeholder analysis

A stakeholder is any person, group, or organization that affects, is affected by, uses, funds, governs, supports, or may be harmed by the system. Stakeholders include obvious users but also administrators, support staff, data-protection officers, payment providers, accessibility users, and organizational decision-makers.

Stakeholder identification

Begin with broad categories:

Stakeholder analysis should not assume that the person paying for software is the person using it. Nor should the most vocal stakeholder automatically receive the highest priority.

A stakeholder register records the information needed to plan engagement:

A power–interest matrix is useful. High-power, high-interest stakeholders require close collaboration. High-power, low-interest stakeholders should receive concise updates and escalation information. Low-power, high-interest stakeholders should be consulted because they often provide detailed operational knowledge. Low-power, low-interest stakeholders can usually be monitored.

Stakeholder analysis also exposes conflicts. A business owner may want minimal data entry, while an administrator may require detailed records. A customer may want instant cancellation, while a provider needs protection from late cancellations. Such conflicts are requirements decisions, not merely communication problems; they must be made visible and resolved through policy, evidence, and trade-off discussion.

3. Requirements elicitation

Elicitation is the systematic discovery of requirements from people, documents, existing systems, observations, data, and experiments. It is not simply asking users what screens they want. Users may describe familiar workarounds rather than underlying needs, and different stakeholders may use the same term differently.

Common elicitation techniques

Interviews provide depth and allow follow-up questions. Use open questions initially: “Tell me about the last time this process failed.” Follow with probes such as “What happened next?” and “How did you decide what to do?” Avoid leading questions such as “Would a mobile app solve this?”

Observation and contextual inquiry reveal actual behaviour, interruptions, workarounds, and environmental constraints. A user may claim to complete a task in two steps but actually consult three systems and a paper record.

Workshops bring stakeholders together to map a process, define terminology, identify rules, and negotiate priorities. A facilitator should separate facts, assumptions, decisions, and unresolved questions.

Document and system analysis examines forms, policies, spreadsheets, support tickets, database structures, logs, and existing APIs. Documents may reveal legal obligations or business rules that users forget to mention.

Surveys efficiently collect responses from many people, but they are less effective for discovering unfamiliar problems. Questions should be unambiguous and answerable by the target population.

Prototyping uses sketches, wireframes, or clickable screens to obtain feedback on a proposed interaction. A prototype is an elicitation aid, not evidence that the complete system has been specified.

Analysis of usage data can reveal abandonment, delays, error rates, or popular workflows. Quantitative evidence should be interpreted with context; a low-use feature may be unhelpful, poorly promoted, or inaccessible.

Elicitation discipline

Before an interview, define its purpose, participant, questions, recording method, and consent process. During the session, distinguish:

• fact: “The receptionist enters each booking in two calendars”;
• need: “The receptionist needs one reliable availability record”;
• assumption: “Most customers own smartphones”;
• preference: “The calendar should be blue”; and
• requirement candidate: “The system shall prevent two confirmed bookings for the same resource and time.”

After elicitation, summarize findings and ask participants to correct misunderstandings. This validation step reduces errors caused by selective notes or ambiguous language.

4. Personas and user journeys

A persona is a research-based archetype representing a significant user group. It is not a fictional biography created from imagination. A useful persona includes goals, behaviours, context, capabilities, frustrations, accessibility considerations, and relevant technical environment.

Example:

Maya — independent tutor

• Goal: keep availability accurate while teaching and travelling.
• Context: uses a phone between lessons and a laptop at home.
• Frustration: receives booking requests through several channels.
• Need: quick updates, clear conflict warnings, and reliable reminders.
• Constraint: limited time and moderate technical confidence.

Personas prevent the team from designing for an abstract “average user.” They should be supported by evidence and revised when research changes.

A user journey describes stages such as awareness, search, selection, booking, attendance, and follow-up. At each stage, record user actions, touchpoints, emotions or difficulties, system opportunities, and evidence. Journey mapping helps reveal requirements outside the central transaction—for example, reminder delivery, cancellation handling, or support after a failed payment.

5. User stories and use cases

A user story is a concise statement of user value, commonly expressed as:

As a [role], I want [capability], so that [benefit].

Example:

As a student, I want to see a tutor’s current available slots, so that I can book a suitable lesson without exchanging messages.

The story is a conversation starter, not a complete specification. The widely used INVEST heuristic says a good story is Independent, Negotiable, Valuable, Estimable, Small, and Testable. Independence is desirable but not always absolute; technical dependencies should be made visible rather than hidden.

Stories should be decomposed when they contain several outcomes or roles. “As a user, I want registration, search, booking, payment, and reviews” is too large. It may become several stories, each with its own acceptance criteria.

A use case describes interactions between an actor and a system to achieve a goal. It is particularly useful for complex workflows and exceptions.

User stories are compact and backlog-friendly; use cases provide richer interaction and exception detail. They complement rather than replace one another.

6. Acceptance criteria and requirements quality

Acceptance criteria define conditions that must be true for a story to be accepted. They should be observable and testable. A common format is Given–When–Then:

• Given a student is authenticated and a slot is available,
• When the student confirms the booking,
• Then the system creates one booking, marks the slot unavailable, and displays a confirmation reference.

Additional criteria should address invalid input, duplicate requests, authorization, failure, and accessibility where relevant. “The page should be user-friendly” is not a useful criterion because it cannot be judged consistently. A better requirement might specify that a keyboard user can complete booking and that validation errors identify the affected field.

A quality requirement is generally:

• correct and necessary;
• unambiguous;
• complete within its stated boundary;
• consistent with other requirements;
• feasible within constraints;
• verifiable by inspection, test, analysis, or demonstration;
• traceable to a stakeholder need; and
• prioritized.

Functional requirements

Functional requirements describe behaviour, services, rules, inputs, outputs, and state changes. Examples include:

• The system shall allow an authenticated tutor to publish available time slots.
• The system shall prevent a student from booking a slot that is already confirmed.
• The system shall send a confirmation notification after successful booking.
• The system shall allow an administrator to export bookings for a selected date range.

Use “shall” for a mandatory requirement and avoid vague terms such as “fast,” “easy,” or “appropriate” unless they are operationally defined.

Non-functional requirements

Non-functional requirements describe quality attributes or constraints. They are still testable requirements, not optional decoration.

Targets must identify measurement method, conditions, and scope. “The system shall be secure” is a goal; “all administrative routes shall require role-based authorization and shall be covered by authorization tests” is more actionable. Quality attributes may conflict: stronger security can add interaction steps, and lower infrastructure cost may reduce availability. Trade-offs should be recorded.

7. Scope, assumptions, constraints, and MVP

Scope defines what the project includes. Scope boundaries also define what it deliberately excludes. A scope statement should identify users, processes, systems, geographic or organizational limits, and delivery period.

Example boundary:

Version 1 supports lesson discovery and booking for one tutoring centre, with email confirmation and administrator cancellation. It excludes online payment, multi-centre operation, video teaching, and automated tutor matching.

An assumption is accepted as true for planning but may require validation. A constraint is a limitation such as budget, deadline, legal rule, available technology, or required integration. An out-of-scope item is intentionally excluded, not forgotten.

The minimum viable product (MVP) is the smallest coherent product that delivers meaningful value to a defined user group and produces learning or evidence. It is not merely the smallest number of features, nor is it a poor-quality prototype. Essential quality, security, privacy, and accessibility requirements remain part of the MVP.

For a booking application, an MVP may include account access, availability display, booking confirmation, cancellation rules, and an administrator view. Online payments, recommendation algorithms, multiple languages, and advanced analytics may be later increments—unless evidence shows they are essential to the core outcome.

A useful MVP decision asks:

1. Which user and problem are primary?
2. What is the smallest end-to-end workflow that creates value?
3. What risks must be tested early?
4. What legal, safety, security, and accessibility capabilities are non-negotiable?
5. What evidence will determine the next release?

8. Prioritizing and managing the product backlog

A product backlog is an ordered, evolving list of product work. It may contain user stories, defects, technical work, research tasks, and quality requirements. Each item should have a clear outcome, acceptance criteria, priority rationale, and sufficient detail for the next development activity.

Prioritization is not the same as ranking personal preferences. Common approaches include:

• MoSCoW: Must, Should, Could, and Won’t for the current release;
• value versus effort: compare expected user or business value with development cost;
• risk-first ordering: investigate uncertain or dangerous assumptions early; and
• cost of delay: consider the harm of postponing an item.

A simple weighted scoring model can support discussion. If an item receives scores for user value V, risk reduction R, and strategic fit S, each on a 1–5 scale, and the team assigns weights w_V, w_R, and w_S that sum to 1, then:

The score is a decision aid, not an objective truth. The variables are dimensionless ordinal ratings; therefore arithmetic should not be presented as a precise measurement of value. The method is applicable when criteria and weights are agreed, documented, and applied consistently. Legal, safety, and security requirements may be mandatory regardless of score.

Example with weights 0.5, 0.3, and 0.2:

For the first item, 0.5×5 + 0.3×5 + 0.2×5 = 5.0. For the second, 0.5×2 + 0.3×1 + 0.2×2 = 1.8. The calculation supports the obvious conclusion that booking integrity should precede visual customization, but stakeholder discussion remains necessary.

Backlog refinement clarifies items, splits oversized work, identifies dependencies, estimates effort, and confirms acceptance criteria. Backlog order should change as evidence, risk, and strategy change. A frozen backlog is usually a sign that learning has stopped, not that requirements are perfect.

9. Traceability and change control

Traceability links a business objective to stakeholder evidence, requirements, backlog items, implementation, and tests. A small traceability matrix might look like this:

Traceability helps determine the impact of a change. If a cancellation policy changes, the team can locate affected stories, database rules, interface messages, tests, and documentation. Change control should record the requested change, rationale, impact, decision authority, and implementation status. Agile development welcomes change, but change still has cost and risk.

10. Illustrative case study: LearnLink tutoring bookings

The following is a fictional illustrative case.

Background

LearnLink is a small tutoring centre using telephone calls, messaging applications, and a shared spreadsheet to arrange lessons. Students frequently wait for confirmation, while tutors occasionally accept the same time from different channels.

Problem and analysis

Interviews with the owner, two administrators, five tutors, and eight students identify inconsistent availability, manual reconciliation, and missed reminders. Observation shows that administrators copy information between messages and the spreadsheet. The owner initially requests “a mobile app,” but discovery indicates that a responsive web application is sufficient for the first release.

Stakeholders include students, tutors, administrators, the owner, an email provider, and people requiring accessible interaction. Key rules include one confirmed booking per tutor and time slot, cancellation no later than 24 hours before a lesson unless an administrator overrides the rule, and restricted access to personal contact information.

Requirements and MVP approach

The MVP includes:

1. student and tutor authentication;
2. tutor availability management;
3. search and display of available slots;
4. booking confirmation with conflict prevention;
5. cancellation according to the stated rule;
6. email confirmation and reminder; and
7. administrator review of bookings and audit events.

It excludes payments, video meetings, tutor ratings, automated matching, and multi-centre support. These exclusions protect the delivery boundary while leaving room for later learning.

A representative story is:

As a student, I want to book an available lesson slot so that I can secure a lesson without waiting for manual confirmation.

Acceptance criteria include: an authenticated student can select an available slot; the system shows tutor, date, time, and cancellation terms; confirmation creates exactly one booking; a simultaneous competing request cannot create a second booking; and the student receives a confirmation message. The system must also present errors clearly and preserve privacy.

Technical implications

The requirements imply a data model containing users, roles, tutor availability, bookings, cancellation events, and notifications. The conflict rule must be enforced transactionally rather than only by checking the interface. Authorization must be applied on the server, because hiding a button does not prevent a malicious request. Notification failure should be recorded and retried or surfaced to administrators rather than silently losing the booking.

Outcome and learning points

A pilot with one centre measures booking completion time, conflict incidents, reminder delivery, support requests, and accessibility problems. The important outcome is not the number of screens delivered but whether scheduling becomes more reliable and less burdensome. The case demonstrates that discovery changes the solution, explicit scope protects the MVP, acceptance criteria guide testing, and operational rules must influence both requirements and architecture.

11. Ethical, privacy, accessibility, and governance considerations

Requirements engineering distributes benefits and burdens. Teams should ask who may be excluded, misrepresented, surveilled, or exposed to harm. Collect only personal data necessary for the stated purpose. Define access permissions, retention periods, correction procedures, breach-response responsibilities, and deletion or archival rules. Privacy should be considered at discovery, not added after database design.

Accessibility is a requirements concern. A responsive layout alone does not guarantee access. Keyboard operation, semantic structure, readable contrast, text alternatives, clear errors, captions where relevant, and compatibility with assistive technologies should be considered in acceptance criteria and testing.

Consent must be meaningful when research participants are interviewed or observed. Explain purpose, recording, storage, withdrawal, and use of quotations. Do not collect sensitive information merely because it might be useful later.

AI-assisted elicitation and coding can summarize interviews, classify feedback, or suggest stories, but generated content may omit minority voices, reproduce bias, or invent facts. Human review, source traceability, privacy protection, and explicit validation are required. AI output is a draft, not stakeholder approval.

12. Common failure patterns and best practices

Common failures include beginning with a preferred technology, interviewing only managers, writing solution-heavy requirements, ignoring exceptions, treating non-functional requirements as optional, overbuilding the MVP, and failing to validate interpretations.

Best practice is to maintain an evidence log, involve representative users, write requirements in observable language, model normal and exceptional flows, define quality targets, expose assumptions, and test the riskiest uncertainties early. Use prototypes to learn, not to conceal missing decisions. Keep a decision log and trace important requirements to tests. Finally, revisit requirements throughout development because understanding improves when stakeholders see working increments.

Chapter Summary

Problem discovery establishes the user need and evidence before a solution is selected. Requirements engineering converts that understanding into clear, feasible, prioritized, and testable capabilities and constraints. Stakeholder analysis identifies affected and influential parties; elicitation obtains evidence through interviews, observation, workshops, documents, prototypes, surveys, and data. Problem statements should remain solution-neutral, while personas and journeys represent user context.

User stories express value concisely, whereas use cases describe interactions and exceptions. Acceptance criteria make expected behaviour verifiable. Functional requirements specify system behaviour; non-functional requirements specify qualities and constraints such as performance, security, accessibility, and availability. Scope boundaries, assumptions, constraints, and MVP decisions prevent uncontrolled expansion. A prioritized backlog turns validated requirements into an adaptable delivery plan. Traceability, ethical practice, privacy, accessibility, and human review of AI-assisted work support responsible web application development.

Key Terms

Stakeholder: A person or organization that affects, is affected by, uses, governs, funds, or supports a system.

Problem discovery: Investigation of user difficulties, context, evidence, and desired outcomes.

Requirements engineering: The disciplined process of eliciting, analysing, documenting, validating, managing, and tracing requirements.

Problem statement: A concise, evidence-based description of a user problem and its consequences.

Elicitation: Systematic discovery of requirements from people, artefacts, observation, systems, and data.

Persona: A research-based archetype representing a significant user group.

User journey: A representation of user stages, actions, touchpoints, and difficulties.

User story: A short statement of a user role, desired capability, and value.

Use case: A structured description of actor–system interactions that achieve a goal.

Acceptance criteria: Testable conditions that determine whether a requirement or story is accepted.

Functional requirement: A required system behaviour, service, rule, input, output, or state change.

Non-functional requirement: A required quality attribute or constraint on a system or process.

Scope: The defined boundary of what a project includes.

Constraint: A limitation imposed by resources, policy, technology, law, or time.

MVP: The smallest coherent product that delivers meaningful value and enables learning.

Product backlog: An ordered, evolving list of product work.

Traceability: The ability to link objectives, evidence, requirements, implementation, and tests.

Stakeholder register: A record of stakeholder roles, interests, influence, and engagement plans.

Review Questions

Short Answer Questions

1. Distinguish a problem, a requirement, and a feature using one example.
2. What information should a stakeholder register contain?
3. Why are acceptance criteria necessary for a user story?
4. State two differences between functional and non-functional requirements.
5. What makes an MVP different from a list of unfinished features?

Long Answer Questions

1. Explain how interviews, observation, workshops, and prototyping can be combined during requirements elicitation.
2. Describe how a power–interest analysis can influence stakeholder engagement.
3. Compare user stories and use cases, including situations in which each is most useful.
4. Explain how scope boundaries, assumptions, constraints, and exclusions control project risk.
5. Discuss the role of privacy, accessibility, and security in requirements engineering.

Analytical / Application Questions

1. A college wants a web application for student counselling appointments. Identify at least six stakeholders, describe their interests, and propose an elicitation method for each.
2. Write a problem statement, two personas, three user stories, and acceptance criteria for a community equipment-lending service.
3. A team proposes payments, ratings, video meetings, multilingual support, and booking for its first release of a tutoring system. Analyse which capabilities belong in an MVP and justify a prioritized backlog order.

Multiple Choice Questions with Answers

Question 1

Which statement is the best example of a solution-neutral problem statement?

A. Build a mobile app for tutors immediately.

B. Tutors experience double bookings because availability is updated inconsistently across channels.

C. Add a blue calendar with push notifications.

D. Use React and PostgreSQL to manage lessons.

Correct Answer: B. Tutors experience double bookings because availability is updated inconsistently across channels.

Question 2

Which technique is most likely to reveal a user’s actual workaround during a task?

A. Contextual observation

B. A technology-selection meeting

C. A code review

D. A release retrospective

Correct Answer: A. Contextual observation

Question 3

Which item is a functional requirement?

A. The page shall load within two seconds.

B. The system shall prevent two confirmed bookings for the same slot.

C. The interface shall be accessible by keyboard.

D. The service shall achieve 99.5% availability.

Correct Answer: B. The system shall prevent two confirmed bookings for the same slot.

Question 4

Which acceptance criterion is most testable?

A. The booking process shall be attractive.

B. The system shall be modern and easy.

C. Given an available slot, when an authenticated student confirms it, then one booking is created and a reference is displayed.

D. Users should enjoy booking lessons.

Correct Answer: C. Given an available slot, when an authenticated student confirms it, then one booking is created and a reference is displayed.

Question 5

What is the principal purpose of a persona?

A. To define the database schema

B. To represent a researched user group and its context

C. To replace all usability testing

D. To specify hosting costs

Correct Answer: B. To represent a researched user group and its context

Question 6

Which item is normally an out-of-scope statement for a tutoring MVP?

A. Students can view available slots.

B. Tutors can publish availability.

C. The system prevents duplicate confirmed bookings.

D. The first release excludes online payments.

Correct Answer: D. The first release excludes online payments.

Question 7

Why should conflict prevention be enforced on the server rather than only in the interface?

A. Interfaces cannot display calendars.

B. A user or attacker can bypass client-side controls with a direct request.

C. Server rules eliminate the need for testing.

D. It makes all requirements non-functional.

Correct Answer: B. A user or attacker can bypass client-side controls with a direct request.

Question 8

In the weighted score Score = w_VV + w_RR + w_SS, what must be true of the weights in the method described?

A. Each weight must be greater than 5.

B. All weights must be zero.

C. The weights should sum to 1.

D. The weights must all be equal.

Correct Answer: C. The weights should sum to 1.

Question 9

Which statement best describes a product backlog?

A. A fixed list containing only interface screens

B. An ordered, evolving list of product work and validated needs

C. A database backup schedule

D. A list of stakeholders who attended interviews

Correct Answer: B. An ordered, evolving list of product work and validated needs

Question 10

What is the main value of traceability?

A. It guarantees that no requirement will ever change.

B. It links objectives and requirements to implementation and tests, supporting impact analysis.

C. It removes the need for stakeholder validation.

D. It prioritizes visual design over security.

Correct Answer: B. It links objectives and requirements to implementation and tests, supporting impact analysis.