Product Backlog and Requirements Engineering

Introduction and context

A product backlog is the evolving, ordered list of work needed to improve a product. In Scrum, it is not a contract, a project schedule, or a collection of technical tasks. It is a transparent statement of product opportunities, customer needs, technical improvements, defects, risks, compliance obligations, and learning activities. The Product Owner is accountable for maximizing product value through effective backlog management, while the Scrum Team contributes technical and domain knowledge.

Requirements engineering provides the disciplined practices used to discover, analyze, specify, validate, prioritize, and manage requirements. Agile requirements engineering applies these practices incrementally. Rather than attempting to define the entire product in detail before development, the team maintains enough shared understanding to make a sound near-term decision and deliberately postpones detail that may change.

This approach is often summarized as “just enough, just in time.” It does not mean “little analysis.” It means that analysis is continuous, evidence-based, and connected to delivery. A backlog item should become more precise as it approaches implementation, while distant items may remain expressed as outcomes, hypotheses, or broad capabilities.

Learning objectives

After studying this chapter, you should be able to:

1. Explain the relationship between requirements engineering, Scrum, and product value.
2. Structure requirements using themes, epics, features, and user stories.
3. Write effective user stories and acceptance criteria.
4. Represent functional, non-functional, regulatory, and technical requirements.
5. Apply ordering models such as value–risk analysis, weighted scoring, and WSJF cautiously.
6. Slice large requirements into independently valuable increments.
7. Evaluate backlog items using Definition of Ready considerations.
8. Refine a backlog collaboratively and recognize common failure modes.

1. Requirements engineering in an Agile product

Traditional requirements engineering commonly emphasizes a sequential flow: elicitation, specification, validation, and change control. Agile teams retain these intellectual activities but distribute them throughout product development. Requirements are hypotheses about a product need, not merely statements handed from analysts to programmers.

A useful distinction is:

• Functional requirement: what the product must do, such as allowing a user to reset a password.
• Non-functional requirement: a quality, constraint, or operational property, such as response time, availability, accessibility, privacy, or maintainability.
• Business requirement: the organizational outcome sought, such as reducing support calls.
• User requirement: a capability expressed from a stakeholder or user perspective.
• Technical requirement: an architectural, platform, integration, or security constraint.
• Transition requirement: a temporary need for migration, training, data conversion, or deployment.

A requirement is valuable only when its meaning is sufficiently shared to support a decision. Quality can be assessed using criteria such as clarity, testability, feasibility, consistency, traceability, and value. Agile teams also ask whether the item can be delivered and evaluated within a Sprint.

The backlog as a product model

The Product Backlog is a living product model. It records the most important known work, but it cannot represent every future detail with equal precision. Its ordering communicates the current economic and strategic judgment of the Product Owner, informed by developers, users, customers, compliance specialists, and other stakeholders.

Backlog transparency requires that people can understand:

• what problem an item addresses;
• who benefits and how;
• what evidence will demonstrate completion;
• what dependencies, assumptions, risks, and constraints exist; and
• why the item occupies its current position.

The backlog is emergent: new information may create, remove, split, merge, or reorder items. Change is not evidence of poor planning by itself; unmanaged or unexplained change is a governance problem.

2. Backlog hierarchy and levels of detail

Teams often use a hierarchy to connect strategy with implementable work. The labels are not a mandatory Scrum structure, but they are useful modeling devices.

An epic is not simply a very large user story. It is a container for a meaningful area of uncertainty or value that must be decomposed. A feature may contain several stories, but a story should normally be understandable, testable, and valuable without requiring the user to know the internal task breakdown.

User stories

A common template is:

As a [role], I want [capability], so that [benefit].

The template encourages attention to actor, action, and purpose. It is not a substitute for conversation. A story may be accompanied by notes, prototypes, business rules, examples, links to research, and acceptance criteria.

The INVEST heuristics are useful, but not absolute laws:

• Independent: minimizes unnecessary coupling.
• Negotiable: describes a problem and outcome rather than prescribing every implementation detail.
• Valuable: produces recognizable value for a stakeholder or reduces meaningful risk.
• Estimable: provides enough shared understanding for relative sizing.
• Small: can fit within a Sprint with appropriate testing and integration.
• Testable: has observable conditions of satisfaction.

For example, “As a learner, I want to filter courses by difficulty so that I can find an appropriate challenge” is stronger than “Build course filtering,” because the former identifies the user and purpose. However, it still requires clarification: Which difficulty scale? Can several levels be selected? What happens when no course matches?

3. Acceptance criteria and examples

Acceptance criteria define conditions that must be satisfied for a backlog item to be accepted. They are not the same as the Definition of Done. Acceptance criteria are specific to an item; the Definition of Done applies to all work meeting a product quality standard.

Effective criteria are:

• observable rather than subjective;
• concise but complete enough to remove material ambiguity;
• testable by examples or inspection;
• aligned with the stated user outcome; and
• explicit about important exceptions and boundaries.

A useful format is Given–When–Then:

• Given the initial context,
• When an event or action occurs,
• Then an observable result follows.

Example:

Story: As a learner, I want to filter courses by difficulty so that I can choose an appropriate challenge.

Acceptance criteria:

1. Given courses tagged Beginner, Intermediate, or Advanced, when the learner selects Intermediate, then only Intermediate courses are displayed.
2. Given that no course matches the selected filter, when the results load, then the system displays a clear empty-state message and a way to remove the filter.
3. Given a learner using keyboard navigation, when the filter is operated, then all controls are reachable and their state is announced to assistive technology.

Criteria should not become a hidden design specification. The team may decide during refinement whether filtering occurs client-side or server-side, provided the product behavior and quality constraints are met.

4. Functional and non-functional requirements

Functional requirements are often easier to see in a backlog, but non-functional requirements can determine whether a product is usable, secure, legal, and commercially viable. They should therefore be visible rather than left as informal assumptions.

Examples include:

• performance: 95% of search requests complete within 2 seconds under an agreed workload;
• availability: the service achieves 99.9% monthly availability, with exclusions explicitly defined;
• accessibility: key user journeys conform to the applicable accessibility standard and are verified through automated and manual testing;
• security: administrative actions require multi-factor authentication and are recorded in an audit log;
• privacy: personal data collection is limited to stated purposes and retained according to policy;
• compatibility: supported browsers and mobile operating systems are identified;
• maintainability: critical services expose monitored health checks and documented interfaces.

A non-functional requirement should be measurable where possible. “The system must be fast” is weak. “For 95% of requests, the product returns results within 2 seconds for 500 concurrent users under the agreed test dataset” is more testable. The metric remains incomplete unless the team also specifies the measurement method, environment, and relevant transaction type.

Quality requirements may be attached to individual stories, represented as cross-cutting backlog items, or governed by the Definition of Done. The choice depends on whether the requirement is local or applies to the product broadly. Security and privacy obligations usually require both backlog visibility and systematic governance.

5. Backlog refinement

Backlog refinement is the collaborative activity of adding detail, estimates, ordering, and size to backlog items. It is continuous rather than a formal Scrum event with a prescribed duration. Participants commonly include the Product Owner, Developers, users or domain experts, and specialists such as security or legal advisers when relevant.

A refinement conversation may ask:

1. What user or business problem is being addressed?
2. What evidence supports the proposed solution?
3. Who is affected, including excluded or vulnerable groups?
4. What behavior is in scope and out of scope?
5. What rules, data, integrations, and failure cases matter?
6. How will success be observed?
7. What risks, dependencies, and assumptions exist?
8. Can the item be delivered and verified within a Sprint?
9. What is the smallest valuable increment?

Refinement should not be used to assign every task in advance or force premature technical certainty. Developers need enough understanding to assess feasibility and identify risks, while the Product Owner retains accountability for value ordering.

Estimation and uncertainty

Relative estimation compares items rather than claiming false precision. Story points, size categories, or ideal days may be used, but none is a unit of business value. A story-point estimate is meaningful only within the team and its established reference examples.

A simple expected-value calculation can support prioritization when outcomes and probabilities are explicit:

Here, probability is a number from 0 to 1, and impact must use a stated scale or monetary unit. This is not a prediction of certainty; it is a decision aid. For example, an experiment with a 0.4 probability of producing a benefit valued at 50 points has an expected benefit of 20 points. If the experiment costs 5 points of effort, the comparison may be attractive, but strategic, ethical, and dependency considerations still matter.

6. Ordering the backlog

“Prioritization” can imply a fixed ranking based on one score. Scrum more accurately emphasizes ordering because the Product Owner must consider multiple dimensions and may deliberately place a lower-scoring item earlier due to learning, sequencing, compliance, or risk reduction.

Value, effort, risk, and urgency

A practical decision frame considers:

• customer and business value;
• time criticality and market window;
• risk reduction and opportunity enablement;
• effort, complexity, and dependencies;
• legal, safety, privacy, or contractual obligations;
• learning value and reversibility.

A weighted scoring model can make assumptions visible. If criteria have weights that sum to 1, the score is:

where S is the total score, wᵢ is the weight of criterion i, rᵢ is the item's rating on that criterion, and n is the number of criteria. If ratings use a 0–5 scale, the score also lies between 0 and 5 when all weights are non-negative and sum to 1. The model is useful only when criteria are defined consistently and scores are not presented as objective facts.

Worked example. Suppose a team assigns weights of 0.40 to customer value, 0.25 to strategic alignment, 0.20 to risk reduction, and 0.15 to urgency. An item receives ratings of 5, 4, 2, and 3 respectively. Then:

The result is 3.85 out of 5. It supports comparison with other items scored using the same rubric; it does not automatically determine the final order.

Cost of delay and WSJF

For products where timing is important, teams sometimes use Weighted Shortest Job First (WSJF). A commonly used form is:

Cost of Delay is often estimated as the sum of three relative components:

The components are relative scores, not necessarily currency. Job Size is a relative estimate of the work. WSJF is most defensible when items are comparable, scores are calibrated, and the organization accepts the assumptions. It should not be treated as a universal law: regulatory work, architectural sequencing, safety, and dependencies may override a numerical ranking.

If an item has Cost of Delay 18 and Job Size 6, its WSJF is 3. If another has Cost of Delay 12 and Job Size 2, its WSJF is 6; the second item has the higher ratio under the stated scale. Because the numerator and denominator are relative, the result has no physical unit.

Cost of delay divided by duration

Another useful economic approximation is:

This form is applicable when the team can estimate how delay accumulates over time and when duration is comparable across options. It should not be mixed casually with the WSJF formulation above. The selected model must be documented so that stakeholders know what the numbers mean.

7. Slicing techniques

Large items create delayed feedback, high integration risk, and uncertain estimates. Slicing means dividing work into increments that each produce usable learning or value. Good slices preserve a thin but complete path through relevant layers rather than dividing only by technical component.

Useful slicing patterns include:

• Workflow step: support registration before adding profile customization.
• Business rule: implement one eligibility rule, then expand the rule set.
• Data variation: support one file type before additional formats.
• User role: deliver a capability for learners before administrators, where this is genuinely valuable.
• Happy path and exceptions: deliver a safe basic path, then add recoverable failure cases; never omit essential safety or legal behavior.
• Read before write: provide visibility into existing data before enabling modification.
• Thin vertical slice: implement interface, service, data access, security, and tests for one narrow outcome.
• Experiment: build the smallest test capable of validating a critical assumption.

Avoid slices that are merely “database,” “API,” and “UI” if none can be evaluated independently. Such technical decomposition may be useful as tasks within a story, but it does not necessarily create product value.

A good slice is independently testable, minimizes duplication, has a clear outcome, and leaves the product in a releasable or safely demonstrable state. Some architectural enabling work is necessary before user value can appear; it should be explained in terms of the risk or capability it enables.

8. Definition of Ready considerations

Scrum does not prescribe a mandatory Definition of Ready. Teams may use readiness criteria as a working agreement, provided the criteria do not become a rigid approval gate that delays learning.

A backlog item may be considered ready for Sprint Planning when:

• the intended outcome and beneficiary are understood;
• acceptance criteria or examples are sufficiently clear;
• important assumptions and dependencies are visible;
• the item is small enough for the team’s normal Sprint;
• the team has identified material security, privacy, accessibility, and compliance implications;
• test data, environments, or external decisions needed immediately are available or have a credible plan; and
• Developers believe the item can be completed to the Definition of Done.

Readiness is relative to uncertainty. A discovery story may be ready even though the final implementation is unknown, because its purpose is to answer a question. Conversely, a detailed story may not be ready if a critical legal decision or integration contract remains unresolved.

The Definition of Ready should not require a complete design, exact estimate, signed document, or Product Owner approval of every technical detail. Such requirements can reduce collaboration and encourage analysis paralysis.

9. Case study: improving appointment scheduling

This is a fictional illustrative case.

Background

A university health service receives appointment requests through telephone and email. Students report long waiting times, and staff spend substantial effort resolving duplicate requests. The product goal is to provide a secure online scheduling service without excluding students who need assisted access.

Problem

The initial backlog contains a single epic: “Build online appointments.” It includes authentication, clinician calendars, appointment booking, cancellation, reminders, payments, reporting, and integration with a health-record system. The epic is too broad to estimate or deliver coherently.

Analysis

The team interviews students, reception staff, clinicians, and privacy personnel. It identifies several constraints:

• health information must be protected and access logged;
• appointment availability changes frequently;
• a student must not receive two bookings for the same time;
• cancellation rules differ by appointment type;
• assisted telephone booking must remain available;
• accessibility must be verified for the booking journey.

The team maps the workflow: find service, view available times, select a time, confirm identity, book, receive confirmation, and cancel or reschedule. It identifies the first valuable outcome as reducing telephone traffic for straightforward bookings, not delivering every administrative report.

Approach and solution

The Product Owner orders a thin vertical slice:

As an authenticated student, I want to view and book an available general consultation so that I can secure an appointment without calling reception.

Acceptance criteria include:

1. Only genuinely available slots are displayed.
2. A slot cannot be booked twice when two users attempt booking concurrently.
3. The confirmation shows service, date, time, location, and cancellation information.
4. The booking journey is keyboard accessible and provides appropriate labels.
5. A booking event is recorded without exposing unnecessary health information.

The team creates separate backlog items for cancellation, reminders, clinician administration, assisted-access workflows, and integration. It also creates a security and privacy item concerning access logging and retention. A technical spike investigates whether the existing calendar system supports atomic reservation; the spike is time-boxed and has a stated decision outcome.

For ordering, the team uses a weighted model, but places privacy and safety work earlier than a raw score would suggest. It slices reminders by email first, only after confirming that consent and contact data are reliable. It does not treat a “database story” as customer value; database work is included in the vertical slice or represented as an enabling item with explicit risk reduction.

Outcome

The first release supports a limited but usable consultation-booking path. Measures include the proportion of eligible bookings completed online, duplicate-booking incidents, accessibility defects, booking completion time, and calls requiring staff assistance. Feedback reveals that students need clearer cancellation information, so the backlog is reordered. The team avoids building low-value reporting before validating the core journey.

Learning points

1. An epic should be decomposed around outcomes and workflows.
2. Non-functional requirements must be visible before implementation, not inspected in at the end.
3. A small release can still require substantial security and concurrency analysis.
4. Ordering is a judgment informed by models, evidence, constraints, and ethics.
5. Product metrics and user feedback are requirements-engineering evidence.

10. Common failure modes and professional practice

The backlog as a requirements dump

A long list of unconnected requests obscures outcomes. Use themes, explicit ordering, and periodic removal of obsolete items.

Over-specification

Detailed solutions written before discovery constrain innovation and create rework. Specify the problem, boundaries, quality constraints, and evidence of success while leaving implementation choices to the team where possible.

Under-specification

Short stories without examples can conceal different interpretations. Use conversations, examples, prototypes, and explicit assumptions.

Hidden non-functional work

If security, accessibility, performance, and operational support are invisible, they are likely to be delayed. Make cross-cutting obligations measurable and assign ownership.

False numerical precision

A score such as 87.4 may look scientific even when ratings are subjective. Record the rubric, confidence, source of evidence, and date; use ranges when appropriate.

Proxy-driven ordering

Optimizing velocity, number of stories, or backlog size can displace user outcomes. Use outcome measures such as task success, retention, defect escape, accessibility, reliability, and customer impact.

Ethical and governance considerations

Requirements decisions affect privacy, inclusion, safety, and access. Teams should identify affected populations, assess unequal impacts, minimize data collection, document consent and retention, and provide human review where automated decisions may cause harm. Regulatory requirements should be translated into testable controls and traceable backlog items, while legal specialists remain responsible for legal interpretation.

AI-assisted requirements work

AI tools can summarize interviews, suggest acceptance-test examples, detect duplicate items, or identify missing edge cases. They can also invent requirements, reproduce bias, expose confidential information, and create unjustified confidence. Use approved tools, protect sensitive data, retain human accountability, verify outputs against authoritative sources, and record significant decisions. AI-generated text is a draft, not evidence that a requirement is correct.

Best-practice checklist

• Connect each major item to a problem, outcome, or risk.
• Order the backlog visibly and revisit the order when evidence changes.
• Make acceptance criteria observable and include important exceptions.
• Represent non-functional, security, privacy, accessibility, and operational needs.
• Slice by thin, testable, valuable increments.
• Use estimation and prioritization models as aids, not substitutes for judgment.
• Keep uncertainty explicit and use experiments where appropriate.
• Apply a lightweight, context-sensitive Definition of Ready.
• Validate delivered outcomes with users and product metrics.
• Remove stale, duplicate, and superseded items.

Chapter Summary

Product backlog management combines product strategy, requirements engineering, collaboration, and empirical learning. A backlog is an ordered and evolving product model, not a fixed list of specifications. Themes, epics, features, and user stories connect strategic outcomes with implementable increments. User stories are strengthened by conversation and testable acceptance criteria, while non-functional requirements make quality, security, privacy, accessibility, performance, and operational constraints visible.

Refinement progressively increases shared understanding. Slicing produces thin, independently testable increments and reduces feedback delay. Ordering should consider value, risk, urgency, effort, dependencies, learning, and governance. Weighted scoring and WSJF can expose assumptions, but numerical models remain decision aids rather than objective answers. A Definition of Ready may help a team assess Sprint readiness, but it is not a mandatory Scrum gate. Professional practice requires ethical attention, evidence-based decisions, and human accountability when digital or AI tools influence requirements.

Key Terms

Product Backlog: Ordered, evolving list of work and opportunities for a product.

Requirements engineering: Discipline of eliciting, analyzing, specifying, validating, and managing requirements.

Product Owner: Scrum accountability responsible for maximizing product value and managing backlog ordering.

Epic: Large area of capability, value, or uncertainty requiring decomposition.

Feature: Coherent product capability that may contain several stories.

User story: Concise description of a user-centered need and intended benefit.

Acceptance criteria: Item-specific conditions used to judge whether a requirement is satisfied.

Definition of Done: Shared quality standard required for completed work.

Non-functional requirement: Quality attribute, constraint, or operational property of a product.

Backlog refinement: Ongoing collaborative activity of clarifying, sizing, and ordering backlog items.

INVEST: Heuristics describing desirable properties of user stories.

Vertical slice: Small increment spanning the necessary product layers to deliver an outcome.

Technical debt: Future cost created when expedient implementation choices reduce maintainability or quality.

Weighted scoring: Decision model combining criterion ratings with explicit weights.

Cost of Delay: Relative or monetary value lost by postponing an item.

WSJF: Weighted Shortest Job First, commonly calculated as Cost of Delay divided by Job Size.

Definition of Ready: Team working agreement describing conditions that support Sprint readiness.

Traceability: Ability to connect requirements with sources, implementation, tests, and outcomes.

Review Questions

Short Answer Questions

1. Distinguish a functional requirement from a non-functional requirement and provide one example of each.
2. What purpose do acceptance criteria serve, and how do they differ from the Definition of Done?
3. Explain why a Product Backlog is described as emergent.
4. State two advantages and one limitation of relative estimation.
5. What does a lightweight Definition of Ready help a Scrum Team assess?

Long Answer Questions

1. Explain how requirements engineering is adapted for an Agile product-development environment.
2. Compare epics, features, user stories, and tasks, including the relationship between them.
3. Describe how a team can make non-functional requirements visible and testable.
4. Explain weighted scoring and WSJF, including why neither should be treated as an automatic ordering rule.
5. Discuss the principles of effective backlog refinement and the risks of over-specification.

Analytical / Application Questions

1. A team has a story that cannot be completed within one Sprint because it includes search, payments, reporting, and administration. Propose a sequence of valuable slices and justify your ordering.
2. Design acceptance criteria for a password-reset story, including security, usability, and failure cases. Explain which criteria are item-specific and which belong in the Definition of Done.
3. A backlog item has criterion ratings of 4, 5, 3, and 2, with weights 0.35, 0.30, 0.20, and 0.15. Calculate its weighted score and explain two reasons why the Product Owner might still place another item earlier.

Multiple Choice Questions with Answers

Question 1

Which statement best describes the Product Backlog?

A. A fixed contract containing every requirement known before development begins

B. An ordered, evolving list of work and opportunities for improving a product

C. A list containing only technical tasks assigned to Developers

D. A Sprint-by-Sprint schedule that cannot be changed

Correct Answer: B. An ordered, evolving list of work and opportunities for improving a product

Question 2

Which item is a non-functional requirement?

A. A user can cancel an appointment

B. An administrator can export a report

C. Ninety-five percent of search requests complete within two seconds under an agreed workload

D. A learner can filter courses by difficulty

Correct Answer: C. Ninety-five percent of search requests complete within two seconds under an agreed workload

Question 3

Which acceptance criterion is most observable and testable?

A. The interface should be pleasant

B. The system should respond quickly

C. The user should feel confident

D. When no course matches a filter, the system displays an empty-state message and a way to remove the filter

Correct Answer: D. When no course matches a filter, the system displays an empty-state message and a way to remove the filter

Question 4

In a weighted scoring model, the weights should normally sum to what value when expressed as proportions?

A. 0

B. 0.5

C. 1

D. 1000

Correct Answer: C. 1

Question 5

Using the chapter’s weighted-scoring formula, what is the score for ratings 5, 4, 2, and 3 with weights 0.40, 0.25, 0.20, and 0.15?

A. 2.85

B. 3.25

C. 3.85

D. 4.25

Correct Answer: C. 3.85

Question 6

If an item has Cost of Delay 18 and Job Size 6, what is its WSJF under the stated formula?

A. 3

B. 6

C. 12

D. 108

Correct Answer: A. 3

Question 7

Which is the strongest example of a thin vertical slice?

A. Create the database schema without connecting it to a user outcome

B. Build the user interface only and postpone all service behavior

C. Deliver one narrow booking path across interface, service, data, security, and testing

D. Write a complete technical architecture document before testing any assumption

Correct Answer: C. Deliver one narrow booking path across interface, service, data, security, and testing

Question 8

Which statement about a Definition of Ready is correct?

A. It is a mandatory Scrum artifact defined with fixed universal criteria

B. It can be a team working agreement supporting readiness without becoming a rigid approval gate

C. It replaces the Definition of Done

D. It requires every implementation detail to be finalized

Correct Answer: B. It can be a team working agreement supporting readiness without becoming a rigid approval gate

Question 9

Which practice best addresses hidden non-functional requirements?

A. Leave them for final testing so they do not slow refinement

B. Represent them visibly through measurable criteria, backlog items, or product-wide quality policies

C. Assume the framework supplies them automatically

D. Replace them with story points

Correct Answer: B. Represent them visibly through measurable criteria, backlog items, or product-wide quality policies

Question 10

A story is “As a learner, I want to filter courses so that I can find an appropriate challenge.” Which missing refinement question is most important?

A. Which font should every page use permanently?

B. Which difficulty levels exist and what happens when no course matches?

C. Which developer will write every task before discussion?

D. How can all future features be specified now?

Correct Answer: B. Which difficulty levels exist and what happens when no course matches?