3. Security Requirements, Risk Assessment and Abuse Cases

Introduction

Secure software is not produced merely by removing defects from source code. Security must be considered before architecture, implementation, and testing decisions become difficult or expensive to change. The starting point is a clear understanding of what the system must protect, from whom, under which conditions, and at what level of assurance. These questions are addressed through security requirements, risk assessment, misuse and abuse cases, security user stories, and explicit acceptance criteria.

Security requirements translate business objectives and security concerns into verifiable properties of a system. Risk assessment determines which threats deserve the greatest attention by examining likelihood, impact, uncertainty, and organizational risk appetite. Abuse cases explore how legitimate functionality might be manipulated or misused. Together, these practices establish a bridge between business priorities, threat modeling, secure design, implementation, and security testing.

This chapter treats requirements and risk analysis as decision-making activities rather than documentation exercises. A requirement that cannot be traced to a risk, business need, legal obligation, or security principle is often difficult to justify. Conversely, a risk that does not lead to an owner, treatment decision, or testable control is unlikely to be managed effectively.

Learning Objectives

After studying this chapter, students should be able to:

1. Distinguish functional, non-functional, regulatory, and operational security requirements.

2. Elicit security requirements from stakeholders, assets, threats, business processes, and trust boundaries.

3. Construct misuse cases, abuse cases, security user stories, and measurable acceptance criteria.

4. Apply qualitative and quantitative methods to analyze information-security risk.

5. Classify data and prioritize security controls according to business impact and risk appetite.

6. Explain how governance, ethics, privacy, and professional responsibility influence security decisions.

7. Evaluate security requirements and risk treatment decisions in a realistic software-development scenario.

From Business Need to Security Requirement

A requirement states a capability, constraint, or quality that a system must satisfy. A security requirement is concerned with protecting confidentiality, integrity, availability, authenticity, accountability, privacy, resilience, or other security properties. Security requirements may be explicit—for example, “the system shall require multifactor authentication for privileged users”—or implicit in business and legal expectations, such as protecting customer payment information.

Functional security requirements describe actions the system must perform. Examples include authenticating a user, authorizing a transaction, recording an audit event, revoking a session, encrypting a backup, or notifying an administrator of suspicious activity. Non-functional security requirements describe qualities or constraints, such as maximum recovery time, password-rate-limiting performance, logging completeness, cryptographic strength, or resistance to denial-of-service conditions.

The distinction is useful but not absolute. “The system shall lock an account after five failed authentication attempts” is functional, while “authentication decisions shall complete within two seconds for 95 percent of requests” combines security functionality with a performance constraint. Requirements should therefore be classified for communication and testing, not treated as isolated categories.

A well-formed security requirement is:

• specific enough to guide design;
• necessary and justified by a business, threat, legal, or policy concern;
• unambiguous and internally consistent;
• feasible within technical and organizational constraints;
• measurable or testable;
• traceable to a stakeholder need, asset, risk, or control; and
• assigned to an owner or responsible delivery team.

Weak requirement: “The application must be secure.”

Stronger requirement: “For every administrative login, the application shall require a phishing-resistant second factor, generate an audit event containing user identity, timestamp, result, and source address, and retain the event for twelve months.”

The stronger statement still requires contextual validation—for example, whether the retention period is legally appropriate and whether the second factor is genuinely phishing-resistant—but it provides a basis for architecture and testing.

Eliciting Requirements

Security requirements rarely come from one stakeholder. Product owners understand business value and tolerable disruption; system architects understand technical dependencies; legal and privacy specialists identify obligations; security professionals analyze threats; operations teams understand monitoring and recovery; and users reveal practical workflow constraints. Effective elicitation brings these perspectives together.

Useful elicitation techniques include interviews, facilitated workshops, document analysis, observation of business processes, review of incidents and audit findings, threat modeling, regulatory mapping, and analysis of comparable systems. A workshop may begin with a process such as “customer changes a bank-account number,” identify assets and trust boundaries, and ask what could go wrong, who could cause it, and what evidence would be needed afterward.

Requirements should be captured with context. A statement such as “encrypt sensitive data” is incomplete unless it identifies data states, applicable algorithms or approved cryptographic services, key ownership, rotation, access control, and recovery expectations. Encryption in transit does not protect a database copied to an unprotected backup, and encryption at rest does not address unauthorized use by an authenticated but compromised service account.

Traceability is central. A requirements traceability matrix may link a business objective to an asset, threat, risk, security requirement, implementation control, test case, and evidence. Traceability supports impact analysis when requirements change and helps demonstrate due diligence during audit or incident investigation.

Security Requirements Across the Lifecycle

Security requirements should be established during inception and refined during analysis, design, implementation, deployment, and operation. Early requirements may be broad: “only authorized clinicians may view a patient record.” During design, this can become a role and attribute-based authorization model. During implementation, it becomes policy enforcement at each service endpoint. During testing, it becomes test cases for role combinations, revoked accounts, direct-object-reference manipulation, and emergency access. During operation, it becomes a monitoring and review requirement.

A useful lifecycle practice is to define security acceptance criteria before development begins. This prevents security from becoming an informal expectation evaluated only at the end. Requirements also need change control. A new integration, cloud service, data type, or business workflow can introduce a new trust boundary and invalidate earlier assumptions.

Misuse Cases, Abuse Cases, and Security User Stories

A use case describes how an actor achieves a legitimate goal. A misuse case describes how an actor may achieve an undesirable outcome by exploiting or misusing system behavior. An abuse case emphasizes intentional malicious behavior, although the terms are often used interchangeably in practice.

For a password-reset feature, the normal use case is that a legitimate user regains access. A misuse case is that an attacker enumerates accounts through different responses. An abuse case is that an attacker takes over an account by manipulating reset tokens. The purpose is not to create dramatic attack narratives; it is to reveal security requirements associated with realistic misuse paths.

A misuse or abuse case normally identifies:

• the initiating actor and relevant capabilities;
• the target asset or security property;
• the normal system behavior being exploited;
• preconditions and assumptions;
• the sequence of malicious actions;
• possible system responses and alternatives;
• business, technical, or legal consequences; and
• controls and acceptance criteria that reduce the threat.

The actor need not be an external criminal. It may be a malicious insider, compromised service, careless administrator, automated bot, fraudulent customer, or legitimate user acting outside authorization. Considering accidental misuse is also valuable because confusing interfaces and unsafe defaults frequently create security incidents without malicious intent.

Security user stories adapt agile user-story practice to security concerns. A common pattern is: “As a [role or defender], I want [security capability], so that [security outcome].” For example: “As a security administrator, I want every privileged action linked to a strong user identity and a request identifier, so that investigations can distinguish individual responsibility from shared credentials.” The story should be accompanied by acceptance criteria and, where appropriate, abuse scenarios.

Acceptance criteria should be observable. For the preceding story, criteria might require that privileged actions are rejected without multifactor authentication; each event contains identity, action, target, timestamp, result, and correlation identifier; events cannot be altered by ordinary administrators; and alerts are generated for specified high-risk actions. Criteria should also address failure modes. What happens if the logging service is unavailable? Is the action blocked, queued, or allowed with a compensating control? Such decisions are risk decisions and should not be left implicit.

Risk as a Basis for Prioritization

Risk is commonly expressed as the effect of uncertainty on objectives. In security practice, a simple model is:

Risk ≈ Likelihood × Impact

This expression is not a physical law. It is a communication and prioritization device. Likelihood may reflect threat capability, exposure, vulnerability, and existing controls. Impact may include financial loss, safety consequences, legal penalties, privacy harm, operational interruption, loss of competitive advantage, and reputational damage. A sophisticated assessment should also record assumptions and uncertainty rather than presenting a numerical score as objective truth.

Risk assessment generally includes risk identification, analysis, evaluation, and treatment. Identification asks what can happen and to what asset. Analysis estimates likelihood and consequence. Evaluation compares the result with criteria such as risk appetite or tolerance. Treatment selects a response: avoid, reduce, transfer, share, or accept the risk. Risk acceptance must be explicit, authorized, time-bounded where appropriate, and supported by an understanding of residual risk.

Qualitative Analysis

Qualitative analysis uses categories such as low, medium, high, and critical. A likelihood-impact matrix can be useful when data is limited or rapid prioritization is required. For instance, a remotely exploitable flaw affecting a payment authorization service may be rated high likelihood and critical impact, while a local issue affecting a non-sensitive development environment may be low likelihood and low impact.

Qualitative methods are efficient and accessible, but they can conceal disagreement. Two assessors may interpret “possible” or “major impact” differently. Organizations should therefore define rating scales with examples, document assumptions, and use calibration sessions. A matrix should support judgment, not replace it.

Quantitative Analysis

Quantitative analysis attempts to express loss in monetary or numerical terms. In a commonly used model, single loss expectancy (SLE) is calculated as:

SLE = Asset Value × Exposure Factor

Annualized loss expectancy (ALE) is:

ALE = SLE × Annual Rate of Occurrence

If a service outage is estimated to cause a loss of $80,000 per event, and comparable events are expected 0.25 times per year, the ALE is $20,000. A control costing $12,000 annually might be economically attractive if it reduces the expected loss sufficiently, although intangible effects, legal duties, safety, and uncertainty must also be considered.

Quantitative analysis can create an impression of precision unsupported by evidence. Estimates should use ranges, sensitivity analysis, and confidence descriptions where possible. A single number should not be used to justify ignoring a low-probability event with catastrophic safety consequences or a privacy obligation that applies regardless of expected financial loss.

Data Classification and Business Impact

Data classification communicates the protection required for information. Typical categories include public, internal, confidential, and restricted, although names vary by organization. Classification should consider confidentiality, integrity, availability, privacy, contractual duties, strategic value, and consequences of unauthorized disclosure or alteration.

Classification is not simply a label applied to a file. It should influence collection, storage, transmission, access, retention, disposal, backup, monitoring, and incident response. A public product brochure may have low confidentiality requirements but high integrity requirements if fraudulent changes could cause safety or financial harm. A dataset may also change classification when combined with other data; seemingly harmless fields can enable identification or profiling.

Business impact analysis examines the consequences of disruption or compromise. It identifies critical processes, dependencies, maximum tolerable downtime, recovery time objectives, recovery point objectives, legal deadlines, and affected stakeholders. The analysis informs both preventive controls and resilience requirements.

Data minimization is an important companion principle. The safest sensitive record is often the one that is not collected, retained, copied, or exposed unnecessarily. Classification should therefore support decisions about whether data is needed at all, not merely how it will be protected after collection.

Prioritizing Controls and Risk Appetite

Risk appetite is the amount and type of risk an organization is willing to pursue or retain in support of its objectives. Risk tolerance defines acceptable variation around objectives. These concepts are organizational, not purely technical. A startup may tolerate short service interruptions but have zero tolerance for unauthorized disclosure of regulated data. A hospital may accept a carefully governed emergency-access risk because delaying treatment can threaten life, while requiring strong auditability and retrospective review.

Controls should be prioritized according to risk reduction, feasibility, dependency, assurance, and business impact. A useful order is to address high-impact threats to critical assets, externally exposed attack paths, systemic weaknesses, and controls that enable other safeguards. Preventive, detective, responsive, and corrective controls should be considered together. Multifactor authentication may prevent many account compromises, but monitoring, session revocation, recovery procedures, and incident response remain necessary.

Control selection should avoid control theater—activities that produce evidence of compliance without materially reducing risk. A mandatory annual training module may be useful, but it is not a substitute for secure defaults, least privilege, technical enforcement, and usable workflows. Conversely, a technically strong control that prevents essential business work may be bypassed or disabled. Effective security balances assurance with usability, availability, cost, and operational reality.

Illustrative Case: The Meridian Benefits Portal

Background

Meridian Benefits is an illustrative regional organization that provides an online portal through which employees select benefits, update dependents, and submit reimbursement claims. The portal integrates with payroll, an identity provider, a document-storage service, and an external claims processor.

Situation and Problem

A proposed feature would allow an employee to change a bank account for reimbursement. The product team initially described it as a simple profile-edit function. Security review identified several assets: bank details, identity information, claim documents, payroll records, and audit evidence. The feature also introduced trust boundaries between the portal, identity provider, payroll system, and claims processor.

The team identified abuse cases including credential-stuffing followed by bank-account replacement, manipulation of an object identifier to alter another employee’s profile, replay of a change request, and abuse by a support employee. Business impact included fraudulent payments, privacy harm, payroll disruption, investigation cost, and loss of employee trust.

Analysis

A qualitative assessment rated unauthorized bank-account change as high likelihood and high impact because the function would be Internet-facing and financially attractive. A replay attack was rated medium likelihood but high impact. A support-agent misuse scenario was lower likelihood but high impact because privileged access could affect many accounts. The risk appetite allowed ordinary profile corrections but did not allow unverified changes to payment destinations.

Data classification placed bank details and identity documents in a restricted category. The business impact analysis established that reimbursement processing was important but could tolerate a short manual-review delay. This created an opportunity to use stronger controls without making the process unavailable.

Approach and Security Requirements

The team converted the analysis into requirements:

• a payment-destination change shall require recent authentication and a phishing-resistant second factor;
• the old and new contact channels shall receive notifications, without disclosing sensitive account details;
• changes shall enter a short review or cooling-off period before payment use;
• authorization shall be enforced server-side using the authenticated employee’s identity, not a client-supplied identifier;
• each request shall contain a one-time, expiring token and be protected against replay;
• privileged support access shall require approved justification, limited scope, and complete audit logging;
• high-risk changes shall be monitored and subject to rapid reversal; and
• the system shall preserve evidence sufficient to reconstruct who initiated, approved, and executed a change.

Acceptance testing included attempts to modify another user’s record, reuse a token, bypass multifactor authentication, submit duplicate requests, and perform a change through an alternative API endpoint. Operational testing verified alert routing, log protection, recovery of the previous account, and handling of identity-provider or notification-service failure.

Outcome

The feature was released with a small delay but avoided treating convenience as the only success criterion. The review also identified a broader architectural issue: several services trusted user-supplied identifiers. Correcting that pattern reduced risk beyond the original feature. The case illustrates why abuse cases are valuable early: they exposed authorization, transaction integrity, monitoring, and recovery requirements that were absent from the initial functional specification.

Learning Points

The most important lesson is that risk analysis changes product design. Data classification influenced control strength; business impact analysis made a cooling-off period acceptable; risk appetite justified rejecting immediate payment use; and acceptance criteria converted security intentions into testable behavior.

Common Failures and Practical Limitations

A frequent mistake is beginning with controls rather than risks. Requiring encryption, logging, or multifactor authentication may be appropriate, but the organization should understand what threat the control addresses, where it applies, and what residual risk remains. Another mistake is treating a completed risk register as evidence that risk is managed. Risks need owners, deadlines, treatment decisions, and review triggers.

Other common failures include vague requirements, duplicated controls, excessive reliance on perimeter defenses, ignoring insider and supply-chain threats, omitting availability and recovery, failing to analyze third-party interfaces, and assuming that authenticated users are trustworthy. Risk ratings may also be manipulated to secure project approval or to make a backlog appear manageable. Independent review and transparent assumptions help reduce this bias.

Security requirements can conflict. Strong verification may reduce fraud but increase accessibility barriers. Detailed logging improves accountability but may create privacy risks. Long retention supports investigation but increases the consequences of a breach. These tensions should be resolved through documented trade-offs, privacy and accessibility review, and authorization by an accountable decision-maker.

Governance, Ethics, and Professional Responsibility

Risk acceptance is a governance decision. Developers and analysts should not silently accept risks that affect customers, employees, public safety, or regulated information. Decision-makers require clear descriptions of uncertainty, alternatives, residual risk, and affected stakeholders. Security professionals have duties of confidentiality, competence, honesty, and proportionality.

Privacy principles—including purpose limitation, data minimization, transparency, accuracy, retention limitation, and individual rights—should be incorporated into requirements where personal data is involved. Abuse-case analysis must be conducted responsibly: realistic attack paths should be documented and tested in authorized environments, while sensitive exploit details should be protected according to organizational policy.

Regulatory and contractual requirements may establish a minimum baseline, but compliance is not equivalent to security. An organization can satisfy a checklist while leaving an exploitable business process unprotected. Conversely, a risk-based control may exceed a formal requirement because the potential harm is substantial.

Contemporary Developments

Modern development practices increasingly express security requirements as code, automated policy checks, infrastructure controls, and continuous evidence. Security acceptance criteria can be integrated into CI/CD pipelines, while software composition analysis, secret detection, API testing, and infrastructure-as-code scanning provide evidence for selected requirements. Automation improves consistency but cannot determine business impact or risk appetite by itself.

Threat-informed prioritization is also becoming more important as organizations use cloud services, APIs, mobile clients, software supply chains, and machine-learning components. Artificial intelligence can assist in clustering findings, generating abuse-case candidates, and identifying anomalous behavior, but its outputs require human validation. AI-generated requirements may be incomplete, overconfident, or biased toward technically visible threats. Sensitive data should not be submitted to external tools without authorization, and automated risk scores should remain explainable and reviewable.

Continuous assessment is replacing the assumption that a one-time requirements workshop is sufficient. New dependencies, vulnerabilities, business processes, and attack techniques create reassessment triggers. Mature organizations connect threat modeling, security testing, monitoring, incident lessons, and product governance so that operational evidence updates future requirements.

Best-Practice Guidance

A disciplined team should:

1. Begin with business processes, assets, stakeholders, and trust boundaries.

2. Use both normal use cases and abuse cases to expose security needs.

3. Write requirements in testable language and define failure behavior.

4. Trace requirements to risks, controls, tests, and operational evidence.

5. Classify data according to business impact and lifecycle handling needs.

6. Use qualitative analysis for speed, quantitative analysis where evidence supports it, and document uncertainty in both.

7. Assign risk owners and obtain explicit, authorized acceptance of residual risk.

8. Prioritize controls according to material risk reduction rather than checklist completion.

9. Reassess when architecture, data, suppliers, interfaces, or business processes change.

10. Treat usability, privacy, accessibility, resilience, and security as connected design concerns.

Chapter Summary

Security requirements convert business, legal, privacy, and threat concerns into verifiable system properties. They may be functional or non-functional and should be specific, feasible, traceable, and testable. Elicitation requires collaboration among business, technical, operational, legal, and security stakeholders.

Misuse and abuse cases examine how legitimate functionality may be manipulated by attackers, insiders, compromised services, or careless users. Security user stories make security work visible in development, while acceptance criteria define observable evidence and failure behavior. Risk assessment identifies, analyzes, evaluates, and treats uncertainty affecting security objectives. Qualitative methods are efficient but judgment-dependent; quantitative methods support economic comparison but depend on uncertain estimates.

Data classification and business impact analysis help determine the protection and resilience required. Controls should be prioritized according to impact, likelihood, risk appetite, feasibility, and residual risk. Governance requires explicit ownership and authorized risk acceptance. Ethical and privacy considerations prevent security decisions from becoming narrowly technical. Continuous reassessment, automation, and human oversight are essential as systems and threats evolve.

Key Terms

Security requirement: A verifiable capability or constraint intended to protect a system, asset, or stakeholder.

Functional security requirement: A required security-related action performed by a system.

Non-functional security requirement: A measurable security quality or constraint on system behavior.

Abuse case: A scenario describing intentional misuse of a system to cause an undesirable outcome.

Misuse case: A scenario showing how a system’s legitimate behavior may be exploited or misused.

Security user story: An agile statement of a security need from the perspective of a role or defender.

Acceptance criterion: An observable condition that must be satisfied for a requirement or story to be accepted.

Asset: Something valuable that requires protection, including data, services, identities, or reputation.

Threat: A potential cause of an unwanted security event.

Vulnerability: A weakness that can be exploited or contribute to an adverse event.

Risk: The effect of uncertainty on objectives, often analyzed through likelihood and impact.

Risk appetite: The amount and type of risk an organization is willing to pursue or retain.

Residual risk: Risk remaining after controls and treatment are applied.

Risk treatment: The decision to avoid, reduce, transfer, share, or accept risk.

Data classification: Categorization of information according to protection and handling needs.

Business impact analysis: Assessment of consequences resulting from disruption or compromise of processes or assets.

Single loss expectancy: Estimated loss from one occurrence of an adverse event.

Annualized loss expectancy: Estimated expected loss over a year based on event frequency.

Trust boundary: A point where assumptions about identity, authority, or data reliability change.

Traceability: The ability to link a requirement through risks, controls, implementation, tests, and evidence.

Review Questions

Short Answer Questions

1. What distinguishes a functional security requirement from a non-functional security requirement?

2. Why should security requirements be traceable to risks or business objectives?

3. What is the purpose of an abuse case?

4. Define residual risk and risk appetite.

5. Why is data classification relevant to software design?

Long Answer Questions

1. Explain a systematic process for eliciting and validating security requirements across the software lifecycle.

2. Compare qualitative and quantitative risk analysis, including their strengths and limitations.

3. Discuss how security user stories and acceptance criteria can improve agile software development.

4. Explain how business impact analysis and data classification influence control prioritization.

5. Analyze the governance and ethical issues involved in accepting residual security risk.

Analytical / Application Questions

1. An online examination system permits students to upload answers and allows administrators to alter grades. Construct two abuse cases and derive at least four security requirements from them.

2. A cloud application stores customer identity documents and public product information. Propose a classification scheme, identify likely impacts, and recommend different controls for the two data categories.

3. A proposed control costs $18,000 annually and is expected to reduce an annualized loss expectancy from $60,000 to $25,000. Evaluate the financial case for the control and identify non-financial factors that could change the decision.

22. Multiple Choice Questions with Answers

Question 1

Which statement is the strongest security requirement?

A. The system must be secure.

B. The system should use good authentication.

C. Administrative login shall require multifactor authentication and create an auditable event containing identity, time, result, and source.

D. Users should not misuse the system.

Correct Answer: C. Administrative login shall require multifactor authentication and create an auditable event containing identity, time, result, and source.

Question 2

What is the principal purpose of an abuse case?

A. To describe the normal business workflow only

B. To identify how functionality may be intentionally misused and what controls are needed

C. To replace all security testing

D. To calculate annualized loss expectancy

Correct Answer: B. To identify how functionality may be intentionally misused and what controls are needed

Question 3

Which item most directly improves the testability of a security user story?

A. A broad statement that security is important

B. A list of unrelated technical products

C. Observable acceptance criteria describing expected security behavior

D. A promise to review security after release

Correct Answer: C. Observable acceptance criteria describing expected security behavior

Question 4

An organization estimates a single loss expectancy of $80,000 and an annual rate of occurrence of 0.25. What is the annualized loss expectancy?

A. $20,000

B. $80,250

C. $320,000

D. $3,200

Correct Answer: A. $20,000

Question 5

Which factor is most important when assigning a data classification?

A. The file’s visual format

B. The number of characters in the data

C. The consequences of unauthorized disclosure, alteration, or unavailability

D. Whether the data is stored on a laptop

Correct Answer: C. The consequences of unauthorized disclosure, alteration, or unavailability

Question 6

A risk assessment rates a payment-destination change as high impact because fraud could redirect many payments. Which response best reflects risk-based design?

A. Allow immediate changes without verification for usability

B. Require stronger authentication, monitoring, and a controlled delay before use

C. Delete all audit records after the change

D. Treat the risk as acceptable because the feature is functional

Correct Answer: B. Require stronger authentication, monitoring, and a controlled delay before use

Question 7

What is a major limitation of a qualitative risk matrix?

A. It cannot represent any security concern

B. It always requires exact financial loss data

C. Different assessors may interpret likelihood and impact categories inconsistently

D. It prevents organizations from prioritizing controls

Correct Answer: C. Different assessors may interpret likelihood and impact categories inconsistently

Question 8

Which statement about risk acceptance is correct?

A. It should be implicit when a project deadline is near

B. It eliminates the underlying threat

C. It should be explicit, authorized, and based on an understanding of residual risk

D. Only developers may accept organizational risk

Correct Answer: C. It should be explicit, authorized, and based on an understanding of residual risk

Question 9

Why is server-side authorization important in the Meridian Benefits case?

A. It makes notifications unnecessary

B. It prevents reliance on a client-supplied object identifier for access decisions

C. It guarantees that passwords cannot be stolen

D. It removes the need for audit logs

Correct Answer: B. It prevents reliance on a client-supplied object identifier for access decisions

Question 10

Which use of artificial intelligence is most appropriate in requirements and risk work?

A. Allowing an AI-generated risk score to make final acceptance decisions automatically

B. Using AI to suggest abuse cases while requiring human validation and governance

C. Submitting all confidential data to public tools without review

D. Replacing business stakeholders with automated threat labels

Correct Answer: B. Using AI to suggest abuse cases while requiring human validation and governance