Security, Privacy, Ethics and Governance

Introduction and Context

Software quality is not limited to functional correctness, usability, or performance. A system may satisfy its specification and still be unacceptable if it exposes personal data, enables unsafe decisions, excludes users with disabilities, violates law, or cannot demonstrate how important decisions were made. Security, privacy, accessibility, safety, ethics, compliance, auditability, and responsible reporting are therefore quality and risk concerns rather than optional additions.

For software project managers, the strategic challenge is to convert broad principles into decisions, controls, evidence, and accountability throughout the product life cycle. This requires governance: the system by which authority, responsibilities, policies, controls, monitoring, and corrective action are established. Governance does not replace engineering. It ensures that engineering effort is directed toward risks that matter to users, the organization, regulators, and society.

A useful distinction is that security protects systems and information from unauthorized actions; privacy governs the fair and lawful handling of information about people; safety prevents software from causing unacceptable physical, economic, or social harm; and ethics addresses what ought to be done, including issues not fully covered by law. These areas overlap but are not identical.

Learning Objectives

By the end of this chapter, learners should be able to:

1. Explain how security, privacy, safety, accessibility, ethics, and compliance contribute to software quality.
2. Apply secure-development and privacy-by-design practices across a project life cycle.
3. Establish governance structures, controls, evidence, and audit trails.
4. Identify and manage supplier and third-party risk.
5. Design responsible vulnerability and incident-reporting processes.
6. Evaluate an applied case involving competing quality, risk, and ethical concerns.

1. A Quality and Risk Governance Perspective

Risk is the effect of uncertainty on objectives. In practice, a project manager asks four questions:

• What could happen?
• Why could it happen?
• What would be the consequence?
• What controls reduce likelihood or impact?

A qualitative risk model often describes risk as a combination of likelihood and impact. It is not a universal mathematical law, but it is a useful prioritization device:

Likelihood and impact may be rated on ordinal scales such as 1 to 5. The resulting score should not be treated as a precise probability or monetary forecast. Ratings depend on assumptions, evidence, and the expertise of assessors. High-impact risks, especially those involving safety, discrimination, or legal rights, may require action even when likelihood is uncertain.

Governance converts risk information into authority and action. Typical governance questions include:

• Who owns the risk?
• Who may accept residual risk?
• Which controls are mandatory?
• What evidence demonstrates compliance?
• What triggers escalation or project suspension?
• How are exceptions approved and reviewed?

A practical governance structure includes a product owner, engineering lead, security and privacy representatives, quality assurance, legal or compliance advisers, operations, and—where relevant—safety, accessibility, or ethics specialists. Responsibility should be explicit rather than assumed.

The risk-control-evidence relationship

A mature project connects each material risk to controls and evidence. For example:

This traceability supports assurance, reduces duplicated effort, and makes audits more meaningful.

2. Secure Development

Secure development integrates protection into requirements, design, implementation, testing, deployment, and maintenance. Treating security only as a final penetration test is inadequate because many vulnerabilities originate in architecture, insecure requirements, weak dependency choices, or unsafe operational assumptions.

Security principles

Important principles include:

• Least privilege: each person, process, or service receives only the access necessary for its role.
• Secure defaults: the safest practical configuration is the default.
• Defense in depth: multiple independent controls reduce dependence on one barrier.
• Fail securely: failures should not disclose information or grant unauthorized access.
• Complete mediation: access should be checked whenever a protected resource is used.
• Separation of duties: critical actions should not depend on one unchecked individual.
• Secure by design: security properties are considered before implementation.
• Dependency transparency: libraries, services, and build components are identified and maintained.

Security requirements should be testable. “The system must be secure” is not measurable. Better requirements specify authentication, authorization, confidentiality, integrity, availability, logging, recovery, and acceptable exposure. For example: “Administrative access shall require multi-factor authentication, shall be reviewed quarterly, and shall be logged with user, time, action, and outcome.”

Secure life-cycle practices

During initiation, identify assets, trust boundaries, threat actors, legal obligations, and security objectives. During analysis and design, use threat modelling to examine abuse cases, data flows, entry points, privilege transitions, and failure modes. During implementation, use code review, secure coding standards, secrets management, dependency scanning, and protected build pipelines. During testing, combine automated tests with specialist review, abuse-case testing, configuration testing, and penetration testing where appropriate. During operation, monitor events, patch vulnerabilities, rotate credentials, test backups, and rehearse response.

Threat modelling is not a prediction of every attack. It is a structured way to expose assumptions. A useful workshop asks:

1. What are the valuable assets?
2. Where does data enter, move, and leave the system?
3. Who can interact with each component?
4. What could an attacker or careless user do?
5. Which controls prevent, detect, or limit the harm?
6. What evidence will show that controls work?

Security testing has limitations. A clean test report does not prove the absence of vulnerabilities. Tests are time-bound, scope-dependent, and influenced by tester skill. Risk owners must understand residual risk rather than treating a certificate or scan result as a guarantee.

3. Privacy and Responsible Data Governance

Privacy concerns people’s control, rights, expectations, and interests in relation to information about them. Personal data may include obvious identifiers, such as names and identification numbers, but also location, device identifiers, behavioural profiles, inferred attributes, and combinations of data that enable identification.

Privacy governance commonly rests on principles such as lawfulness and fairness, transparency, purpose limitation, data minimization, accuracy, storage limitation, confidentiality, and accountability. Exact legal duties vary by jurisdiction and sector; project teams must obtain appropriate legal advice rather than assuming that one privacy framework applies everywhere.

Privacy by design

Privacy by design means that privacy controls are built into architecture and process rather than added after deployment. Practical measures include:

• collecting only data necessary for a stated purpose;
• separating identifying data from operational data where feasible;
• using pseudonymization or anonymization carefully, recognizing that re-identification may remain possible;
• encrypting data in transit and at rest;
• limiting access and retaining access logs;
• defining retention and secure deletion rules;
• providing understandable notices and meaningful choices;
• conducting privacy impact assessments for high-risk processing;
• managing international transfers and processors contractually and operationally.

A data inventory should record what data is collected, its source, purpose, sensitivity, location, recipients, retention period, lawful basis where applicable, and owner. It should include derived data and machine-generated inferences, not only user-entered fields.

Privacy and security are related but distinct. Strong encryption may protect data from attackers, yet a system can still violate privacy by collecting excessive information or using it for an undisclosed purpose. Conversely, a transparent and limited data practice may still be insecure if access controls are weak.

4. Accessibility, Safety and Human Impact

Accessibility

Accessibility is the degree to which people with diverse abilities can perceive, understand, navigate, and operate a product. It is both an inclusion obligation and a quality characteristic. Barriers may involve vision, hearing, mobility, cognition, speech, literacy, language, or temporary impairment.

Requirements should address keyboard operation, focus order, text alternatives, colour contrast, captions, error identification, scalable text, clear language, predictable navigation, and compatibility with assistive technologies. Automated tools can identify some problems but cannot reliably assess all aspects of usability or meaningful accessibility. Testing with disabled users and specialist review remain important.

Accessibility should be included in procurement, design systems, acceptance criteria, and definition of done. Retrofitting it late is usually more expensive and may require architectural changes.

Safety

Safety is concerned with preventing unacceptable harm. It is especially important in medical, industrial, transport, infrastructure, financial, and public-sector systems. Safety analysis considers hazards, causes, severity, likelihood, detection, mitigation, and safe states.

A safety requirement might state that a system must enter a controlled state when sensor data is invalid, rather than continuing with a misleading value. Safety cases should provide a structured argument supported by evidence: identified hazards, mitigations, tests, operational assumptions, and residual-risk acceptance.

Security and safety can conflict. A security control such as automatic account lockout might prevent misuse but could also obstruct emergency access. Conversely, a safety override could create a security weakness. Such conflicts require explicit analysis, authority, logging, and compensating controls.

5. Ethics and Responsible Technology

Law establishes minimum enforceable requirements; ethical responsibility often extends further. Ethical analysis asks whether a practice is fair, transparent, proportionate, explainable, respectful of autonomy, and consistent with professional responsibility.

Common ethical risks include discriminatory outcomes, manipulative design, hidden surveillance, exploitative pricing, opaque automated decisions, unsafe release pressure, conflicts of interest, and unequal distribution of benefits and harms.

A practical ethics review can examine:

• affected stakeholders, including people who are not customers;
• foreseeable benefits and harms;
• groups that may be disproportionately affected;
• whether consent or notice is meaningful;
• whether users can challenge or appeal important decisions;
• whether the system preserves human control;
• whether business incentives encourage harmful behaviour;
• whether the team would defend the practice publicly and to an affected person.

Artificial intelligence

AI systems add risks involving data provenance, model drift, bias, explainability, automation bias, prompt or input manipulation, and uncertain outputs. Governance should define approved uses, prohibited uses, human oversight, evaluation data, performance boundaries, monitoring, incident response, and documentation. Generative AI outputs should not be treated as authoritative without verification, particularly in security, legal, medical, employment, or safety contexts.

Responsible use also requires protecting confidential information entered into AI tools, checking intellectual-property and licensing conditions, and recording material human decisions. AI can assist quality activities, but accountability remains with the organization and authorized decision makers.

6. Compliance, Auditability and Evidence

Compliance means meeting applicable legal, regulatory, contractual, policy, and standards-based obligations. A standard or framework may provide useful controls, but adopting it does not automatically prove compliance with every law. The project must identify which obligations apply and translate them into actionable requirements.

A compliance obligation should have an owner, control, evidence source, review frequency, and escalation path. Examples of evidence include approved requirements, design decisions, training records, access reviews, test results, vulnerability remediation, incident records, supplier assessments, change approvals, and deletion logs.

Auditability

Auditability is the ability to reconstruct what happened, who acted, which decision was made, what evidence supported it, and whether controls operated as intended. Good audit trails are:

• attributable to an authenticated identity;
• time-stamped using a reliable time source;
• protected against unauthorized alteration;
• sufficient to explain significant events;
• retained for a justified period;
• accessible to authorized reviewers;
• minimized so that logging does not create unnecessary privacy risk.

More logging is not automatically better. Sensitive values should generally not be placed in logs unless necessary and protected. Logs should be monitored, tested, and connected to incident response. A control that is documented but never tested provides weak assurance.

Internal audit and independent assurance serve different purposes. Internal reviews improve management control; independent assessments increase confidence for stakeholders. Neither eliminates the need for operational ownership.

7. Responsible Reporting and Incident Response

Responsible reporting applies to vulnerabilities, privacy incidents, safety events, ethical concerns, and control failures. A culture that punishes good-faith reporting encourages concealment and increases harm.

An effective reporting process provides:

1. a clearly published reporting channel;
2. acknowledgement and triage criteria;
3. confidentiality and protection against retaliation where appropriate;
4. severity assessment and assignment of an owner;
5. containment, investigation, remediation, and verification;
6. legal, regulatory, customer, and stakeholder notifications when required;
7. lessons learned and control improvement.

External vulnerability disclosure should be coordinated with affected suppliers, maintainers, and customers. Disclosure timing must balance user protection, remediation readiness, public interest, and the risk of enabling exploitation. Reports should be factual, reproducible, proportionate, and free from unnecessary exposure of personal information.

Incident response is not merely a technical activity. Communications, legal duties, customer support, business continuity, executive decisions, and evidence preservation may all be involved. Post-incident reviews should examine systemic causes rather than simply blaming an individual.

8. Supplier and Third-Party Risk

Modern software depends on cloud providers, payment services, open-source libraries, consultants, data processors, identity providers, analytics tools, and software supply chains. Third-party risk arises when the organization’s security, privacy, availability, or ethical outcomes depend on another party’s actions.

Supplier management should be risk-based. Due diligence may examine ownership, financial and operational resilience, security controls, privacy practices, incident history, subcontractors, geographical processing, accessibility, business continuity, and recovery capability. The depth of review should reflect the sensitivity and criticality of the service.

Contracts should address:

• permitted data use and processing;
• confidentiality and security requirements;
• access control and personnel assurance;
• vulnerability and incident notification;
• audit or assurance rights;
• subcontractor approval;
• service levels and recovery objectives;
• data return and secure deletion;
• intellectual property and licensing;
• exit assistance and portability.

Open-source software is not automatically unsafe or safe. Teams should maintain a software bill of materials where practical, monitor vulnerabilities, assess project maintenance, verify package provenance, pin or control versions, and remove unnecessary dependencies. A dependency with no maintained version or unclear origin may represent material supply-chain risk.

Third-party assurance reports are useful but limited. They describe a scope, period, and control set; they do not prove that every use case is safe. The organization remains responsible for configuring services correctly and managing risks outside the supplier’s control.

9. A Governance Operating Model

A practical operating model can be organized into five connected activities:

Governance metrics should support decisions rather than reward superficial compliance. Useful indicators may include overdue critical vulnerabilities, privileged-access review completion, time to contain incidents, unresolved accessibility defects, supplier assessment coverage, backup restoration success, privacy requests completed within required periods, and recurrence of control failures. Metrics need definitions, owners, data quality checks, and context.

A project should define escalation thresholds. For example, a critical vulnerability in an internet-facing component, an unexplained discriminatory outcome, or an inability to restore essential data may require release suspension. Thresholds should be established before commercial pressure makes decisions difficult.

10. Illustrative Case: The CareLink Remote-Monitoring Platform

This is a fictional illustrative case.

Background

CareLink is a remote-monitoring platform used by clinics to receive patient measurements from connected devices. A supplier provides cloud hosting, another supplies the mobile application framework, and an analytics service generates alerts for clinicians.

Problem

The project is behind schedule. A review identifies four concerns: the mobile application is difficult to use with screen readers; the analytics service retains raw measurements indefinitely; a default administrative account remains enabled in a test environment connected to production services; and the supplier contract does not require rapid notification of security incidents. The product team proposes release because clinical partners are expecting a pilot.

Analysis

The accessibility problem can prevent some patients from using the service and may create unequal access. Indefinite retention conflicts with data minimization and increases breach consequences. The administrative account creates a high-severity security risk. Weak notification terms delay response and may undermine regulatory duties. The fact that the pilot is limited does not remove these risks: health information is sensitive, and a pilot can establish operational habits that persist.

The project manager convenes product, engineering, security, privacy, accessibility, clinical safety, procurement, and legal representatives. The team classifies the administrative account as a release blocker, defines a safe configuration, and requires independent verification. It maps data flows, sets a justified retention period, obtains privacy approval, and documents deletion testing. Accessibility defects are prioritized according to user impact, with critical barriers corrected before release and a funded remediation plan for remaining issues. The supplier contract is amended to include notification, cooperation, deletion, access, and subcontractor provisions. The analytics model is restricted to decision support; clinicians remain responsible for reviewing alerts.

Outcome

The pilot is delayed by two weeks. The organization avoids releasing a known critical weakness, obtains evidence of secure configuration and deletion, improves accessibility, and establishes a response route with suppliers. The delay has a visible cost, but it is lower than the potential cost of patient harm, regulatory action, loss of trust, or emergency remediation.

Learning points

1. A schedule commitment does not override a material safety or security risk.
2. Privacy, accessibility, supplier controls, and clinical responsibility must be considered together.
3. Release decisions require evidence, not only assurances.
4. Residual risk must be accepted by an authorized person who understands its consequences.
5. Governance is effective when it changes decisions and behaviour, not merely when policies exist.

11. Challenges and Limitations

Security and governance programs face recurring difficulties. Requirements may conflict with usability, delivery speed, cost, or accessibility. Risk ratings can be subjective and may understate low-probability catastrophic events. Compliance can become a paperwork exercise if evidence is collected without testing control effectiveness. Small organizations may lack specialists and must prioritize proportionately, use trusted external expertise, and avoid claiming assurance they cannot support.

Privacy controls can themselves affect functionality; excessive authentication may exclude users, while excessive data minimization may reduce legitimate service quality. Automated tools produce false positives and false negatives. Third-party assurances may be outdated or narrowly scoped. Ethical review can be influenced by organizational power and may overlook people who are not represented in meetings. These limitations require diverse participation, independent challenge, periodic reassessment, and transparent documentation of assumptions.

12. Best-Practice Checklist

Before release, the project manager should be able to answer yes to the following questions:

• Are security, privacy, accessibility, safety, and ethical requirements explicit and testable?
• Have assets, data flows, threats, hazards, and affected stakeholders been identified?
• Are high-risk controls implemented and independently checked?
• Are access, logging, retention, deletion, backup, and recovery arrangements tested?
• Are suppliers, dependencies, subcontractors, and exit arrangements understood?
• Is there a vulnerability, incident, and responsible-reporting process?
• Are unresolved risks documented with an accountable and authorized owner?
• Can the organization produce reliable evidence of important decisions and controls?
• Are monitoring and post-release improvement activities funded?

Chapter Summary

Security, privacy, accessibility, safety, ethics, compliance, and auditability are strategic dimensions of software quality. Governance connects risks to accountable owners, controls, evidence, decisions, and improvement. Secure development begins with requirements and architecture and continues through operation. Privacy requires purposeful collection, minimization, transparency, retention control, and accountability. Accessibility and safety address the real-world effects of software on diverse users and affected communities. Ethical governance extends beyond legal compliance to fairness, autonomy, explainability, and prevention of avoidable harm.

Auditability depends on trustworthy, proportionate evidence. Responsible reporting protects users and improves systems by enabling timely disclosure and learning. Supplier and third-party risk must be managed through due diligence, contracts, monitoring, dependency control, and exit planning. Effective project managers do not treat governance as a final approval gate; they embed it in everyday planning, design, delivery, release, and operations.

Key Terms

Security: Protection of systems, services, and information from unauthorized actions or harm.

Privacy: Fair and responsible governance of information relating to identifiable or identifiable people.

Safety: Prevention of unacceptable harm caused by system behaviour or failure.

Accessibility: Ability of people with diverse abilities to use a product effectively.

Ethics: Principles concerning responsible, fair, and appropriate conduct.

Governance: Direction, accountability, control, and oversight of an organization or project.

Least privilege: Limiting access to only what is necessary for an authorized task.

Defense in depth: Use of multiple complementary controls.

Threat modelling: Structured analysis of assets, threats, trust boundaries, and mitigations.

Data minimization: Collecting and retaining no more personal data than necessary.

Purpose limitation: Using data only for specified and legitimate purposes.

Privacy impact assessment: Structured evaluation of privacy risks and mitigations.

Auditability: Ability to reconstruct actions, decisions, controls, and evidence.

Residual risk: Risk remaining after controls are applied.

Vulnerability disclosure: Reporting and coordinating information about a security weakness.

Supplier risk: Risk arising from dependence on external providers or components.

Software bill of materials: An inventory of software components and dependencies.

Separation of duties: Distribution of critical responsibilities to reduce unchecked action.

Review Questions

Short Answer Questions

1. Distinguish security, privacy, safety, and ethics.
2. What is the purpose of threat modelling?
3. State three principles of privacy by design.
4. Why is auditability important in software governance?
5. Identify four areas that should be addressed in supplier contracts.

Long Answer Questions

1. Explain how security should be integrated across the software development life cycle.
2. Discuss why compliance does not necessarily demonstrate ethical or high-quality software.
3. Describe a governance operating model linking risks, controls, evidence, and improvement.
4. Explain the relationship between accessibility, safety, and software quality.
5. Evaluate the limitations of third-party assurance reports and propose complementary controls.

Analytical / Application Questions

1. A project wants to release a service with a known high-severity authentication weakness because the pilot has only 100 users. Analyse the decision using risk ownership, evidence, and release governance.
2. A supplier processes customer data in a new country through an undisclosed subcontractor. Identify the privacy, contractual, security, and governance actions required.
3. An AI-based eligibility tool produces lower approval rates for a demographic group. Design an investigation and responsible-reporting process, including technical, ethical, and managerial controls.

Multiple Choice Questions with Answers

Question 1

Which statement best describes governance in software quality and risk management?

A. A final inspection performed only by an external auditor

B. A system of authority, accountability, controls, evidence, and oversight

C. A method for increasing software execution speed

D. A replacement for software engineering practices

Correct Answer: B. A system of authority, accountability, controls, evidence, and oversight

Question 2

What does the principle of least privilege require?

A. Every user receives administrator access during development

B. Access is granted only to the data and actions necessary for an authorized task

C. Access rights are never reviewed after assignment

D. All systems use one shared service account

Correct Answer: B. Access is granted only to the data and actions necessary for an authorized task

Question 3

Which practice most directly supports data minimization?

A. Collecting all potentially useful data indefinitely

B. Sharing raw data with every supplier

C. Collecting and retaining only data necessary for a stated purpose

D. Disabling deletion to preserve historical records

Correct Answer: C. Collecting and retaining only data necessary for a stated purpose

Question 4

Why is a clean penetration-test report not proof that a system is secure?

A. Penetration tests are always legally invalid

B. Testing is limited by scope, time, methods, and tester expertise

C. Penetration testing examines only accessibility

D. Security cannot be tested under any circumstances

Correct Answer: B. Testing is limited by scope, time, methods, and tester expertise

Question 5

Which item is an example of audit evidence?

A. An unsupported claim that controls are effective

B. A verified access-review record showing accountable approvals

C. A marketing slogan about privacy

D. An undocumented verbal promise from a supplier

Correct Answer: B. A verified access-review record showing accountable approvals

Question 6

What is a principal purpose of accessibility testing with disabled users?

A. To replace all automated testing

B. To identify real barriers that automated tools may not detect

C. To reduce the need for user requirements

D. To test only system throughput

Correct Answer: B. To identify real barriers that automated tools may not detect

Question 7

Which contractual provision is especially relevant to third-party incident management?

A. A ban on all software updates

B. A requirement for timely incident notification and cooperation

C. An unlimited right to retain customer data

D. A promise that no audit evidence will be supplied

Correct Answer: B. A requirement for timely incident notification and cooperation

Question 8

What is the main purpose of responsible vulnerability disclosure?

A. To maximize publicity before users can be protected

B. To coordinate communication and remediation while reducing avoidable harm

C. To conceal every vulnerability permanently

D. To transfer all organizational accountability to the reporter

Correct Answer: B. To coordinate communication and remediation while reducing avoidable harm

Question 9

Which statement about compliance is most accurate?

A. Compliance with one standard automatically satisfies every law

B. Compliance removes all residual risk

C. Applicable obligations must be identified and translated into owned, evidenced controls

D. Compliance is relevant only after a product is released

Correct Answer: C. Applicable obligations must be identified and translated into owned, evidenced controls

Question 10

Why should an organization maintain an inventory of software dependencies?

A. To make all dependencies permanent

B. To identify components, assess supply-chain exposure, and manage vulnerabilities

C. To eliminate the need for secure coding

D. To ensure that every dependency is open source

Correct Answer: B. To identify components, assess supply-chain exposure, and manage vulnerabilities