Security, Privacy, Ethics and Governance by Design

Introduction

Modern information systems do not merely process transactions; they influence opportunities, rights, safety, reputation and institutional decision-making. A student-management system determines access to educational records, a health application handles sensitive personal information, and an artificial-intelligence service may recommend or deny services. Consequently, security, privacy, ethics and governance cannot be added after programming is complete. They must shape requirements, architecture, interfaces, data models, testing, deployment and retirement.

This chapter presents a systems-analysis perspective on responsible design. It integrates the security objectives of confidentiality, integrity and availability; identity and access management; preventive and detective controls; privacy engineering; compliance and auditability; accessibility; responsible data use; and professional ethics. The central principle is governance by design: organizational values, legal duties, risk controls and accountability are translated into explicit system requirements and verifiable design decisions.

Learning Objectives

After studying this chapter, students should be able to:

1. Explain confidentiality, integrity, availability, authenticity, accountability and non-repudiation.
2. Identify security, privacy, ethical and governance requirements during systems analysis.
3. Apply threat modelling, risk assessment and control selection to a proposed system.
4. Distinguish authentication, authorization, auditing and identity lifecycle management.
5. Design privacy-preserving data flows and explain data-subject rights.
6. Evaluate accessibility, fairness, transparency and responsible use of automation.
7. Develop an illustrative governance and assurance plan for a system.

1. Security and Governance as Design Concerns

A traditional development sequence treats security as a final testing activity. This approach is inadequate because late security changes are expensive and may conflict with the data model or business process. For example, if an application stores passwords in recoverable form, no later interface improvement can make that design safe. Similarly, if a business collects a national identifier without a defined purpose, deleting the field later may not undo the privacy harm.

A responsible system begins by identifying stakeholders and their interests. Stakeholders include users, administrators, customers, employees, regulators, suppliers, people represented in data, and members of the public who may be indirectly affected. Analysts should ask:

• What could go wrong, and who would be harmed?
• Which information requires protection, and from whom?
• What decisions may be made automatically or semi-automatically?
• What evidence must the organization retain to demonstrate compliance?
• How can a person challenge, correct or appeal an outcome?
• Can people with different abilities, devices, languages and levels of digital access use the system?

Governance is the allocation of authority, responsibility and oversight. It specifies who may approve data uses, who owns risks, who operates controls, how exceptions are handled and how performance is reviewed. Governance is therefore broader than information security. Security protects systems and information; governance establishes why information is collected, who is accountable and whether the use is legitimate and socially acceptable.

2. Core Security Objectives

Confidentiality, integrity and availability

The classical security triad is:

• Confidentiality: information is disclosed only to authorized parties.
• Integrity: information remains accurate, complete and protected from unauthorized alteration.
• Availability: authorized users can access systems and information when required.

These objectives can conflict. Strong authentication may improve confidentiality but create barriers to availability. Extensive logging may support integrity and accountability but increase privacy risk. Systems analysis therefore involves balancing controls against business needs, threat exposure, cost and human consequences.

Additional properties are important:

• Authenticity establishes that an entity, message or document is genuine.
• Accountability connects actions to responsible identities and organizational roles.
• Non-repudiation provides evidence that a party performed an action, although technical evidence does not eliminate all legal or social disputes.
• Resilience enables a system to continue or recover after disruption.

Threats, vulnerabilities and risks

A threat is a potential cause of harm, such as phishing, malware, insider misuse, equipment failure or a dishonest data broker. A vulnerability is a weakness that a threat can exploit. Risk is the possibility and consequence of harm arising from that relationship.

A commonly used qualitative or semi-quantitative model is:

Likelihood and impact may be expressed as ordinal scores, such as 1 to 5, rather than precise probabilities. The multiplication is a prioritization aid, not a scientifically exact prediction. If likelihood is scored 4 and impact 5, the risk score is 20; the organization must still explain what the scores mean and why the result warrants treatment. For probabilistic analysis, expected loss may be represented as:

Probability is between 0 and 1, and loss must use a consistent unit, such as currency. These models do not capture every factor, including uncertainty, cascading effects, dignity or irreversible harm. Qualitative judgment remains necessary.

3. Security Requirements and Threat Modelling

Security requirements should be written as testable statements. “The system must be secure” is not testable. Better requirements include:

• The system shall require multi-factor authentication for privileged administrators.
• The system shall deny access by default when no applicable authorization rule exists.
• The system shall record the identity, time, resource, action and result of administrative changes.
• The system shall encrypt sensitive data in transit and at rest using organization-approved mechanisms.
• The system shall restore critical service within the approved recovery objective after a defined failure.

Threat modelling is a structured method for anticipating misuse. A team can map assets, trust boundaries, entry points, actors, threats and mitigations. A data-flow diagram should show where data enters, where it is processed, where it is stored and with whom it is shared. The analyst then examines threats such as spoofing identity, tampering with records, repudiating actions, exposing information, denying service and escalating privileges.

Threat modelling is iterative. It should be revisited when a new integration, mobile interface, cloud service, algorithm or business process is introduced. It is also important to model abuse by legitimate users. An employee may have valid access to a customer system but misuse it to browse records unrelated to assigned work.

4. Identity and Access Management

Identity and access management (IAM) governs digital identities throughout their lifecycle. It includes identity proofing, account creation, authentication, authorization, privilege review, suspension and deletion.

Identification claims an identity, such as a username. Authentication verifies the claim, using something the user knows, has or is. A password is knowledge; a hardware token is possession; a biometric characteristic is an inherent attribute. Multi-factor authentication combines independent factor types. Two passwords are not two independent factors.

Authorization determines what an authenticated identity may do. The principle of least privilege grants only the access necessary for an approved task. Separation of duties divides sensitive activities so that no single person can complete an entire high-risk process without review. Role-based access control assigns permissions to roles, while attribute-based access control may use attributes such as department, location, time or case assignment.

An authorization decision can be expressed conceptually as:

This is a policy expression rather than a numerical equation. It assumes that attributes are trustworthy, current and evaluated consistently. An access-control design should also address emergency access, delegation, failed login attempts, session expiry, service accounts, privileged accounts and access recertification.

The joiner–mover–leaver process is essential. New personnel receive approved access; role changes trigger reassessment; departing personnel are promptly disabled. Dormant accounts, shared accounts and excessive administrator privileges are common sources of avoidable risk.

5. Controls, Assurance and Auditability

Controls are measures that prevent, detect, correct or deter undesirable events. A useful classification is:

Controls may also be administrative, technical or physical. A firewall is technical; a background-check policy is administrative; restricted server-room access is physical. Effective assurance uses more than one control layer, often called defense in depth.

Auditability means that significant events can be reconstructed and evaluated. A useful audit record may contain actor, timestamp, action, object, source, result and reason. Logs should be protected from alteration, synchronized to a reliable time source, retained only as long as justified, and made accessible to authorized investigators. Logging everything forever is not automatically good governance: it increases storage, exposure and privacy risk.

Audit trails should support accountability without becoming surveillance by default. Access to logs must itself be controlled and reviewed. Systems should distinguish business records from security logs and document retention, legal holds and secure disposal.

6. Privacy by Design and Responsible Data Use

Privacy concerns the appropriate collection, use, disclosure, retention and disposal of information about identifiable or reasonably identifiable people. Privacy is not merely secrecy. It includes autonomy, freedom from unjustified surveillance, contextual integrity and the ability to exercise rights.

Privacy by design commonly involves the following practices:

1. Purpose specification: state why data is needed before collecting it.
2. Data minimization: collect only what is necessary for the stated purpose.
3. Accuracy: provide ways to correct material errors.
4. Storage limitation: define retention periods and deletion or anonymization rules.
5. Transparency: explain processing in understandable language.
6. User control: support appropriate consent, access, correction, objection or appeal.
7. Protection: apply security controls proportionate to sensitivity and risk.
8. Accountability: document decisions, responsibilities and evidence.

Consent is not a universal solution. It may be invalid when people lack a genuine choice, when notices are incomprehensible or when the organization has another legal basis for processing. Analysts must distinguish required data from optional data and avoid “dark patterns” that manipulate people into accepting unnecessary collection.

Data classification helps determine protection requirements. Public information, internal information, confidential information and highly sensitive information should not be treated identically. De-identification can reduce risk but is not automatically irreversible; combinations of apparently harmless fields may re-identify individuals. Aggregation, tokenization, masking and differential privacy each have limitations and should be selected according to the threat model and intended use.

Responsible data use also concerns secondary use. Data collected to provide education should not automatically be repurposed for employee surveillance or commercial profiling. Data-sharing agreements should specify purpose, fields, security, onward disclosure, retention, breach notification and deletion.

7. Compliance, Governance and Professional Ethics

Legal and regulatory requirements vary by jurisdiction and sector. Examples of concerns include data-protection principles, records-management obligations, consumer protection, financial controls, health confidentiality, intellectual-property rights, accessibility law and breach notification. A system analyst should not assume that a technical control alone establishes compliance. Compliance requires interpretation, documented accountability, organizational procedures and evidence.

A compliance matrix links obligations to requirements, controls, owners, tests and evidence:

Professional ethics extends beyond minimum legal compliance. Codes of conduct in computing commonly emphasize avoiding harm, respecting privacy, being honest about system limitations, maintaining competence, and treating people fairly. An analyst should raise concerns when a proposed feature is deceptive, discriminatory, unsafe or outside the organization’s authority. Ethical escalation may involve a product owner, security officer, privacy officer, ethics committee, regulator or professional body.

Conflicts are unavoidable. A manager may demand rapid deployment, while a risk assessment identifies unacceptable exposure. Ethical practice requires evidence-based communication, documented risk acceptance and, where necessary, refusal to implement harmful conduct. “The client asked for it” is not a sufficient professional justification.

8. Accessibility, Fairness and Human-Centred Design

Accessibility means that people with disabilities can perceive, understand, navigate and operate the system. Requirements should address keyboard operation, sufficient contrast, meaningful labels, captions, alternative text, predictable navigation, error messages, screen-reader compatibility and accessible authentication. Accessibility should be tested with users and assistive technologies rather than inferred from a checklist alone.

Inclusive design also considers older users, low-bandwidth environments, different languages, limited digital literacy and temporary impairments. A security mechanism that cannot be used by a legitimate population may create exclusion or unsafe workarounds.

Fairness is especially important when systems rank, classify, recommend or decide. Analysts should identify affected groups, examine data quality and representativeness, test error patterns, document limitations and provide human review where stakes are high. Fairness is not a single universal metric; improving one group-level measure may worsen another when base rates or decision contexts differ. The appropriate approach depends on the domain, harms, legal requirements and stakeholder values.

Explainability should be proportionate to impact. A person denied a benefit needs more than a generic statement that “the algorithm decided.” The system should provide understandable reasons, relevant evidence, a correction route and an appeal process. Human oversight must be meaningful rather than a nominal approval step.

9. Security and Privacy in the System Life Cycle

During initiation, identify stakeholders, system purpose, business impact, legal context and unacceptable harms. During requirements analysis, document security, privacy, accessibility, retention, audit and recovery requirements. During design, define trust boundaries, data flows, IAM, encryption, control ownership and failure behavior. During implementation, use secure coding, dependency management, secrets protection, peer review and controlled configuration.

During testing, conduct functional, security, privacy, accessibility, resilience and usability tests. Testing should include misuse cases and negative cases, such as unauthorized access, malformed input, excessive requests, lost devices and incomplete transactions. During deployment, use change control, secure defaults, monitoring, backup verification and staff training. During operations, patch vulnerabilities, review privileges, investigate alerts, conduct impact assessments and rehearse incident response. During retirement, archive only what is justified, revoke access, securely dispose of media and notify stakeholders where required.

A system should be designed for failure. Backups should be protected from the same incident as production data, restoration should be tested, and recovery priorities should be explicit. Availability planning may use recovery time objective (RTO), the maximum acceptable time to restore service, and recovery point objective (RPO), the maximum acceptable period of data that may be lost measured backward from the disruption. These are management requirements, not guarantees.

10. Illustrative Case: University Student Support Platform

Background

A fictional university proposes a platform combining enrollment records, attendance information, disability-support documentation, financial-aid status and an AI model that identifies students who may need academic assistance.

Problem

The proposal promises earlier support but creates risks. Sensitive records may be visible to staff without a legitimate need. Students may not understand how predictions are made. An inaccurate risk score could stigmatize a student or influence decisions about progression. The platform also needs to serve students using screen readers and mobile connections.

Analysis

The analysis team maps data flows and classifies disability and financial information as highly sensitive. It identifies threats including account takeover, excessive staff access, re-identification from reports, model bias, unauthorized secondary use and insecure data exports. Stakeholder interviews reveal that students want understandable notices, correction mechanisms and a way to speak with a person.

Approach and solution

The team adopts purpose limitation: the model may recommend an invitation to voluntary support but may not automatically deny enrollment, funding or progression. Access is role-based and case-based; counselors see assigned cases, while instructors see only information necessary for teaching. Multi-factor authentication is required for staff with sensitive access. Every sensitive view is logged, and unusual access is reviewed.

The platform displays a plain-language privacy notice, collects only required fields, separates disability documentation from routine academic data, and establishes retention periods. Students can review key data, request correction and ask for human reconsideration. Model performance is evaluated across relevant groups, with documented limitations and a human review process. Accessibility testing involves keyboard-only operation and screen-reader users. A governance committee approves new data uses and reviews incidents and model performance.

Outcome

The university delays full deployment until the controls and appeal process are tested. A pilot is limited to supportive recommendations, and staff are trained not to treat a prediction as a fact. The resulting system provides earlier outreach while reducing the risk that confidential information becomes a hidden disciplinary or admissions mechanism.

Learning points

The case demonstrates that responsible design changes scope, data architecture, access rules, interface language, testing and governance. Privacy and ethics are not documents added after development; they influence the system’s purpose and permitted decisions.

11. Contemporary Developments and Practical Best Practices

Cloud services, mobile devices, connected sensors and generative AI expand both capability and attack surface. Organizations should establish supplier assurance, shared-responsibility boundaries, secure application programming interfaces, data-location awareness and exit plans. AI systems require particular attention to training-data provenance, prompt and output leakage, model drift, automation bias, adversarial manipulation and confidential information being submitted to external services.

A practical design review should ask:

• Is the purpose legitimate, specific and understandable?
• Are data fields necessary, accurate and appropriately classified?
• Are access decisions least-privilege, reviewed and logged?
• Can the system resist, detect and recover from credible threats?
• Can affected people see, correct and challenge important outcomes?
• Are accessibility and inclusion tested with representative users?
• Are controls mapped to accountable owners and evidence?
• Are retention, deletion, suppliers and incident responsibilities explicit?
• What happens when the system, model, network or identity provider fails?

No control is perfect. Encryption does not prevent an authorized person from misusing decrypted information; consent does not justify harmful design; audit logs do not guarantee honest behavior; and an AI explanation does not prove fairness. Sound analysis recognizes residual risk, communicates uncertainty and ensures that someone with authority accepts or treats the risk.

Chapter Summary

Security, privacy, ethics and governance are foundational system requirements. Confidentiality, integrity and availability must be considered together with authenticity, accountability, resilience and human consequences. Threat modelling connects assets and vulnerabilities to proportionate controls, while IAM manages identities, authentication, authorization and the joiner–mover–leaver lifecycle. Preventive, detective, corrective, directive and compensating controls provide layered assurance, and auditability creates evidence without encouraging unlimited surveillance.

Privacy by design requires purpose limitation, minimization, accuracy, transparency, appropriate user control, retention discipline and accountability. Compliance matrices translate obligations into requirements, controls, owners and evidence. Professional ethics requires analysts to prevent avoidable harm, communicate limitations and challenge deceptive, discriminatory or unsafe proposals. Accessibility, fairness and meaningful human review are essential when systems affect opportunities or rights. Finally, these concerns must be integrated throughout the life cycle, including retirement, because responsible systems are governed systems.

Key Terms

Confidentiality: Protection against unauthorized disclosure.

Integrity: Protection against unauthorized or accidental alteration.

Availability: Accessibility of systems and information when authorized users need them.

Authenticity: Confidence that an entity, message or record is genuine.

Accountability: Ability to associate actions with responsible identities and processes.

Threat: A potential cause of harm.

Vulnerability: A weakness that a threat may exploit.

Risk: The possibility and consequence of harm.

Threat modelling: Structured analysis of assets, data flows, trust boundaries and threats.

Authentication: Verification of a claimed identity.

Authorization: Determination of permitted actions after authentication.

Least privilege: Granting only the access necessary for an approved task.

Auditability: Ability to reconstruct and evaluate significant events using evidence.

Privacy by design: Integrating privacy protection into system design and operation.

Data minimization: Collecting and retaining only data necessary for a defined purpose.

De-identification: Reducing the association between data and identifiable individuals.

Accessibility: Design that enables people with diverse abilities to use a system.

Purpose limitation: Restricting data use to specified, legitimate purposes.

Separation of duties: Dividing sensitive responsibilities among different people or roles.

Recovery point objective: Maximum acceptable period of data loss measured backward from disruption.

Review Questions

Short Answer Questions

1. Define confidentiality, integrity and availability, and give one example of a control supporting each.
2. Distinguish authentication from authorization.
3. What is the purpose of a compliance matrix?
4. Explain data minimization and purpose limitation.
5. Why should audit logs be protected and access to them restricted?

Long Answer Questions

1. Explain how security and privacy requirements should be incorporated across the system development life cycle.
2. Discuss the advantages and limitations of role-based and attribute-based access control.
3. Explain how professional ethics extends beyond legal compliance in systems analysis.
4. Describe a threat-modelling process for a cloud-based student or banking application.
5. Discuss how accessibility, fairness and explainability affect systems that use automated recommendations.

Analytical / Application Questions

1. A hospital wants all employees to have access to a searchable patient database “for efficiency.” Analyze the proposal using least privilege, purpose limitation, auditability and emergency access principles. Recommend a revised design.
2. A university AI model predicts which students may drop out. Design a governance plan covering data sources, access, validation, transparency, human review, appeals, retention and monitoring for bias.
3. An online service suffers a ransomware attack and restores from a backup that is six days old. Analyze the incident using availability, RTO, RPO, defense in depth and recovery governance. Recommend improvements.

Multiple Choice Questions with Answers

Question 1

Which security objective concerns preventing unauthorized disclosure of information?

A. Availability

B. Confidentiality

C. Integrity

D. Resilience

Correct Answer: B. Confidentiality

Question 2

What does authentication primarily establish?

A. What actions a user may perform

B. Whether a claimed identity is genuine

C. How long data should be retained

D. Whether a system is accessible

Correct Answer: B. Whether a claimed identity is genuine

Question 3

Which principle grants a user only the permissions necessary for an approved task?

A. Least privilege

B. Non-repudiation

C. Data aggregation

D. Open access

Correct Answer: A. Least privilege

Question 4

A control that records and alerts on unusual access is mainly what type of control?

A. Preventive

B. Detective

C. Directive

D. Physical

Correct Answer: B. Detective

Question 5

What is the primary purpose of data minimization?

A. To collect as many fields as possible for future use

B. To retain all information indefinitely

C. To collect only data necessary for a defined purpose

D. To eliminate the need for access controls

Correct Answer: C. To collect only data necessary for a defined purpose

Question 6

Which statement about consent is most accurate?

A. Consent always makes any data use ethical

B. Consent is valid even when no genuine choice exists

C. Consent is one possible basis for processing but does not replace responsible design

D. Consent eliminates the need for transparency

Correct Answer: C. Consent is one possible basis for processing but does not replace responsible design

Question 7

What does auditability enable an organization to do?

A. Guarantee that no misuse will occur

B. Reconstruct and evaluate significant events using evidence

C. Permit all employees to inspect security logs

D. Avoid defining control ownership

Correct Answer: B. Reconstruct and evaluate significant events using evidence

Question 8

A recovery point objective specifies:

A. The maximum acceptable time to restore service

B. The maximum acceptable period of data that may be lost

C. The number of administrators required

D. The minimum password length

Correct Answer: B. The maximum acceptable period of data that may be lost

Question 9

Which design choice best supports responsible use of a high-impact AI recommendation?

A. Automatically enforce every prediction

B. Hide the model’s limitations from users

C. Provide understandable reasons and a meaningful human appeal route

D. Remove all audit records to protect privacy

Correct Answer: C. Provide understandable reasons and a meaningful human appeal route

Question 10

Why is accessibility a systems-analysis concern?

A. It applies only after deployment

B. It ensures people with diverse abilities can perceive and operate the system

C. It replaces security testing

D. It requires every user to have identical hardware

Correct Answer: B. It ensures people with diverse abilities can perceive and operate the system