Ethical Theories and Moral Reasoning

Introduction

Artificial intelligence systems increasingly influence decisions about employment, education, credit, healthcare, policing, insurance, public benefits, and access to information. These systems do not merely produce technical outputs. They distribute opportunities, burdens, risks, and recognition among people. Consequently, responsible AI requires more than accuracy, security, or compliance with law. It requires reasoned judgments about what ought to be done.

Ethical theories provide structured ways to make such judgments. Consequentialism asks whether an action produces the best overall results. Deontology focuses on duties, rules, and respect for persons. Virtue ethics examines the character and practical wisdom of decision-makers. Rights-based ethics protects fundamental claims such as privacy, equality, and freedom of expression. Justice concerns fair distribution and fair procedures. Care ethics emphasizes relationships, vulnerability, dependence, and responsiveness. Social contract theory asks whether rules could be justified to those governed by them. Value pluralism recognizes that several genuine values may conflict and cannot always be reduced to one common measure.

No single theory resolves every AI dilemma. Their purpose is to improve ethical analysis by making assumptions explicit, revealing overlooked stakeholders, and supporting defensible decisions.

Learning Objectives

By the end of this chapter, students should be able to:

1. Explain the central ideas and limitations of major ethical theories.

2. Apply ethical theories to AI systems and socio-technical practices.

3. Distinguish facts, values, principles, and conclusions in an ethical argument.

4. Analyse conflicts among privacy, safety, fairness, autonomy, efficiency, and innovation.

5. Construct and evaluate a reasoned recommendation for responsible AI design or deployment.

Why Moral Reasoning Matters in AI

An AI system is embedded in a larger socio-technical system consisting of data, models, interfaces, institutions, users, affected communities, and governance arrangements. Ethical responsibility therefore cannot be assigned only to the model. A technically accurate prediction can still support an unjust policy; a legally permitted data practice can still undermine autonomy; and a well-intentioned system can produce harmful effects when deployed in a different social context.

Ethical reasoning is especially important because AI decisions may be opaque, scaled rapidly, difficult to contest, and based on historical data containing social inequalities. The central questions include:

• Who benefits and who bears risk?
• Which interests are represented in the design process?
• What information and consent are required?
• Is the decision fair in its outcomes and procedure?
• Can affected people understand, challenge, and correct the decision?
• What responsibilities remain with human organisations after automation?

Ethical analysis begins with empirical facts, but facts alone do not determine what should be done. For example, evidence that a facial-recognition system improves identification accuracy does not by itself establish that the system should be used. A moral premise is needed concerning privacy, public safety, proportionality, or acceptable risk.

Structuring an Ethical Argument

A useful ethical argument contains five elements:

1. Description of the practice: What system is being designed or used, by whom, and in what context?

2. Stakeholders and impacts: Who may benefit, suffer, lose opportunities, or gain power?

3. Normative principles: Which duties, rights, values, or virtues are relevant?

4. Reasoned evaluation: Why do the principles support or oppose particular actions?

5. Recommendation and conditions: What should be done, under what safeguards, and how should the decision be revisited?

A simple argument may be expressed as follows:

• Premise 1: Organisations should not impose significant, foreseeable harm without adequate justification and safeguards.
• Premise 2: The proposed system creates a substantial risk of unjustified exclusion.
• Conclusion: The system should not be deployed in its current form.

A strong argument also considers objections. It may acknowledge that the system produces benefits, then explain why those benefits do not outweigh the risks, or specify safeguards that would change the conclusion. Ethical reasoning is not the expression of personal preference; it is a practice of giving reasons that can be examined by others.

Consequentialism

Consequentialism evaluates actions by their consequences. An action is morally preferable when it produces better overall results than available alternatives. Utilitarianism is the best-known consequentialist approach and commonly seeks to maximise overall welfare or minimise suffering.

In AI, consequentialist analysis may compare the expected benefits and harms of deploying a model. Relevant consequences can include lives saved, errors, discrimination, privacy loss, economic opportunity, environmental costs, public trust, and effects on future behaviour. The analysis should include direct and indirect effects, short- and long-term effects, and effects on people who are not the system’s users.

A simplified expected-harm comparison can be written as:

Expected harm = sum of (probability of outcome × severity of outcome)

This is a decision aid rather than a complete moral theory. Probabilities must be evidence-based, severity judgments must be justified, and some harms—such as humiliation, loss of dignity, or rights violations—may not be adequately represented by a numerical score.

Consequentialism is useful when policy-makers must compare alternatives and scarce resources. For example, an AI triage tool may be justified if it substantially improves access to urgent care while producing fewer and less serious errors than existing practice. However, consequentialism can permit sacrificing a minority for a larger aggregate benefit. It may also undervalue distribution, procedural fairness, and rights. A system that benefits many people might still be unacceptable if it deliberately targets an innocent group.

Deontology and Duty-Based Ethics

Deontological ethics evaluates actions according to duties, rules, and principles rather than consequences alone. Immanuel Kant’s formulation emphasizes treating people as ends in themselves, never merely as means. In AI governance, this supports requirements for respect, informed participation, non-deception, and meaningful human responsibility.

A duty-based analysis asks:

• Was the person treated with respect?
• Was consent obtained where appropriate?
• Was the action consistent with a rule that could be applied fairly to everyone?
• Were people manipulated, deceived, or used merely as instruments?
• Did the organisation fulfil its duty to explain, protect, and provide recourse?

For example, secretly using intimate personal conversations to train a commercial system may be objectionable even if the resulting model provides useful services. The problem is not only the outcome; it is the failure to respect people’s authority over information about themselves.

Deontology protects moral limits. It can therefore resist practices such as discriminatory profiling even when such profiling appears statistically efficient. Its limitation is that duties may conflict. A hospital may have duties to preserve confidentiality, prevent harm, and allocate scarce treatment fairly. Duty-based reasoning must then interpret which duties apply, whether one is stronger, and whether a proportionate exception is justified.

Virtue Ethics

Virtue ethics focuses on character and practical wisdom. Instead of asking only “What rule applies?” it asks “What would a responsible, honest, just, courageous, and compassionate professional do in this situation?” Virtues are stable dispositions developed through practice and guided by judgment.

AI development requires virtues at both individual and organisational levels. Relevant virtues include:

• Honesty: accurately representing system capability and uncertainty.
• Humility: recognising limitations, bias, and the possibility of error.
• Justice: taking unequal impacts seriously.
• Courage: raising concerns despite commercial or managerial pressure.
• Care: attending to vulnerable and less powerful stakeholders.
• Practical wisdom: balancing principles appropriately in context.

Virtue ethics is valuable in circumstances where rules are incomplete or new risks are emerging. An engineer may notice that a model meets formal performance targets but is unsuitable for a vulnerable population. A culture of humility and courage encourages the engineer to escalate the concern.

The theory’s limitation is that virtues can be interpreted differently across cultures and institutions. It also requires organisational structures that support good conduct; telling employees to be virtuous does not compensate for unrealistic deadlines, retaliation against whistleblowers, or incentives that reward harmful deployment.

Rights-Based Ethics

Rights-based ethics treats certain claims as morally important because they protect human status and agency. Rights may include privacy, bodily integrity, equality, freedom of expression, due process, property, and access to essential services. Rights can impose duties on organisations and governments, including duties to respect, protect, and provide remedies.

AI systems raise rights questions in several ways. Large-scale data collection may interfere with privacy. Automated decisions may threaten equality or due process. Generative systems may affect expression and reputation. Biometric monitoring may affect bodily and informational autonomy. A rights-based approach asks not only whether the system is useful, but whether it crosses a moral boundary.

Rights are not always absolute. Privacy may be limited to prevent serious crime, but the limitation should have a legitimate purpose, be necessary, proportionate, and subject to oversight. Rights analysis therefore requires attention to justification, scope, safeguards, and remedies. A significant safeguard is contestability: affected individuals should have a practical opportunity to know that an important decision was made, challenge relevant information, and obtain human review where appropriate.

Justice and Fairness

Justice concerns how benefits, burdens, opportunities, and decision-making power are distributed. It also concerns whether procedures are fair. In AI, fairness is not a single technical property. Different fairness objectives may conflict, especially when groups have different base rates or when data reflect structural disadvantage.

Three dimensions are particularly useful:

1. Distributive justice: Who receives benefits and who bears risks?

2. Procedural justice: Were decisions made transparently, consistently, and with opportunities for participation and appeal?

3. Recognitional justice: Are the identities, experiences, and social positions of affected groups acknowledged rather than ignored?

A hiring system may have similar selection rates across groups yet still be unjust if the job criteria reflect exclusionary assumptions or if applicants cannot challenge errors. Conversely, a system with unequal selection rates may not be unjust if the difference reflects a legitimate, relevant criterion—but that claim requires careful evidence and scrutiny.

Fairness analysis should therefore combine statistical testing with domain knowledge and stakeholder engagement. Teams should examine data provenance, label quality, subgroup performance, accessibility, error severity, and institutional consequences. Remediation may involve changing the objective, collecting better data, removing inappropriate features, adding human review, or deciding not to automate.

Care Ethics

Care ethics emphasises relationships, dependency, context, empathy, and responsiveness to vulnerability. It challenges the idea that people are always independent, interchangeable decision subjects. Children, older adults, patients, disabled people, refugees, and people in poverty may experience AI systems differently because they have less power to refuse or contest them.

A care-oriented approach asks:

• What relationships does the system create or weaken?
• Who depends on the organisation’s decisions?
• Whose voice is absent from the design process?
• Could automation increase isolation or transfer care burdens to families and workers?
• What would respectful support look like in this context?

For example, an AI chatbot used in mental-health support may be available continuously, but care ethics requires attention to emotional vulnerability, escalation to qualified professionals, cultural sensitivity, and the possibility that users mistake simulation for a caring relationship. Efficiency cannot substitute for all forms of human presence.

Care ethics can be criticised for offering less determinate decision procedures. Its strength is precisely its sensitivity to context and relationships that abstract rules may miss. It should complement, not replace, rights, safety, and justice analysis.

Social Contract Theory

Social contract theories ask what rules free and equal people could reasonably accept as the basis of collective life. Contemporary approaches, especially those associated with John Rawls, examine fairness under conditions in which individuals do not know their own social position. This “veil of ignorance” is a thought experiment: decision-makers consider rules without knowing whether they will be rich or poor, healthy or disabled, part of a majority or minority group.

Applied to AI, social contract reasoning asks whether citizens could reasonably accept widespread automated surveillance, opaque eligibility decisions, or unequal access to high-quality services. It highlights legitimacy, democratic accountability, and public justification. A system may be technically effective yet politically illegitimate if communities have no meaningful voice in its adoption.

Social contract theory supports institutional safeguards such as public consultation, independent oversight, transparent procurement, impact assessment, and appeal mechanisms. Its limitation is that apparent agreement may conceal unequal power. Consultation must therefore be designed so that participation is accessible and dissent is not treated as mere resistance to innovation.

Value Pluralism and Ethical Trade-Offs

Value pluralism holds that human values are multiple, genuine, and sometimes incommensurable. Privacy, safety, liberty, equality, efficiency, solidarity, innovation, and accountability may all matter, but there may be no single scale that settles every conflict.

Pluralism does not mean that “anything goes.” It requires disciplined comparison, recognition of tragic choices, and justification of the balance adopted. A pluralist process may include:

1. Identify all materially affected values.

2. Clarify what each value means in the specific context.

3. Identify conflicts and determine whether design changes can protect several values simultaneously.

4. Give priority to fundamental rights and protection against severe, irreversible harm.

5. Compare remaining options transparently.

6. Document uncertainty, dissent, safeguards, and review conditions.

This approach encourages value-sensitive design. Instead of asking how to optimise one objective, designers ask how system requirements can embody multiple values—for example, combining privacy-preserving data practices, accessible explanations, bias testing, human escalation, and user control.

A Practical Deliberation Framework

The following framework can guide an AI ethics review.

1. Define the decision

Specify the proposed use, purpose, affected population, decision-maker, and alternatives, including the option of not deploying the system.

2. Establish the facts

Examine data sources, model performance, uncertainty, operating environment, foreseeable misuse, and institutional incentives. Separate verified facts from assumptions.

3. Map stakeholders and power

Identify direct users, indirectly affected people, workers, vulnerable groups, communities, future users, and the environment. Record who can influence the decision and who can challenge it.

4. Apply multiple theories

Use consequentialism to assess outcomes; deontology and rights to identify limits; justice to examine distribution and process; care ethics to examine vulnerability and relationships; virtue ethics to assess professional character and organisational culture; and social contract theory to assess legitimacy.

5. Generate alternatives

Alternatives may include narrower scope, additional consent, human review, delayed deployment, independent auditing, improved accessibility, or non-automated practice.

6. Deliberate and decide

Compare options, explain trade-offs, and state why the selected option is proportionate. Record dissent and unresolved uncertainty.

7. Monitor and revise

Ethical approval is not permanent. Monitor real-world outcomes, complaints, subgroup effects, security incidents, and changing social conditions. Establish thresholds that trigger suspension or redesign.

Illustrative Case: Automated Student Risk Prediction

Background: A university proposes an AI system that predicts which first-year students are at risk of dropping out. It uses attendance, grades, learning-platform activity, financial information, and demographic data. Advisers will contact students identified as high risk.

Problem: The university claims that early intervention will improve retention. Students and staff are concerned about privacy, stigma, inaccurate predictions, and the possibility that resources will be diverted from students who are not flagged.

Consequentialist analysis: If the system accurately identifies students who need support, it may improve educational outcomes. However, false positives may cause anxiety and stigma, while false negatives may create unjustified reassurance. The university must compare the system with alternatives such as universal advising and assess long-term effects, including whether students change their behaviour because they feel monitored.

Deontological and rights analysis: Students should not be treated merely as data points. The university should provide meaningful information about the system’s purpose, data use, and limits. Sensitive data require strong governance. Students should be able to question inaccurate records and obtain human review. Using the prediction to punish students would be ethically different from using it to offer voluntary support.

Justice analysis: Historical data may reproduce unequal access to technology, disability-related attendance patterns, or socioeconomic disadvantage. If students from a particular group are flagged more often, the university must investigate whether the model identifies genuine support needs or encodes institutional bias. Resources should not be allocated solely according to a prediction that reflects past under-support.

Care ethics analysis: Students are in a relationship of dependence with the university. Contact should be supportive rather than surveillance-oriented. Advisers should ask students what assistance they want and recognise that financial, health, family, and cultural circumstances cannot always be represented in administrative data.

Virtue and social contract analysis: Staff need humility about prediction and courage to reject a system that performs well statistically but harms trust. Students should have a voice in governance because the legitimacy of educational monitoring depends on more than managerial approval.

Approach: An ethically stronger design would use the system only as a limited prompt for additional, voluntary support; exclude unnecessary sensitive attributes from routine use; publish plain-language documentation; test subgroup error rates; allow correction and appeal; train advisers; prohibit punitive use; conduct an independent impact assessment; and review outcomes after deployment. Universal baseline support should remain available so that students are not dependent solely on prediction.

Outcome: The case illustrates that the ethical question is not simply whether the model is accurate. The central issue is what institutional practice the model creates and whether students are treated as participants in education rather than objects of surveillance.

Professional and Governance Implications

AI professionals have responsibilities beyond coding. Product managers define purposes and incentives; data scientists assess validity and uncertainty; designers shape user understanding; legal teams interpret obligations; executives determine deployment; and domain professionals act on outputs. Responsibility is distributed, but it must not become diluted. Organisations should assign named owners for risk, maintain decision records, provide escalation channels, and protect good-faith reporting.

Responsible practice includes data minimisation, purpose limitation, accessibility, security, documentation, independent testing, impact assessment, human oversight, and effective remedy. Governance should be proportionate to risk. A recommendation system for entertainment generally warrants different controls from an AI system affecting liberty, employment, healthcare, or public benefits.

Law is an important floor, not the whole of ethics. Compliance may establish minimum requirements, while ethical reasoning asks whether the practice is legitimate, respectful, fair, and worthy of public trust. Professional codes and emerging AI governance frameworks increasingly emphasise human oversight, transparency, accountability, safety, privacy, and non-discrimination, but implementation still requires context-sensitive judgment.

Challenges and Limitations

Ethical analysis faces several difficulties. First, empirical uncertainty makes consequences hard to predict. Second, stakeholders may disagree about values or evidence. Third, organisations may engage in ethics washing by publishing principles without changing incentives. Fourth, numerical fairness metrics can distract from structural injustice or procedural exclusion. Fifth, explanations may be technically accurate but useless to affected people. Sixth, individual safeguards cannot fully correct harms produced by unequal institutions.

These limitations do not make ethical reasoning futile. They show why deliberation should be iterative, participatory, evidence-informed, and connected to enforceable governance. A responsible decision may acknowledge that no option is harmless while still identifying the least unjustifiable option and reducing avoidable harm.

Best Practices for Ethical AI Deliberation

• Define the social purpose before selecting a model.
• Include affected communities, especially those with limited power.
• Consider non-automated and non-deployment alternatives.
• Use several ethical theories rather than relying on one metric.
• Distinguish accuracy from legitimacy and fairness.
• Treat privacy, dignity, and contestability as design requirements.
• Document assumptions, uncertainty, trade-offs, and dissent.
• Provide accessible explanations and practical appeal mechanisms.
• Monitor outcomes after deployment and be prepared to stop the system.
• Align organisational incentives with responsible conduct.

Chapter Summary

Ethical theories provide complementary lenses for analysing AI. Consequentialism evaluates overall benefits and harms; deontology emphasises duties and respect for persons; virtue ethics focuses on character and practical wisdom; rights-based ethics protects claims such as privacy, equality, and due process; justice examines distribution, procedure, and recognition; care ethics highlights relationships and vulnerability; social contract theory addresses legitimacy and public justification; and value pluralism guides deliberation when important values conflict.

A sound AI ethics argument identifies facts, stakeholders, values, alternatives, and reasons. It uses multiple perspectives, gives special attention to severe and irreversible harms, includes affected communities, and connects conclusions to safeguards, accountability, and review. Responsible innovation is therefore not simply the creation of accurate systems. It is the disciplined design and governance of socio-technical practices that can be justified to the people they affect.

Key Terms

Consequentialism: Ethical theory that evaluates actions by their outcomes.

Utilitarianism: A consequentialist approach seeking to maximise overall welfare or minimise suffering.

Deontology: Ethical theory centred on duties, rules, and moral constraints.

Virtue ethics: Approach that evaluates character, habits, and practical wisdom.

Rights-based ethics: Ethical approach that protects fundamental claims and corresponding duties.

Justice: Fair distribution, fair procedures, and recognition of affected groups.

Care ethics: Approach emphasising relationships, dependency, vulnerability, and responsiveness.

Social contract theory: Theories concerning rules that free and equal people could reasonably accept.

Value pluralism: View that multiple genuine values may conflict without one universal measure.

Stakeholder: Person or group affected by, or able to affect, a decision.

Moral premise: A value-based statement supporting an ethical conclusion.

Contestability: The practical ability to question, correct, or appeal a decision.

Proportionality: Requirement that an intervention’s scope and burdens be justified by its purpose.

Procedural justice: Fairness of decision-making processes.

Distributive justice: Fairness in the allocation of benefits and burdens.

Recognitional justice: Acknowledgement of identities, experiences, and social positions.

Value-sensitive design: Designing technology to incorporate human and social values.

Ethics washing: Presenting an appearance of ethical responsibility without meaningful change.

Review Questions

Short Answer Questions

1. What is the difference between a factual premise and a moral premise?

2. How does deontological ethics evaluate an AI practice?

3. Identify three dimensions of justice relevant to AI.

4. Why is contestability important in automated decision-making?

5. What does value pluralism contribute to ethical deliberation?

Long Answer Questions

1. Compare consequentialism and rights-based ethics in the evaluation of facial-recognition technology.

2. Explain how virtue ethics and care ethics can influence the culture of an AI development team.

3. Discuss the role of social contract theory in establishing legitimate AI governance.

4. Explain why fairness cannot be reduced to a single statistical metric.

5. Describe a complete ethical deliberation process for a high-impact AI system.

Analytical / Application Questions

1. A bank’s AI credit model improves repayment prediction but rejects applicants from a historically disadvantaged neighbourhood at a higher rate. Analyse the case using consequentialism, deontology, justice, and care ethics.

2. A public authority proposes continuous facial recognition in transport stations to prevent serious crime. Construct a reasoned argument for or against the proposal, including proportionality, rights, consequences, and democratic legitimacy.

3. Design an ethical governance plan for an AI system that predicts student dropout risk. Identify stakeholders, risks, safeguards, review procedures, and conditions under which deployment should be suspended.

Multiple Choice Questions with Answers

Question 1

Which ethical theory primarily evaluates an AI action by its overall consequences?

A. Consequentialism

B. Deontology

C. Virtue ethics

D. Care ethics

Correct Answer: A. Consequentialism

Question 2

Treating people as ends in themselves is most closely associated with which approach?

A. Utilitarianism

B. Kantian deontology

C. Value pluralism

D. Social contract theory

Correct Answer: B. Kantian deontology

Question 3

Which dimension of justice concerns whether people can understand and challenge a decision?

A. Distributive justice

B. Procedural justice

C. Recognitional justice

D. Retributive justice

Correct Answer: B. Procedural justice

Question 4

What is a central concern of care ethics in AI?

A. Maximising computational speed

B. Eliminating all human relationships

C. Responding to vulnerability and dependence

D. Applying one rule identically in every context

Correct Answer: C. Responding to vulnerability and dependence

Question 5

Which statement best describes value pluralism?

A. Only consequences matter morally

B. All values can always be reduced to money

C. Several genuine values may conflict and lack a single common measure

D. Moral disagreement proves that no ethical reasoning is possible

Correct Answer: C. Several genuine values may conflict and lack a single common measure

Question 6

In the student-risk case, why is voluntary support ethically preferable to punitive use of predictions?

A. Predictions are always accurate

B. It respects student agency and reduces harmful consequences of error

C. It removes the need for data governance

D. It guarantees equal outcomes for every student

Correct Answer: B. It respects student agency and reduces harmful consequences of error

Question 7

Which question is most characteristic of social contract theory?

A. Could people reasonably accept this rule as a basis for collective life?

B. Does the algorithm have the lowest memory usage?

C. Is the developer personally popular?

D. Can every value be assigned a precise numerical score?

Correct Answer: A. Could people reasonably accept this rule as a basis for collective life?

Question 8

What does contestability require in an important automated decision?

A. That the system never make mistakes

B. That affected people have a practical opportunity to question or correct it

C. That all decisions be made without human involvement

D. That the model remain secret from everyone

Correct Answer: B. That affected people have a practical opportunity to question or correct it

Question 9

Which is an example of distributive justice?

A. Providing an appeal process

B. Explaining how a model works

C. Examining who receives benefits and who bears risks

D. Training engineers in humility

Correct Answer: C. Examining who receives benefits and who bears risks

Question 10

Why should ethical analysis consider the option of not deploying an AI system?

A. Non-deployment is always the most innovative option

B. It provides a genuine alternative against which benefits and risks can be compared

C. It eliminates the need to identify stakeholders

D. It guarantees that no social harm exists

Correct Answer: B. It provides a genuine alternative against which benefits and risks can be compared