Fairness, Explainability, and Human Oversight

Introduction

Automated decision systems increasingly influence hiring, lending, insurance, education, healthcare, policing, benefits administration, and customer service. Their use can improve consistency, speed, and analytical capacity, but it can also reproduce historical disadvantage, obscure responsibility, and make errors difficult to contest. Data governance therefore cannot be limited to data quality, security, or legal compliance. It must also address whether an AI-supported decision is fair, understandable, reviewable, accessible, and appropriately supervised by people.

Fairness, explainability, and human oversight are related but distinct. A model may be statistically accurate yet unfair to a protected group. A model may be explainable yet based on an unjust decision rule. Human review may exist formally but be ineffective if reviewers simply approve algorithmic recommendations. A mature governance program treats these concerns as a socio-technical system involving data, models, organizational processes, affected people, and institutional power.

This chapter examines sources of bias, fairness metrics, explanation methods, contestability, accessibility, human-in-the-loop controls, model limitations, stakeholder impacts, and trade-offs among accuracy, fairness, privacy, and utility.

Learning Objectives

By the end of this chapter, readers should be able to:

1. Distinguish historical, sampling, measurement, label, aggregation, deployment, and automation biases.
2. Select and interpret appropriate group and individual fairness metrics.
3. Explain the difference between global, local, counterfactual, and example-based explanations.
4. Design contestability and human-oversight processes that are meaningful rather than symbolic.
5. Evaluate accessibility and stakeholder impacts across the AI system life cycle.
6. Analyze trade-offs among predictive performance, fairness, privacy, and operational utility.
7. Propose governance controls for responsible deployment and monitoring.

1. Fairness as a Governance Question

Fairness is not a single technical property. It is a normative judgment about how benefits, burdens, opportunities, and errors should be distributed. Different stakeholders may reasonably prefer different fairness principles. For example, equal selection rates may appear fair when historical access has been unequal, while equal error rates may appear fair when the consequences of false positives and false negatives are severe.

A useful governance process begins by asking:

• Who is affected by the decision?
• Which outcome is being optimized?
• Which groups have historically experienced disadvantage?
• What types of error are possible, and who bears their cost?
• Which variables are legally protected, socially sensitive, or operationally consequential?
• Who can challenge the decision, and what remedy is available?

Fairness should therefore be assessed at three levels:

Data level: Are the observations representative, accurately measured, and legitimately obtained?

Model level: Does the model produce systematically different error patterns or outcomes for groups or individuals?

Institutional level: How is the model used, by whom, under what incentives, and with what appeal and accountability mechanisms?

A fairness audit that examines only model outputs may miss unfair eligibility rules, inaccessible application processes, unequal human review, or harmful downstream consequences.

2. Sources of Bias Across the AI Life Cycle

Bias is not synonymous with intentional discrimination. In analytics, bias often means a systematic difference between a process and a relevant target, but ethical analysis also considers whether that difference creates unjust disadvantage.

2.1 Historical and institutional bias

Historical bias occurs when training data reflect unequal treatment or unequal opportunity. A hiring dataset may show that men were promoted more often because of past managerial practices, not because men were more qualified. Training a model on those outcomes can encode the institution’s history as if it were an objective standard.

Institutional bias may also arise from rules that appear neutral but impose greater burdens on particular groups. Requiring extensive digital documentation can disadvantage people with limited internet access, disabilities, unstable housing, or uncertain immigration status.

2.2 Sampling and representation bias

Sampling bias occurs when the data do not represent the population to which the model will be applied. Underrepresentation can increase uncertainty and error for minority groups. A facial-analysis system trained mainly on lighter-skinned faces may have substantially poorer performance for darker-skinned users.

Representation is not only a question of proportions. A group may be present in sufficient numbers but absent from relevant contexts, such as older adults in a voice-recognition dataset or rural customers in a credit model.

2.3 Measurement and proxy bias

Measurement bias arises when a recorded variable is an imperfect or unequal measure of the underlying concept. Medical expenditure, for example, may be used as a proxy for health need, although spending is also influenced by insurance coverage and access to care.

Proxy bias occurs when apparently neutral features encode protected characteristics. Postal code, language, device type, employment gaps, or browsing behavior may correlate with race, disability, gender, or socioeconomic status. Removing explicit protected attributes does not necessarily remove discriminatory information.

2.4 Label and outcome bias

A label may reflect an institutional decision rather than the true construct of interest. “Defaulted on a loan,” “was arrested,” or “received a high performance rating” is not identical to creditworthiness, criminal propensity, or ability. If the label reflects biased observation, the model learns the observation process.

2.5 Aggregation and deployment bias

Aggregation bias occurs when one model is applied to groups with different relationships between variables and outcomes. A single threshold may be inappropriate where base rates, resources, or error costs differ.

Deployment bias occurs when a model is used in a context different from the one for which it was validated. A risk score designed to prioritize social-service support may be repurposed to deny support. The same prediction can have different ethical implications depending on whether it triggers assistance, surveillance, or punishment.

2.6 Automation and feedback-loop bias

Automation bias is the tendency of people to overtrust machine recommendations. Feedback loops arise when model outputs alter the data later used to retrain the model. For instance, a policing model directs officers to certain neighborhoods; recorded incidents then increase there, apparently confirming the model’s original allocation.

3. Fairness Metrics and Their Interpretation

Fairness metrics operationalize particular normative ideas. They do not prove that a system is fair. Let:

• Y be the actual outcome,
• \hat{Y} be the predicted binary outcome,
• A be group membership,
• S be a predicted score or probability.

A confusion matrix classifies outcomes as true positives, false positives, true negatives, and false negatives. The principal rates are:

These definitions assume a binary decision and clearly defined actual outcomes. TPR is sensitivity or recall; FPR is the proportion of actual negatives incorrectly selected; PPV is precision.

3.1 Demographic parity

Demographic parity requires equal positive prediction rates across groups:

It may be appropriate when equal access to an opportunity is the central objective. However, it can be misleading when groups differ in relevant need or when equal selection rates conceal unequal error rates.

3.2 Equal opportunity and equalized odds

Equal opportunity requires equal true-positive rates:

Equalized odds requires both equal true-positive and false-positive rates:

These criteria focus on error equality conditional on the actual outcome. They require a defensible label and may be difficult to achieve simultaneously with other criteria.

3.3 Predictive parity

Predictive parity requires equal positive predictive values:

This means that a positive prediction has comparable meaning across groups. It does not guarantee equal opportunity: groups can have equal precision while experiencing different rates of false negatives.

3.4 Calibration

A score is calibrated within groups if people receiving the same score have approximately the same observed outcome frequency, regardless of group. If a score of 0.7 is calibrated, approximately 70% of people with that score should experience the outcome in the relevant validation population.

Calibration is useful for probabilistic decisions, but it does not ensure equal selection or equal error rates. In populations with different base rates, calibration and some error-rate criteria cannot generally all hold simultaneously for a non-perfect predictor. This is a consequence of the mathematical relationship among base rates, predictive values, and error rates, not merely an implementation defect.

3.5 Individual and causal fairness

Group metrics can conceal within-group disparities. Individual fairness asks whether similar individuals are treated similarly, but it requires a defensible similarity measure. A causal approach asks whether changing a protected attribute, or a pathway causally connected to it, would change the decision while holding relevant circumstances constant. Causal analysis is difficult because causal graphs require assumptions about historical and social processes that may be disputed.

3.6 Metric selection in practice

Metric selection should follow the decision’s purpose, error costs, legal setting, and stakeholder priorities. Analysts should report disaggregated rates, confidence intervals where appropriate, sample sizes, intersectional groups, and performance over time. A single fairness number is inadequate.

4. Explainability and Interpretability

Interpretability concerns how readily a system can be understood. Explainability concerns the production of reasons or representations that help people understand a model’s behavior. An explanation is not automatically a justification: a model may explain a discriminatory rule accurately.

4.1 Global and local explanations

A global explanation describes overall model behavior, such as feature importance, partial dependence, or a transparent decision rule. A local explanation describes why a particular case received a particular prediction.

Global explanations support model validation and policy review. Local explanations support communication, appeal, debugging, and case management. They answer different questions and should not be substituted for one another.

4.2 Common methods

• Intrinsic interpretability: linear models, scorecards, shallow decision trees, and rule lists are understandable by design, subject to appropriate feature engineering.
• Feature importance: summarizes which features contribute most to predictions, but may be unstable when features are correlated.
• Partial-dependence plots: estimate the average relationship between a feature and predictions, assuming the relevant combinations of features are meaningful.
• Local surrogate methods: approximate a complex model around one case using a simpler model; fidelity may be limited outside that local neighborhood.
• Shapley-style attribution: distributes a prediction difference among features according to contribution rules. Results depend on the chosen baseline and assumptions about feature dependence.
• Counterfactual explanations: describe a minimally changed scenario that would produce a different decision, such as “income would need to increase by ….” The change must be actionable, feasible, and legally and ethically appropriate.
• Example-based explanations: present comparable cases, prototypes, or influential training examples. Privacy and representativeness must be considered.

Explanations should be evaluated for fidelity, stability, completeness, comprehensibility, accessibility, and usefulness to the intended recipient. A technically faithful explanation may overwhelm a customer; a simplified explanation may be understandable but inaccurate.

4.3 Explanation risks

Explanations can reveal personal data, trade secrets, security-sensitive thresholds, or information enabling gaming. They may encourage people to focus on modifiable features while ignoring structural disadvantage. Explanations should therefore be tailored to the recipient and governed as information disclosures.

5. Contestability, Accessibility, and Remedy

Contestability is the practical ability to question, challenge, and seek correction of an automated decision. It is stronger than merely receiving a notice that automation was used.

A meaningful contestability process should provide:

1. Notice that automation materially influenced the decision.
2. A comprehensible explanation of the relevant factors.
3. A channel accessible to people with disabilities and limited digital access.
4. A way to submit corrections, context, or contrary evidence.
5. Review by a suitably trained and sufficiently independent person.
6. A timely decision and reasons for the outcome.
7. A remedy, such as correction, reconsideration, payment, reinstatement, or deletion.
8. A record for audit and systemic learning.

Accessibility includes physical, sensory, cognitive, linguistic, economic, and technological dimensions. A web form that works with a screen reader but requires high-speed broadband is not fully accessible. Voice interfaces may help some users but create privacy risks for others. Human assistance, telephone channels, translated materials, plain language, and offline options may be necessary.

Organizations should avoid placing the burden entirely on affected individuals. People may not know that an algorithm was involved, may not understand what evidence is relevant, or may fear retaliation. Proactive monitoring and independent review are essential complements to appeals.

6. Human Oversight and Human-in-the-Loop Design

“Human in the loop” is not a sufficient control description. The quality of oversight depends on authority, competence, time, information, incentives, and independence.

Three arrangements are common:

• Human-in-the-loop: a person must approve or reject each consequential decision.
• Human-on-the-loop: the system acts automatically while people monitor performance and intervene when conditions require.
• Human-in-command: people define objectives, constraints, escalation rules, and the authority to suspend the system.

Effective oversight requires documented decision rights. Reviewers should know when to override a model, how to record reasons, and who bears responsibility. Organizations should monitor override rates, disagreement patterns, processing time, and whether overrides are themselves unequal across groups.

Human reviewers can introduce bias, fatigue, inconsistency, and automation complacency. A mandatory approval click does not create meaningful control if reviewers lack time or are penalized for disagreement. Conversely, a well-designed system may use automation for routine triage while reserving complex, uncertain, or high-impact cases for expert review.

A practical escalation policy may trigger human review when:

• model confidence is low;
• data are missing, contradictory, or outside the training distribution;
• the case involves a vulnerable person;
• the decision could cause severe or irreversible harm;
• a fairness monitor detects material disparity;
• the affected person contests the result;
• the system is operating outside its validated population or purpose.

7. Model Limitations and Monitoring

Every model has a domain of validity. Limitations may arise from incomplete data, label error, distribution shift, concept drift, measurement error, unobserved confounders, and inappropriate assumptions. Performance reported on a test set does not establish safety in production.

Organizations should maintain a model card or equivalent record describing intended use, excluded uses, population, data sources, evaluation methods, known limitations, subgroup results, privacy constraints, and escalation procedures. A data sheet should document dataset provenance, collection conditions, missingness, licenses, representation, labeling procedures, and known risks.

Monitoring should cover:

• predictive performance and calibration;
• subgroup error rates and intersectional disparities;
• changes in population and feature distributions;
• rates of missing or rejected data;
• override, appeal, and remedy outcomes;
• privacy and security incidents;
• downstream effects, including resource allocation and exclusion.

Fairness monitoring must distinguish random variation from meaningful change. Small subgroup samples can produce unstable rates, while aggregate parity can hide severe intersectional disparities. Statistical uncertainty, practical significance, and stakeholder impact should all be considered.

8. Trade-offs Among Accuracy, Fairness, Privacy, and Utility

Responsible design rarely maximizes every objective simultaneously. Accuracy may conflict with fairness when historical data encode unequal outcomes. Fairness interventions may reduce aggregate accuracy while improving distributional justice. Privacy protections such as data minimization, aggregation, or noise can reduce predictive utility. Explainability may favor simpler models that perform less well, although transparent feature design can sometimes improve both governance and performance.

The relevant question is not “Which model is most accurate?” but “Which system produces acceptable benefits and harms for this purpose under legitimate constraints?”

A multi-objective evaluation can represent utility as:

where A, F, P, and R represent normalized measures of predictive accuracy, fairness, privacy, and robustness or reviewability, and the non-negative weights w_A, w_F, w_P, and w_R express institutional priorities. The weights should not be treated as objective facts; they are governance choices that require justification and stakeholder input. This expression is useful for structuring discussion, not for converting ethical judgment into a purely technical calculation.

A constrained formulation may be more appropriate:

Here ε is an accepted disparity tolerance, δ is a privacy-loss limit, and r is a minimum reviewability requirement. The constraints must be defined operationally and validated; arbitrary thresholds can create false reassurance.

Privacy and fairness may also conflict. Collecting protected-attribute data can create privacy risks, yet not collecting them may prevent disparity measurement. A defensible approach is purpose-limited, access-controlled collection with transparency, retention limits, and strict separation between audit use and decision use where appropriate.

9. Stakeholder Impact Analysis

Affected-stakeholder analysis identifies direct users, decision subjects, workers, communities, regulators, suppliers, and people indirectly affected by system outputs. Stakeholders differ in power and exposure to harm. A bank employee may be able to challenge a flawed workflow, while an applicant may face immediate financial consequences without knowing how the decision was made.

A useful impact assessment asks:

• What benefit is the system intended to provide?
• What burdens or risks may be imposed?
• Are harms reversible?
• Are vulnerable or historically excluded groups affected differently?
• Can people refuse, appeal, or obtain an alternative process?
• What organizational incentives could undermine safeguards?
• What evidence will show whether the deployment is beneficial?

Participatory design, community consultation, and adversarial testing can reveal harms that technical evaluation misses. Consultation should not be symbolic: stakeholders need sufficient information, time, and influence to affect design and deployment decisions.

10. Illustrative Case: Automated Credit-Line Review

Background

A fictional regional bank deploys a model to recommend whether existing customers should receive an increased credit line. The model uses repayment history, income, account activity, debt ratios, and digital engagement indicators. Human analysts review recommendations above a specified risk threshold.

Problem

After deployment, the bank observes that customers in several postal regions receive fewer increases. The model does not use race explicitly, but postal region and device data correlate with race and income. Customers report that explanations are vague and that the appeal form is inaccessible on mobile devices.

Analysis

The governance team investigates at three levels. At the data level, historical credit-line decisions reflect previous analyst discretion and unequal access to financial products. Digital engagement is measured more completely for customers using the bank’s mobile application. At the model level, the group receiving fewer increases has a higher false-negative rate: many customers who would have repaid an increased line are denied. At the institutional level, analysts rarely override recommendations because override decisions are reviewed as possible performance failures.

The team compares selection rates, TPR, FPR, PPV, calibration, sample sizes, and intersectional results. It also tests whether the model remains reliable for customers with missing digital data. It finds that overall accuracy is acceptable, but performance deteriorates for customers with sparse app activity.

Approach and solution

The bank takes several actions:

1. It removes digital engagement as a primary eligibility feature and tests less intrusive alternatives.
2. It collects protected-attribute information separately, under strict governance, for fairness auditing.
3. It recalibrates and validates the model by relevant customer segments.
4. It introduces a plain-language explanation identifying influential factors, data limitations, and correction channels.
5. It creates telephone, branch, and accessible digital appeal routes.
6. It permits analysts to override recommendations without penalty when reasons are documented.
7. It requires human review for low-confidence cases, missing data, and appeals.
8. It monitors subgroup error rates and appeal outcomes quarterly.
9. It pauses automatic expansion if disparity or performance thresholds are exceeded.

Outcome

The revised system produces slightly lower aggregate predictive accuracy but reduces false negatives for the affected group and increases successful appeals. The bank also discovers that some customers were denied because income records were stale, leading to a data-correction program. The outcome demonstrates that fairness intervention is not limited to changing an algorithm; it can require changes to data collection, workflow, incentives, and customer service.

Learning points

First, removing protected attributes does not eliminate proxies. Second, explanation and appeal design are part of fairness. Third, human oversight must include permission and incentives to disagree. Fourth, lower aggregate accuracy may be justified when it reduces serious and concentrated harm. Finally, monitoring must continue after deployment because customer behavior, economic conditions, and data quality change.

11. Governance Practices for Organizations

A robust governance program should:

• define the decision purpose and prohibited uses before model development;
• conduct data, algorithmic-impact, privacy, and accessibility assessments;
• involve affected stakeholders and domain experts;
• establish documented fairness objectives rather than selecting metrics after results are known;
• evaluate intersectional groups and uncertainty;
• use representative validation data and stress tests;
• document assumptions, limitations, versions, and changes;
• provide accessible notice, explanation, appeal, and remedy;
• preserve human authority over high-impact or irreversible decisions;
• monitor outcomes, overrides, appeals, and drift;
• assign accountable owners with authority to suspend deployment;
• audit vendors and require access to relevant performance and incident information;
• retain records sufficient for investigation without unnecessary personal-data retention.

Regulatory requirements vary by jurisdiction and sector, but governance should not be reduced to compliance checklists. Legal compliance is a baseline. Ethical legitimacy also depends on proportionality, necessity, transparency, participation, and effective remedy.

Challenges and Emerging Developments

Current research increasingly examines intersectional fairness, distribution shift, foundation models, synthetic data, privacy-preserving analytics, and fairness under uncertainty. Large language and multimodal models create additional concerns: explanations may be plausible but unfaithful; generated content may amplify stereotypes; and models may behave differently across languages and cultural contexts.

Privacy-preserving techniques, including differential privacy, can reduce disclosure risk, but they may affect minority-group utility disproportionately when data are sparse. Synthetic data may support testing but can reproduce the biases of the source data and should not automatically be treated as representative.

A continuing scholarly debate concerns whether fairness should be encoded as formal parity, causal non-discrimination, equal opportunity, democratic participation, or reduction of material disadvantage. No universal metric resolves this debate. Professional judgment must remain explicit, evidence-based, and open to revision.

Chapter Summary

Fairness, explainability, and human oversight are connected governance responsibilities rather than isolated technical features. Bias can enter through historical institutions, sampling, measurement, labels, proxies, aggregation, deployment, automation, and feedback loops. Fairness metrics such as demographic parity, equal opportunity, equalized odds, predictive parity, and calibration express different values and cannot generally all be satisfied simultaneously.

Explainability includes global, local, counterfactual, feature-based, and example-based methods. Explanations should be faithful, understandable, accessible, secure, and useful for correction—not merely persuasive. Contestability requires notice, reasons, accessible channels, independent review, timely decisions, and effective remedy.

Human-in-the-loop controls are meaningful only when reviewers have authority, competence, time, information, and incentives to challenge the model. Organizations must monitor model limitations, subgroup performance, drift, appeals, overrides, and downstream impacts. Decisions about accuracy, fairness, privacy, and utility should be made transparently through justified objectives and constraints, with affected stakeholders included in governance.

Key Terms

Bias: A systematic distortion or pattern that can produce inaccurate or unjust outcomes.

Calibration: Agreement between predicted probabilities and observed outcome frequencies.

Contestability: The ability to question, challenge, correct, and seek remedy for a decision.

Counterfactual explanation: An explanation describing a feasible change that would alter a model’s outcome.

Demographic parity: Equality of positive prediction rates across groups.

Deployment bias: Bias arising when a model is used in a context different from its validated purpose.

Equal opportunity: Equality of true-positive rates across groups.

Equalized odds: Equality of both true-positive and false-positive rates across groups.

Explainability: The provision of information intended to clarify how a model produces an output.

Fairness metric: A quantitative criterion used to evaluate disparity or equitable performance.

Feedback loop: A process in which system outputs influence later data or decisions.

Human-in-the-loop: A design requiring human participation in individual automated decisions.

Intersectionality: The analysis of overlapping identities and their combined effects on outcomes.

Measurement bias: Distortion caused by unequal or inaccurate measurement of a construct.

Proxy variable: A feature that indirectly encodes a protected or sensitive characteristic.

Predictive parity: Equality of positive predictive values across groups.

Stakeholder impact assessment: A structured analysis of benefits, burdens, risks, and remedies for affected parties.

Transparency: The provision of relevant information about a system, its purpose, operation, limitations, and governance.

Review Questions

Short Answer Questions

1. Distinguish historical bias from deployment bias.
2. What does equal opportunity require?
3. Why does removing a protected attribute not necessarily remove discrimination?
4. What is the difference between a global and a local explanation?
5. Identify two requirements of meaningful contestability.

Long Answer Questions

1. Compare demographic parity, equalized odds, predictive parity, and calibration. Explain why an organization may need to choose among them.
2. Discuss how human oversight can fail even when a person formally approves every decision.
3. Explain how privacy-preserving data practices may both support and complicate fairness auditing.
4. Analyze the role of accessibility in explanations, appeals, and AI-supported public services.
5. Develop a governance framework for monitoring fairness after model deployment.

Analytical / Application Questions

1. A hiring model has equal selection rates across two groups but a substantially higher false-negative rate for one group. Which fairness criterion is satisfied, which is violated, and what additional evidence would you request?
2. Design an appeal and human-review process for an automated benefits eligibility system used by people with limited digital access and disabilities.
3. Using the credit-line case, recommend whether the bank should prioritize aggregate accuracy, equalized error rates, privacy, or another objective. Justify your recommendation and identify the trade-offs.

Multiple Choice Questions with Answers

Question 1

Which condition defines demographic parity?

A. Equal true-positive rates across groups

B. Equal positive prediction rates across groups

C. Equal positive predictive values across groups

D. Equal predicted probabilities and outcomes within each group

Correct Answer: B. Equal positive prediction rates across groups

Question 2

A model has equal true-positive rates and equal false-positive rates across two groups. Which criterion does this describe?

A. Equalized odds

B. Demographic parity

C. Predictive parity

D. Individual fairness

Correct Answer: A. Equalized odds

Question 3

Why can removing race from a model fail to eliminate discriminatory effects?

A. Models cannot operate without race variables

B. All models are inherently discriminatory

C. Other variables may act as proxies for race

D. Race is required to calculate accuracy

Correct Answer: C. Other variables may act as proxies for race

Question 4

Which explanation describes a feasible change that would produce a different model decision?

A. Global feature importance

B. Counterfactual explanation

C. Calibration plot

D. Confusion matrix

Correct Answer: B. Counterfactual explanation

Question 5

Which feature is essential to meaningful contestability?

A. A requirement that the affected person accept the result

B. A hidden model threshold

C. An accessible route to submit evidence and seek review

D. Automatic deletion of all decision records

Correct Answer: C. An accessible route to submit evidence and seek review

Question 6

A human reviewer approves every model recommendation because disagreement may reduce their performance rating. What principal oversight problem is illustrated?

A. Meaningful human authority

B. Automation complacency and weak incentives to challenge

C. Perfect calibration

D. Data minimization

Correct Answer: B. Automation complacency and weak incentives to challenge

Question 7

What is deployment bias?

A. Using a model outside the context for which it was validated

B. Using too few protected attributes in training

C. Making a model explainable to users

D. Collecting data with consent

Correct Answer: A. Using a model outside the context for which it was validated

Question 8

A score is calibrated within two groups when people with the same score have approximately the same observed outcome frequency in each group. What does calibration alone not guarantee?

A. That scores exist

B. That the model uses numerical inputs

C. Equal selection rates or equal error rates

D. That outcomes can be measured

Correct Answer: C. Equal selection rates or equal error rates

Question 9

Why might an organization collect protected-attribute data under strict controls?

A. To automatically use it for every individual decision

B. To prevent all privacy obligations

C. To audit disparities that cannot otherwise be measured

D. To guarantee demographic parity

Correct Answer: C. To audit disparities that cannot otherwise be measured

Question 10

Which case most clearly warrants escalation to human review?

A. A routine case within the validated population with complete data

B. A low-confidence case with missing and contradictory information

C. A case with a high model score and no appeal

D. A case already reviewed and corrected

Correct Answer: B. A low-confidence case with missing and contradictory information