12. AI Assurance, Auditing and Continuous Monitoring

Introduction and Context

An artificial intelligence system is not assured merely because it performed well during development or passed a pre-deployment review. Its data may change, users may adapt their behaviour, business processes may be redesigned, and social or regulatory expectations may evolve. A model that was accurate and fair at launch can therefore become unreliable, discriminatory, opaque, insecure, or operationally unsafe after deployment. AI assurance addresses this continuing problem by creating systematic evidence that an AI system remains fit for its intended purpose and is governed throughout its life cycle.

AI assurance is broader than model testing. It includes the examination of objectives, data, design decisions, implementation, operational controls, human oversight, user communication, incident management, and retirement. Auditing provides a structured and independent assessment of these elements, while continuous monitoring supplies recurring evidence about actual behaviour in operation. Together, they transform responsible AI from a statement of intent into an accountable management capability.

For postgraduate practitioners, the central challenge is to connect technical indicators with organizational decisions. A drift score is useful only if it triggers a defined action. A fairness disparity is meaningful only when the population, outcome, comparator, and decision context are specified. An explanation is valuable only when it helps an affected person or authorized reviewer understand, contest, or correct a decision. Assurance is therefore socio-technical: it combines statistics, engineering, governance, law, ethics, risk management, and professional judgment.

Learning Objectives

After studying this chapter, students should be able to:

1. Distinguish AI assurance, auditing, validation, monitoring, and evaluation.

2. Design an internal and external audit plan across the AI life cycle.

3. Select and interpret performance, fairness, drift, robustness, and explainability indicators.

4. Establish traceability, evidence repositories, reporting channels, redress procedures, and human oversight.

5. Develop incident-response and post-deployment monitoring processes.

6. Evaluate the organizational, ethical, regulatory, and technical limitations of AI assurance.

7. Apply these ideas to a realistic high-impact AI deployment.

From Model Validation to AI Assurance

Model validation usually asks whether a model is technically suitable for a stated use under specified conditions. AI assurance asks a wider question: whether the complete system continues to produce acceptable outcomes, under accountable governance, for the people and institutions affected by it.

Several related terms should be separated. Verification examines whether a system has been built according to specified requirements. Validation examines whether it solves the intended problem with adequate performance. Audit is a systematic, documented, and sufficiently independent examination against defined criteria. Monitoring is the ongoing collection and interpretation of operational evidence. Evaluation is a broader assessment of effectiveness, impacts, and consequences, often including qualitative evidence. Assurance is the organized body of processes, controls, evidence, and judgments that supports confidence in a system.

Assurance is not the same as a guarantee. Statistical evidence always has uncertainty, and a successful audit reflects the scope, data, methods, and time of the examination. An audit can also fail if its criteria are incomplete, its independence is weak, or it examines documentation without testing actual outcomes. A mature assurance program therefore treats evidence as provisional and revisable rather than as a permanent certificate of safety.

The Assurance Lifecycle

A practical assurance lifecycle begins before model development and continues until system retirement.

1. Define purpose, risk, and accountability

The organization should document the system’s intended purpose, users, affected groups, decisions supported, degree of automation, foreseeable misuse, and unacceptable uses. A risk classification should consider severity of harm, scale, reversibility, vulnerability of affected persons, and the possibility of human intervention. Responsibility must be assigned to named roles rather than left to an abstract “AI team.” Typical roles include a business owner, model owner, data steward, risk officer, security lead, compliance function, and human reviewer.

2. Establish requirements and a control baseline

Requirements should cover accuracy, timeliness, fairness, privacy, security, explainability, accessibility, safety, record keeping, and escalation. Each requirement should have an owner, a measurable indicator where possible, an evidence source, a review frequency, and a response threshold. This control baseline provides the audit criteria.

3. Build a traceable development record

A traceable record connects the problem definition to data sources, data transformations, features, model versions, code, hyperparameters, evaluation datasets, approval decisions, and deployment configurations. Version control, data lineage, model registries, experiment tracking, and immutable logs support this connection. Traceability is especially important when a team must reconstruct why a particular model produced a particular outcome.

4. Conduct pre-deployment assessment

Pre-deployment assessment should include representative testing, subgroup analysis, robustness and security testing, explainability assessment, privacy review, usability testing, and scenario analysis. Testing should reflect the operational environment rather than only a random hold-out sample. If the system affects high-stakes decisions, the assessment should also examine foreseeable failure modes and the adequacy of human review.

5. Approve with explicit conditions

Approval should specify scope, operating limits, monitoring indicators, review dates, rollback arrangements, and conditions under which deployment must be paused. Conditional approval is often more realistic than a binary safe/unsafe judgment. For example, a system may be approved for recommendation but not autonomous decision-making, or for a narrow population but not for groups inadequately represented in the data.

6. Monitor, investigate, and improve

Post-deployment monitoring compares live behaviour with the approved baseline. Alerts should initiate triage, not automatic conclusions. An investigation may determine that an alert reflects genuine degradation, a change in data collection, a temporary event, or a measurement error. Corrective responses may include recalibration, retraining, threshold adjustment, additional human review, user notification, rollback, suspension, or retirement.

Designing an Audit Program

An audit program should begin with an audit charter. The charter defines scope, objectives, criteria, methods, evidence requirements, independence, reporting recipients, and follow-up responsibilities. Scope can be system-wide or focused on a particular model, data pipeline, vendor, population, or control. A risk-based schedule generally gives more frequent and deeper review to systems that influence employment, credit, health, education, public services, safety, or legal status.

Internal audits

Internal audits are conducted by personnel within the organization or by an internal assurance function. Their strengths include access to operational knowledge, logs, staff, and remediation resources. Their weakness is possible conflict of interest, particularly when auditors report to the same managers whose performance is being assessed. Independence can be strengthened through separate reporting lines, protected access to evidence, rotation of auditors, documented conflicts, and escalation to a risk or audit committee.

An internal audit may examine whether:

• the system is being used for its approved purpose;
• data and model versions are identifiable and reproducible;
• performance and fairness indicators are measured as specified;
• access, security, privacy, and change controls operate effectively;
• human reviewers receive appropriate information and training;
• incidents, complaints, and overrides are recorded and investigated; and
• previous findings have been closed with evidence rather than assurances.

External audits and independent assessments

External audits can increase credibility and provide expertise unavailable internally. They may be conducted by an accredited auditor, specialist assessor, regulator, customer, or independent researcher. External review is particularly valuable where the system provider has commercial incentives to minimize disclosed problems, or where affected parties need assurance that the assessment is not controlled by the deploying organization.

External review is not automatically superior. Auditors may have limited access to proprietary code or sensitive data, may rely heavily on management-selected evidence, or may apply generic criteria that do not fit the use case. Contracts should therefore specify access rights, sampling authority, confidentiality, publication rules, conflict-of-interest requirements, auditor competence, and remediation follow-up.

Audit methods and evidence

A sound audit combines document review, interviews, observation, technical testing, statistical analysis, sampling, and outcome examination. Evidence should be sufficient, reliable, relevant, and reproducible. Examples include model cards, data sheets, risk assessments, code and configuration records, test results, monitoring dashboards, access logs, incident records, user notices, complaint files, override statistics, and meeting decisions.

A useful audit finding identifies the criterion, observed condition, evidence, risk or impact, root cause, responsible owner, corrective action, due date, and verification method. “The model is biased” is not an adequate finding. A stronger finding might state that the false-negative rate for a defined subgroup exceeded the approved tolerance over a specified period, that monitoring failed to alert the owner, and that the control gap resulted from an incomplete subgroup definition.

What Should Be Monitored?

No single metric captures the health of an AI system. Indicators should be selected as a portfolio and interpreted in context.

Performance and operational indicators

Common predictive indicators include accuracy, precision, recall, F1 score, specificity, area under a relevant curve, calibration, and task-specific utility. For regression, mean absolute error, root mean squared error, and error distributions may be appropriate. In ranking or recommendation systems, ranking quality, coverage, diversity, and user outcomes may be more meaningful than classification accuracy.

Operational indicators include latency, availability, failure rates, data-processing errors, queue times, override rates, abstention rates, and the proportion of cases referred to humans. A rise in overrides may indicate model degradation, poor interface design, changing policy, or appropriate professional judgment; it should not be interpreted without investigation.

Thresholds should reflect consequences. A medical triage system may prioritize sensitivity, whereas an automated fraud flag may tolerate more false positives if every flag receives effective review. The selected metric must therefore be connected to the harm of different errors, not chosen solely because it is familiar.

Fairness indicators

Fairness monitoring begins by defining the protected or relevant groups, the decision outcome, the reference population, and the ethical or legal rationale for comparison. Possible indicators include selection-rate parity, demographic parity, equal opportunity, equalized odds, subgroup calibration, error-rate ratios, and intersectional performance. These criteria can conflict, especially when base rates differ. No mathematical metric resolves the underlying normative question by itself.

Monitoring should examine both model outputs and downstream outcomes. A system can show similar selection rates while producing unequal burdens because people receive different opportunities to appeal, or because errors have different consequences. Small samples also create unstable subgroup estimates. Confidence intervals, minimum sample rules, qualitative review, and aggregation over time can reduce misleading conclusions.

Drift and change indicators

Data drift occurs when the distribution of inputs changes. Concept drift occurs when the relationship between inputs and outcomes changes. Label drift occurs when outcome prevalence changes. Drift measures may use population stability indices, distributional distances, characteristic stability, missingness rates, or changes in category frequencies. However, drift is not automatically harmful: a genuine change in population may be legitimate, while a stable input distribution may conceal deteriorating outcomes.

Monitoring should therefore connect drift to performance, fairness, and operational context. It should also detect pipeline changes, sensor failures, altered definitions, and changes in user behaviour. Alerts need severity levels, persistence rules, and clear actions. Excessive alerts create fatigue and encourage staff to ignore the monitoring system.

Explainability and transparency indicators

Explainability must be assessed for the audience and purpose. Useful indicators may include the proportion of cases for which an explanation is generated, explanation stability under small input changes, fidelity of a surrogate explanation to the model, comprehensibility in user testing, and the rate at which explanations support successful review or correction. A technically faithful explanation may still be unusable for an affected person; a simple explanation may be understandable but misleading if it omits important interactions.

Global explanations describe general model behaviour, while local explanations address an individual output. Feature importance is not necessarily causal explanation, and post-hoc explanations do not prove that the model used the stated rationale. In high-impact settings, explanation monitoring should be supplemented by documentation of the model’s purpose, limitations, decision logic, data provenance, and routes for contesting outcomes.

Security, privacy, and safety indicators

Security monitoring may track unauthorized access, suspicious queries, data exfiltration, adversarial inputs, and changes to model or data artifacts. Privacy indicators may include access violations, excessive retention, re-identification risk, and inappropriate use of sensitive attributes. Safety indicators can include hazardous outputs, near misses, unsafe automation, and failures of fallback procedures. These indicators should be integrated with enterprise security and safety operations rather than treated as isolated AI dashboards.

Traceability, Evidence, and the Audit Trail

Traceability enables an organization to answer four questions: what was intended, what was built, what happened, and who decided what to do next. A model identifier alone is insufficient. The organization should be able to link a production prediction to the model version, relevant data and feature transformations, configuration, timestamp, policy version, human actions, and subsequent outcome, subject to privacy and retention requirements.

Evidence management must balance accountability with data minimization. Logs should be tamper-evident, access-controlled, time-synchronized, and retained for a justified period. Sensitive records should be protected through role-based access, encryption, pseudonymization, and secure deletion. Reproducibility may require preserving an exact environment, but retaining every personal record indefinitely is neither necessary nor ethically defensible.

Documentation should distinguish facts, interpretations, assumptions, and unresolved uncertainties. A decision log is valuable because governance failures often arise not from missing technical information but from undocumented judgments about acceptable risk, exceptions, or competing objectives.

Incidents, Redress, and Human Oversight

An AI incident is an event in which system behaviour, use, or failure creates actual or credible risk of harm, rights infringement, material error, security compromise, or governance failure. Near misses deserve attention because they reveal weaknesses before harm becomes irreversible.

An incident process normally includes detection, immediate containment, classification, notification, investigation, remediation, communication, and closure verification. Severity should consider affected people, scale, duration, reversibility, vulnerability, legal duties, and recurrence. Containment may involve disabling a feature, reverting to a previous model, increasing human review, preserving evidence, or notifying affected stakeholders.

Redress is more than a complaint mailbox. An effective mechanism is accessible, timely, understandable, and capable of producing a meaningful remedy. Remedies may include explanation, correction of data, reconsideration by a qualified human, reversal of a decision, compensation where appropriate, or system-level correction. Affected individuals should not be required to understand machine-learning terminology to challenge an outcome.

Human oversight must be substantive. A reviewer who merely clicks approval under time pressure is not a meaningful safeguard. Oversight design should specify reviewer competence, authority to override, workload, information available, escalation routes, independence, and documentation. Automation bias should be addressed through training, interface design, random quality checks, and review of override patterns. Human involvement does not automatically make a system ethical; poorly designed human review can simply legitimize automated errors.

An Illustrative Case: Continuous Assurance for a Credit-Limit Recommendation System

Background

Consider an illustrative financial institution, Northbridge Bank, which deploys a machine-learning system to recommend credit-limit adjustments for existing customers. The system does not formally make the final decision; a trained employee reviews recommendations. The model uses transaction history, repayment behaviour, account tenure, and affordability indicators.

Situation and problem

Six months after deployment, aggregate approval rates remain within the original range. However, customer complaints increase, employee overrides become more frequent, and a monitoring analyst notices that missing-income fields have risen after a change in an external data feed. A simple accuracy dashboard does not show a significant problem because labels arrive slowly and only for a subset of cases.

Analysis

The assurance team first reconstructs the system version, data-feed change, affected time period, and population. It compares missingness, recommendation rates, override rates, and error indicators across demographic and income bands. It discovers that the new feed represents self-employed applicants less completely. The model interprets missingness in a way associated with lower recommended limits. Although the bank did not use protected characteristics directly, the data problem produces unequal effects.

The team also examines explanations shown to employees. The interface highlights repayment history but does not clearly indicate that missing income information is influencing the recommendation. Reviewers consequently accept some recommendations without seeking clarification. This is both a data-quality and human-oversight failure.

Approach and solution

The bank pauses automatic limit increases for affected cases and routes them to enhanced human review. It restores the prior data transformation, adds a missingness indicator to monitoring, and requires confirmation of affordability information before a recommendation is acted upon. Fairness analysis is expanded to include intersectional groups and confidence intervals. The explanation interface is redesigned to distinguish observed information from missing information and to show important limitations.

The audit committee requires a pre-approved change-control process for external data feeds, a data-quality gate that can block deployment, monthly subgroup monitoring, quarterly internal audits during the first year, and an independent assessment after major model changes. Customers who received an adverse limit recommendation during the affected period are given a review route without penalty.

Outcome

The bank reports improved data completeness, a reduction in unexplained overrides, and more consistent recommendations. The most important improvement is not a single fairness score; it is the addition of controls connecting data changes, model behaviour, employee decisions, and customer redress.

Learning points

The case illustrates that stable aggregate performance can conceal subgroup harm. It also demonstrates why monitoring must include data pipelines, explanations, human behaviour, and complaints. Finally, an effective assurance response combines technical correction with process change and remedy for affected people.

Governance, Regulation, and Professional Judgment

AI assurance should be aligned with applicable law, sector rules, contractual duties, organizational policies, and recognized risk-management standards. Legal compliance is a floor, not a complete ethical assessment. A system may comply with a formal requirement while still being unjust, inaccessible, or inappropriate for its social context.

Governance bodies should receive concise risk reports that distinguish indicator status, material incidents, unresolved limitations, and decisions required. Senior leaders must understand that accountability cannot be delegated entirely to data scientists or vendors. Procurement contracts should require documentation, audit access, incident notification, performance commitments, data-use restrictions, and cooperation with redress. Vendor assurance should not rely on marketing claims or generic certifications alone.

Professional judgment is unavoidable. Teams must decide which groups are relevant, which errors are tolerable, when evidence is sufficient, and whether a system should be withdrawn. Such judgments should be transparent, reasoned, reviewable, and open to challenge. Independent voices, affected communities, domain experts, and frontline staff can identify harms that technical dashboards miss.

Common Failure Modes and Limitations

Several recurring practices weaken assurance. Treating a model card as proof of safety confuses documentation with evidence. Measuring only aggregate accuracy hides subgroup and tail risks. Setting thresholds without defining actions produces alerts that no one owns. Auditing only the model ignores data pipelines, interfaces, organizational incentives, and downstream use. Relying on one fairness metric creates false confidence. Using post-hoc explanations as if they were causal accounts misleads users. Human-in-the-loop language can conceal rubber-stamping. Finally, monitoring can become surveillance of employees or affected communities if it collects excessive data without clear purpose.

Metrics themselves have limitations. Labels may be delayed, biased, or unavailable. Small subgroups produce uncertainty. Drift tests can be sensitive to sample size. Fairness metrics may conflict. Explanation methods can be unstable. Incident counts may rise because reporting improved rather than because the system worsened. These limitations should be recorded as part of the assurance case, not hidden from decision-makers.

Emerging Directions and Best Practices

Assurance is increasingly becoming continuous and integrated into machine-learning operations. Model registries, automated data-quality tests, policy-as-code controls, feature and prediction logging, fairness dashboards, canary releases, shadow deployment, rollback automation, and reproducible environments support more timely oversight. These tools do not replace governance; they make governance operationally feasible.

Organizations should also prepare for foundation models and generative systems, where outputs are variable, training data may be difficult to inspect, and misuse can occur outside the original workflow. Monitoring may need to include prompt and output sampling with privacy protections, harmful-content testing, retrieval-source verification, uncertainty communication, usage restrictions, and evaluation of tool actions. Continuous assurance for such systems must address not only model quality but also the evolving application context.

A mature assurance program follows several principles: define purpose narrowly; classify risk before deployment; assign accountable owners; measure what matters to affected people; maintain end-to-end traceability; combine technical and qualitative evidence; make thresholds actionable; protect audit independence; provide meaningful redress; test human oversight; learn from near misses; and retire systems that cannot be kept within acceptable limits.

Chapter Summary

AI assurance is the continuing process of building justified confidence that an AI system is appropriate, accountable, and controlled throughout its life cycle. Auditing supplies structured and independent examination, while continuous monitoring detects changes in performance, fairness, data, explanations, security, safety, and human use. Effective programs define purpose and risk, assign responsibility, maintain traceability, collect reliable evidence, establish actionable thresholds, and connect alerts to investigation and remediation.

Internal audits provide operational access, while external assessments can strengthen independence and credibility. No single metric is sufficient: aggregate performance must be combined with subgroup analysis, drift detection, operational indicators, explanation assessment, security controls, incident records, and redress outcomes. Human oversight must include real authority and suitable working conditions. The illustrative credit-limit case shows how data-feed changes, explanations, employee behaviour, and customer remedies must be examined together. Ultimately, assurance is a socio-technical and governance discipline, not merely a testing exercise.

Key Terms

Assurance: Organized processes and evidence used to support confidence that an AI system is appropriate and controlled.

Audit: A systematic, documented, and sufficiently independent examination against defined criteria.

Continuous monitoring: Ongoing collection and interpretation of evidence about system behaviour and context.

Verification: Examination of whether a system was built according to specified requirements.

Validation: Assessment of whether a system solves its intended problem adequately.

Traceability: The ability to link decisions and outputs to data, model, configuration, actions, and outcomes.

Data drift: Change in the distribution of model inputs over time.

Concept drift: Change in the relationship between inputs and the target outcome.

Label drift: Change in the prevalence or distribution of outcome labels.

Fairness indicator: A quantitative or qualitative measure used to examine unequal treatment or impact.

Calibration: Agreement between predicted probabilities and observed frequencies.

Explainability: The degree to which system behaviour can be meaningfully understood by an appropriate audience.

Human oversight: Human capacity and authority to supervise, intervene in, override, or stop an AI system.

Redress: A process through which affected people can challenge decisions and obtain correction or remedy.

Incident: An event involving actual or credible harm, error, security compromise, or governance failure.

Near miss: A harmful event that was avoided but reveals a control weakness.

Model registry: A controlled repository of model versions, metadata, approvals, and deployment status.

Audit trail: A chronological and protected record of system events, decisions, and changes.

Assurance case: A structured argument supported by evidence that specified risks are controlled.

Automation bias: The tendency to over-rely on automated recommendations or outputs.

Review Questions

Short Answer Questions

1. Distinguish AI assurance from model validation.

2. What is the difference between data drift and concept drift?

3. Why is traceability important in an AI audit?

4. Identify two characteristics of meaningful human oversight.

5. What elements should an incident report contain?

Long Answer Questions

1. Explain the AI assurance lifecycle from purpose definition to retirement.

2. Compare internal and external AI audits, including their strengths and limitations.

3. Discuss how performance, fairness, drift, and explainability indicators should be interpreted together.

4. Explain the role of evidence management, logging, and data minimization in AI governance.

5. Describe how an organization can design an effective redress and incident-response process.

Analytical / Application Questions

1. A deployed recruitment model shows stable aggregate accuracy but a rising rejection rate for a small intersectional group. Design an investigation and remediation plan.

2. A monitoring dashboard generates frequent drift alerts, but operational outcomes remain stable. Analyze possible causes and propose a revised alert policy.

3. An employee routinely approves model recommendations without examining explanations. Evaluate the oversight arrangement and recommend technical, organizational, and governance changes.

22. Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes AI assurance from model validation?

A. Assurance examines only predictive accuracy, whereas validation examines governance.

B. Assurance covers continuing socio-technical controls and evidence, whereas validation primarily assesses suitability for an intended problem.

C. Assurance is performed only by regulators, whereas validation is performed only by developers.

D. Assurance occurs before development, whereas validation occurs only after retirement.

Correct Answer: B. Assurance covers continuing socio-technical controls and evidence, whereas validation primarily assesses suitability for an intended problem.

Question 2

A model’s input distribution changes after a data-feed modification, but the relationship between inputs and outcomes has not yet been shown to change. This is primarily an example of:

A. Concept drift

B. Data drift

C. Automation bias

D. Calibration failure

Correct Answer: B. Data drift

Question 3

Which audit finding is the most professionally adequate?

A. The model is biased.

B. The system appears unfair to some users.

C. The false-negative rate for a defined subgroup exceeded the approved tolerance during the stated period, and no alert was generated.

D. The data science team should improve fairness.

Correct Answer: C. The false-negative rate for a defined subgroup exceeded the approved tolerance during the stated period, and no alert was generated.

Question 4

Why can aggregate performance monitoring be insufficient?

A. Aggregate metrics cannot be calculated for machine-learning systems.

B. Aggregate results may conceal subgroup disparities and unequal error consequences.

C. Aggregate metrics are always less accurate than qualitative opinions.

D. Aggregate monitoring automatically violates privacy law.

Correct Answer: B. Aggregate results may conceal subgroup disparities and unequal error consequences.

Question 5

Which response best represents meaningful human oversight?

A. A reviewer approves every output automatically to meet processing targets.

B. A reviewer can understand relevant limitations, investigate cases, override recommendations, and escalate concerns.

C. A developer remains available by email but cannot alter deployed decisions.

D. A manager signs a yearly statement that humans remain responsible.

Correct Answer: B. A reviewer can understand relevant limitations, investigate cases, override recommendations, and escalate concerns.

Question 6

In the illustrative credit-limit case, what combination most directly revealed the problem?

A. Aggregate accuracy and model size

B. Data completeness, subgroup outcomes, overrides, explanations, and complaints

C. Training loss and hardware utilization

D. Number of software engineers and deployment cost

Correct Answer: B. Data completeness, subgroup outcomes, overrides, explanations, and complaints

Question 7

What is a major limitation of using a single fairness metric?

A. Fairness metrics never require outcome data.

B. Different fairness criteria can conflict and may not capture downstream burdens.

C. A single metric always produces too many alerts.

D. Fairness metrics can be used only before deployment.

Correct Answer: B. Different fairness criteria can conflict and may not capture downstream burdens.

Question 8

Which control most directly supports reconstruction of why a particular production prediction occurred?

A. A public advertisement describing the product

B. An immutable, access-controlled audit trail linking the output to model, data, configuration, and actions

C. A higher model parameter count

D. A user satisfaction survey without timestamps

Correct Answer: B. An immutable, access-controlled audit trail linking the output to model, data, configuration, and actions

Question 9

A rise in employee overrides should initially be interpreted as:

A. Proof that the model is invalid.

B. Proof that employees are improperly resisting automation.

C. A signal requiring investigation because it may reflect degradation, interface problems, changing context, or appropriate judgment.

D. An event that should be removed from monitoring.

Correct Answer: C. A signal requiring investigation because it may reflect degradation, interface problems, changing context, or appropriate judgment.

Question 10

Which practice best reduces the risk that continuous monitoring becomes ineffective alert generation?

A. Create alerts without assigning owners or actions.

B. Use severity levels, persistence rules, clear thresholds, and predefined responses.

C. Monitor only the model’s average accuracy.

D. Disable alerts whenever they occur frequently.

Correct Answer: B. Use severity levels, persistence rules, clear thresholds, and predefined responses.