Machine Learning Basics

Chapter 14
Decision Analytics, Management and Digital Transformation

Bachelors Program in AI & ML
Machine Learning Basics   •   Bachelor

Decision Analytics, Management and Digital Transformation

Introduction and Context

Machine-learning systems create value only when their outputs improve decisions. A highly accurate classifier may have little organizational value if it is connected to no action, generates unaffordable intervention costs, violates regulation, or is not trusted by its users. Decision analytics therefore extends beyond prediction. It asks what action should be taken, for whom, when, at what cost, with what risks, and under whose authority.

This chapter connects machine-learning outputs to managerial decisions, key performance indicators (KPIs), risk management, cost–benefit analysis, experimentation, organizational readiness, stakeholder communication, and responsible adoption. The central distinction is between a prediction and a decision policy. A prediction estimates an uncertain outcome; a policy maps information into an action. For example, a fraud model may estimate the probability that a transaction is fraudulent, while a policy determines whether to approve, reject, or review the transaction.

Digital transformation is not simply the replacement of manual work with software. It involves redesigning processes, responsibilities, information flows, controls, and customer experiences around digital capabilities. Machine learning can support this transformation, but it also introduces model risk, data dependency, automation bias, cybersecurity exposure, and accountability challenges.

Learning Objectives

By the end of this chapter, students should be able to:

1. Distinguish predictions, decisions, outcomes, and KPIs.

2. Translate model performance into managerial and economic value.

3. Construct a mutually exclusive expected-value analysis without double-counting benefits or losses.

4. Design experiments using treatment effects, confidence intervals, and basic power considerations.

5. Evaluate build-versus-buy choices and organizational readiness.

6. Communicate model limitations and decisions to technical and non-technical stakeholders.

7. Propose responsible governance and implementation strategies for machine-learning systems.

1. From Prediction to Decision

A supervised model commonly estimates a conditional probability or score. In binary classification,

\[

p(x)=P(Y=1\mid X=x),

\]

where \(Y=1\) is an event of interest and \(X=x\) is the observed feature vector. A threshold rule may define an action:

\[

a(x)=

\begin{cases}

1, & p(x)\ge t,\\

0, & p(x)<t.

\end{cases}

\]

Here, \(t\) is a decision threshold, not a universal measure of model quality. Its appropriate value depends on the consequences of false positives, false negatives, intervention capacity, legal requirements, and customer experience.

A useful decision chain is:

Data → prediction → action → operational outcome → organizational objective.

Each link can fail. Data may be incomplete; predictions may be poorly calibrated; actions may not be executed; outcomes may be affected by external events; and the chosen KPI may reward undesirable behavior.

Prediction, decision, and outcome terminology

  • Prediction: an estimate about an uncertain future or unobserved state.
  • Decision: a selected action under constraints.
  • Intervention: an action intended to change an outcome.
  • Outcome: what actually happens after a decision.
  • Counterfactual: what would have happened under an alternative decision.
  • Policy: a rule connecting information to actions.
  • KPI: a measurable indicator linked to an organizational objective.
  • Decision latency: the time between receiving information and acting on it.

A model should not be evaluated only by accuracy, area under the ROC curve, or mean squared error. Those measures describe predictive performance, whereas management requires evidence of operational and economic performance.

2. Aligning Models with Objectives and KPIs

A good implementation begins with an explicit decision statement:

> Who or what is being assessed, which action is available, what constraints apply, and how will success be measured?

A KPI hierarchy helps prevent local optimization. For a customer-support triage system, for example:

  • Strategic outcome: improve customer retention.
  • Operational outcome: resolve important issues rapidly.
  • Process KPIs: first-response time, escalation rate, backlog age.
  • Quality and safety guardrails: complaint rate, unequal service rates, human override rate.
  • Financial KPI: cost per resolved case and retained-customer value.

A single metric can be misleading. Reducing average handling time might improve productivity while lowering resolution quality. Increasing approval rates might raise sales while also increasing defaults. Balanced scorecards therefore combine outcome metrics, process metrics, risk indicators, and fairness or compliance measures.

Leading and lagging indicators

A leading indicator provides early information about future performance, such as model coverage or response time. A lagging indicator records a later result, such as loss rate or customer retention. Leading indicators support operational control, while lagging indicators test whether the intervention created meaningful value.

Metrics should have a defined owner, data source, calculation rule, reporting frequency, target, and escalation procedure. KPI definitions should also specify the population and time window. “Accuracy improved” is incomplete unless the organization states accuracy for which population, period, and decision context.

3. Decision-Theoretic Value and Expected Payoff

3.1 The mutually exclusive payoff framework

Expected-value analysis is useful when outcomes are uncertain and actions have different consequences. Let \(a\) denote an action and \(s\) a mutually exclusive state of the world. The expected payoff is

\[

EV(a)=\sum_s P(s\mid I)\,V(a,s),

\]

where \(I\) is the information available at the decision point and \(V(a,s)\) is the net payoff of action \(a\) if state \(s\) occurs. The states must form a complete, non-overlapping partition, and every cost or benefit must appear once in the payoff table.

This last condition is essential. An avoided loss must not be counted both as a benefit of intervention and as a loss in the same outcome. A clear payoff table prevents this error.

3.2 Worked fraud example

Suppose a bank must decide whether to review a transaction. If the transaction is fraudulent, the bank loses $150 when it approves the transaction. A review costs $4. If review correctly identifies fraud, the transaction is stopped and the $150 loss is avoided. Assume review is perfectly effective for this simplified example and that a legitimate transaction produces no additional payoff or loss.

The two states are fraud and legitimate. The two actions are review and approve.

| Action and state | Net payoff relative to taking no action |

|---|---:|

| Review, fraud | \(-\$4\) |

| Review, legitimate | \(-\$4\) |

| Approve, fraud | \(-\$150\) |

| Approve, legitimate | \(\$0\) |

Let \(p=P(\text{fraud}\mid I)\). Then

\[

EV(\text{review})=-4,

\]

and

\[

EV(\text{approve})=p(-150)+(1-p)(0)=-150p.

\]

Review is preferred when

\[

-4>-150p,

\]

which gives

\[

p>\frac{4}{150}=0.0267.

\]

Thus the break-even fraud probability is approximately 2.67%. The $150 appears only in the payoff for approving a fraudulent transaction. It is not also entered as a separate “missed-event cost” in the review row. If review were imperfect, the table would include separate probabilities of detection and missed fraud, with the corresponding payoffs still assigned to mutually exclusive outcomes.

3.3 Costs beyond direct financial loss

A realistic payoff may include investigation cost, customer friction, recovery probability, regulatory penalties, reputational effects, and capacity constraints. These quantities should be estimated separately and documented. Expected value is not a substitute for judgment where losses are catastrophic, uncertain, legally constrained, or ethically unacceptable. Organizations may impose risk limits even when an action has positive expected monetary value.

3.4 Value of information

A model can be valuable even when it does not change every decision. The expected value of information compares the expected payoff when decisions use model information with the payoff when decisions are made without it. In practice, this comparison should include model development, infrastructure, monitoring, staff time, interventions, errors, and opportunity costs.

4. Risk, Uncertainty, and Model Governance

Model risk is the possibility that a model or its use produces harmful or incorrect decisions. Sources include data drift, target leakage, sampling bias, poor calibration, implementation defects, adversarial behavior, and misuse outside the development population.

Important risk dimensions include:

  • Statistical risk: prediction errors and uncertainty.
  • Operational risk: failures in deployment, integration, or human workflow.
  • Compliance risk: breach of privacy, consumer-protection, or sectoral rules.
  • Strategic risk: investment in a system that does not support organizational goals.
  • Reputational risk: loss of public or stakeholder trust.
  • Security risk: manipulation, theft, or unauthorized access.

A governance process should define model ownership, approval authority, documentation standards, validation independence, change control, monitoring, incident response, and retirement conditions. High-impact uses generally require stronger controls than low-risk recommendation systems.

Monitoring should compare current data and outcomes with development assumptions. Useful signals include population stability, missingness, calibration, error rates by subgroup, intervention rates, override rates, complaint rates, and realized financial outcomes. A performance decline may require recalibration, retraining, threshold adjustment, temporary suspension, or human review.

5. Experimentation and Causal Evaluation

Historical performance does not prove that a model caused improvement. If a model is deployed only to high-risk customers, their subsequent outcomes cannot be compared naively with those of low-risk customers because the groups differ before intervention.

5.1 Randomized experiments

In an A/B test, eligible units are randomly assigned to a control condition or treatment condition. Let \(Y_i(1)\) be the outcome for unit \(i\) under treatment and \(Y_i(0)\) under control. The average treatment effect is

\[

ATE=E[Y(1)-Y(0)].

\]

Because both potential outcomes cannot be observed for the same unit, randomization estimates the average effect by comparing sample means:

\[

\widehat{ATE}=\bar Y_T-\bar Y_C.

\]

Randomization supports causal interpretation when assignment is implemented correctly, interference is limited, outcomes are measured consistently, and the treatment and control conditions differ only in the intended intervention.

5.2 Confidence intervals

A point estimate alone does not indicate precision. Under common large-sample assumptions, an approximate 95% confidence interval for a difference in independent means is

\[

\widehat{ATE}\pm 1.96\,SE(\widehat{ATE}),

\]

where \(SE\) is the estimated standard error. For a difference in independent proportions, a commonly used approximate standard error is

\[

SE=\sqrt{\frac{\hat p_T(1-\hat p_T)}{n_T}+\frac{\hat p_C(1-\hat p_C)}{n_C}}.

\]

The normal approximation is less reliable for very small samples or proportions near zero or one; exact or alternative methods may then be appropriate. A 95% confidence interval describes a procedure’s long-run coverage, not a 95% probability that a fixed parameter lies in one already calculated interval.

5.3 Power and sample-size planning

Power planning should occur before the experiment. Researchers specify a baseline rate, minimum practically important effect \(\delta\), significance level \(\alpha\), desired power \(1-\beta\), allocation ratio, and expected attrition or noncompliance.

For a simplified two-group comparison of means with equal group sizes and common standard deviation \(\sigma\), an approximate per-group sample size is

\[

n\approx \frac{2\sigma^2(z_{1-\alpha/2}+z_{1-\beta})^2}{\delta^2}.

\]

This formula assumes independent observations, a continuous outcome, equal variances, equal allocation, and a two-sided test. It is an approximation, not a universal formula. Binary outcomes, clustered assignment, sequential testing, multiple comparisons, and unequal allocation require adjustments. Cluster randomization often reduces effective sample size because observations within a cluster are correlated; a common design effect is \(1+(m-1)\rho\), where \(m\) is average cluster size and \(\rho\) is the intracluster correlation.

A statistically detectable effect may be too small to matter economically. Conversely, a valuable effect may fail to reach significance in an underpowered study. Managers should predefine both statistical and practical success criteria.

5.4 Experiment ethics and operations

Experiments require privacy protection, informed governance, fair participant selection, stopping rules, and monitoring for harm. A treatment should not knowingly expose participants to unreasonable risk merely to gain information. Metrics should include guardrails such as complaints, unequal impact, service failures, and security incidents.

6. Build-versus-Buy and Total Cost of Ownership

The choice between building an ML capability internally, buying a vendor product, or using a hybrid arrangement is strategic rather than purely technical.

Build advantages

  • control over data, features, models, and roadmap;
  • closer fit to distinctive processes;
  • development of internal expertise; and
  • greater ability to inspect and modify the system.

Build disadvantages

  • higher initial cost and longer time to value;
  • recruitment and retention requirements;
  • responsibility for security, reliability, compliance, and maintenance; and
  • risk of creating a system that cannot be operationalized.

Buy advantages and risks

A vendor may provide mature infrastructure, specialist expertise, support, and faster deployment. Risks include vendor lock-in, opaque model behavior, data-transfer concerns, uncertain pricing, weak integration, limited customization, and dependence on the vendor’s update schedule.

Total cost of ownership should include acquisition or engineering, data preparation, integration, infrastructure, licenses, monitoring, validation, training, incident response, legal review, and retirement. A weighted decision matrix can score alternatives against strategic fit, risk, cost, time, performance, explainability, portability, and support. Weighting should be agreed by relevant stakeholders rather than chosen to justify a preferred option.

Contract terms should address data ownership, model-change notification, audit rights, service levels, security controls, subcontractors, incident reporting, exit assistance, and responsibility for harmful decisions.

7. Organizational Readiness and Digital Transformation

Successful adoption depends on more than model quality. A readiness assessment should examine six capabilities:

1. Strategic readiness: a defined problem, accountable sponsor, and alignment with organizational priorities.

2. Data readiness: lawful access, quality, representativeness, lineage, and stewardship.

3. Technical readiness: APIs, identity management, computing, observability, and integration with operational systems.

4. Process readiness: redesigned workflows, escalation rules, service-level expectations, and exception handling.

5. People readiness: skills, training, incentives, role clarity, and willingness to use the system.

6. Governance readiness: policies, risk classification, auditability, privacy, security, and accountability.

Change management is essential because automation changes jobs and authority. Users need to understand what the model does, when it can be overridden, and how feedback is recorded. If staff are held responsible for outcomes but cannot challenge a model, the operating design is unfair and unsafe.

A staged transformation commonly proceeds through discovery, data and process assessment, a limited pilot, controlled production, scaling, and continuous review. Each stage should have a go/no-go criterion. A pilot should test integration and behavior under realistic conditions, not merely demonstrate a favorable offline metric.

8. Stakeholder Communication and Explainability

Different stakeholders need different explanations. Executives need strategic value, risk, cost, and decision rights. Operational staff need actionable guidance and exception procedures. Data scientists need assumptions, validation design, and uncertainty. Customers and regulators may need reasons for consequential outcomes, avenues for correction, and information about data use.

An explanation should answer:

  • What decision is being supported?
  • What information materially influenced it?
  • How reliable is the system for this population?
  • What are the known limitations?
  • Who reviews exceptions and appeals?
  • What evidence supports the expected benefit?

Global feature importance does not explain every individual decision. Post-hoc explanations may be unstable or approximate. A simpler interpretable model may be preferable where transparency is more valuable than marginal predictive improvement. Explanation is not a replacement for fairness testing, validation, or human accountability.

Communication should avoid claims such as “the model is unbiased” or “the algorithm decides objectively.” All models embody choices about targets, data, labels, thresholds, and acceptable error. Clear uncertainty communication is a professional responsibility.

9. Responsible and Compliant ML Adoption

Responsible adoption combines principles with enforceable practices. Core principles include human accountability, lawfulness, privacy, security, fairness, transparency, contestability, reliability, and proportionality.

Privacy measures may include purpose limitation, data minimization, access control, retention limits, encryption, pseudonymization, and privacy impact assessment. Fairness analysis should examine relevant groups and decision stages, but no single fairness metric is universally correct. Equal selection rates, equal error rates, calibration, and individual fairness can conflict, especially when base rates differ. The organization must justify which harms are most important and document trade-offs.

Human oversight must be meaningful. A nominal human who automatically accepts every recommendation does not provide effective oversight. Reviewers need time, authority, training, and information sufficient to challenge the system.

Regulatory obligations vary by jurisdiction and sector. Accordingly, teams should identify applicable privacy, anti-discrimination, consumer-protection, employment, financial, safety, and AI-specific requirements rather than assuming that a general ethical statement is sufficient. Records should support traceability from data and model version to decision and outcome.

10. Illustrative Case: Predictive Maintenance in a Manufacturing Plant

Background and problem

A plant experiences unplanned machine stoppages. The operations team proposes a model that predicts whether a machine will fail within seven days. The initial business request—“deploy an accurate model”—is reformulated as: “reduce unplanned downtime without increasing unnecessary maintenance or safety risk.”

Analysis

Historical labels are defined as a verified failure within seven days. Engineers identify sensor drift, maintenance history, operating conditions, and machine age as candidate predictors. A time-based validation split is used because random splitting could allow information from the future to influence training or evaluation. Performance is assessed using calibration, recall for serious failures, false-alert rate, lead time, and estimated downtime avoided.

The decision policy has three actions: continue operation, schedule maintenance, or stop immediately for safety-critical signals. The model is not authorized to override safety rules. Maintenance capacity is limited, so the threshold must account for available technician hours.

Experiment and economics

Machines are randomly assigned, where operationally safe, to either the existing maintenance process or the model-assisted process. The primary outcome is unplanned downtime hours per machine-month. Secondary outcomes include maintenance cost, emergency repairs, production quality, and safety incidents. Before launch, the team specifies the minimum worthwhile reduction and calculates a sample size using historical variance and desired power. Results are reported with confidence intervals rather than only a point estimate.

Suppose the model-assisted group shows an estimated reduction of 1.8 downtime hours per machine-month, with a 95% confidence interval from 0.6 to 3.0 hours. The interval excludes zero under the stated assumptions, but management still considers whether the lower-bound benefit exceeds implementation and maintenance costs. If the system increases preventive-maintenance expense by more than the value of downtime avoided, deployment may not be justified.

Outcome and governance

The plant begins with one production line, establishes a model owner and an engineering reviewer, and monitors sensor missingness, calibration, alert volume, technician overrides, and safety events. After a sensor upgrade changes the data distribution, the model is temporarily restricted while it is revalidated. This demonstrates that deployment is a continuing control process, not a one-time launch.

Learning points

The case illustrates the importance of defining the decision before selecting a metric, respecting operational constraints, using causal experimentation where feasible, quantifying uncertainty, protecting safety overrides, and planning for drift.

11. Contemporary Developments and Best Practices

Recent digital transformation programs increasingly combine predictive models with workflow automation, large language models, cloud platforms, and real-time data products. These technologies can accelerate analysis but also increase risks involving fabricated outputs, prompt injection, data leakage, unclear provenance, and rapidly changing vendor behavior. Human review, retrieval controls, logging, access restrictions, evaluation sets, and incident processes remain necessary.

Practical best practices include:

1. Start with a decision and measurable outcome, not a model type.

2. Establish a baseline policy and compare against it.

3. Use mutually exclusive payoff states in economic analysis.

4. Separate predictive validation from causal evaluation.

5. Plan sample size, power, confidence intervals, and practical significance before experimentation.

6. Test data quality and subgroup performance before deployment.

7. Document assumptions, ownership, versioning, and limitations.

8. Design workflows for overrides, appeals, and failure recovery.

9. Monitor outcomes, not just model scores.

10. Retire or redesign systems that no longer meet their purpose.

Challenges and Limitations

Decision analytics depends on estimates that may be uncertain or contested. Monetary values for reputation, dignity, safety, and long-term trust are difficult to quantify. Historical data may encode past discrimination or reflect decisions made under obsolete policies. Randomized experiments may be infeasible in high-risk settings, while observational studies require stronger assumptions and careful adjustment. Organizational incentives can also distort KPIs, producing metric gaming or under-reporting of failures.

No framework eliminates judgment. The appropriate response is disciplined transparency: state assumptions, quantify uncertainty where possible, involve affected stakeholders, use safeguards for severe harms, and revise decisions when evidence changes.

Chapter Summary

Machine learning becomes strategically valuable when predictions are connected to well-defined decisions, operational processes, and organizational objectives. Managers should distinguish predictive metrics from outcome KPIs and use balanced measures that include value, quality, risk, and fairness. Expected-value analysis requires mutually exclusive states and complete payoff tables; this prevents double-counting avoided losses. Fraud and other applications should compare the net payoff of alternative actions under each state.

Causal experiments provide stronger evidence than historical comparisons when randomization is feasible. Sound experimentation includes treatment-effect definitions, confidence intervals, power and sample-size planning, practical significance, ethical safeguards, and operational guardrails. Build-versus-buy decisions should consider total cost of ownership, strategic fit, vendor risk, governance, and exit options. Organizational readiness includes strategy, data, technology, processes, people, and governance.

Responsible transformation requires meaningful human oversight, privacy and security controls, fairness analysis, documentation, stakeholder-specific communication, monitoring, and the ability to suspend or retire systems. The central managerial task is not to maximize model sophistication, but to create a reliable, accountable decision system that produces sustainable value.

Key Terms

Prediction: An estimate about an uncertain event or condition.

Decision policy: A rule that maps information to an action.

KPI: A measurable indicator linked to an organizational objective.

Counterfactual: The outcome that would have occurred under an alternative action.

Expected value: The probability-weighted average payoff across mutually exclusive states.

Model risk: The possibility of harmful or incorrect results from a model or its use.

Calibration: Agreement between predicted probabilities and observed frequencies.

Treatment effect: The difference in outcome caused by an intervention relative to a comparison condition.

Average treatment effect: The average causal effect across a defined population.

Confidence interval: An interval produced by a statistical procedure intended to cover a parameter at a stated long-run rate.

Statistical power: The probability of detecting a specified effect under assumed conditions.

Minimum practically important effect: The smallest effect considered meaningful for decisions.

Total cost of ownership: All relevant costs over a system’s life cycle.

Organizational readiness: The capabilities required to adopt and sustain a system.

Human oversight: Meaningful human authority and capacity to review or intervene.

Data drift: Change in the distribution or quality of input data over time.

Model monitoring: Ongoing measurement of model, process, risk, and outcome indicators.

Contestability: The ability to question, appeal, or seek correction of a decision.

Responsible ML: Development and use of ML consistent with legal, ethical, social, and organizational obligations.

Digital transformation: Redesign of organizational processes and capabilities through digital technologies.

Review Questions

Short Answer Questions

1. Why is a prediction not the same as a decision?

2. What is the purpose of a mutually exclusive payoff table?

3. Distinguish a leading KPI from a lagging KPI.

4. What does statistical power measure?

5. Name four dimensions of organizational readiness.

Long Answer Questions

1. Explain how a machine-learning model can be evaluated using strategic, operational, financial, and risk-related KPIs.

2. Describe the assumptions and limitations of randomized experiments in organizational settings.

3. Compare building an ML system internally with buying a vendor solution.

4. Explain why confidence intervals and practical significance should accompany experiment results.

5. Discuss the governance controls required for a high-impact ML decision system.

Analytical / Application Questions

1. A fraud review costs $6 and prevents a $200 loss whenever fraud is detected. Under a perfect-review assumption and no other payoff differences, derive the break-even fraud probability and construct the payoff table. Explain why the avoided $200 must not be counted twice.

2. A treatment group has 400 customers, of whom 80 purchase; a control group has 400 customers, of whom 60 purchase. Calculate the difference in purchase rates and use the approximate independent-proportions standard error to form a 95% confidence interval. Interpret the result cautiously.

3. An organization must choose among building, buying, and using a hybrid ML system. Design a weighted decision matrix containing at least six criteria, justify the weights, and identify governance questions that must be answered before procurement or deployment.

Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes a prediction from a decision policy?

A. A prediction estimates an uncertain outcome, while a policy maps information to an action.

B. A prediction always has monetary value, while a policy never does.

C. A prediction is qualitative, while a policy is always statistical.

D. A prediction requires randomization, while a policy requires no data.

Correct Answer: A. A prediction estimates an uncertain outcome, while a policy maps information to an action.

Question 2

In the simplified fraud example, review costs $4 and approval of a fraudulent transaction costs $150. Assuming perfect review and no other payoff differences, review is preferred when the fraud probability is:

A. Greater than 0.267%

B. Greater than 2.67%

C. Greater than 26.7%

D. Greater than 40%

Correct Answer: B. Greater than 2.67%

Question 3

What is the main purpose of using mutually exclusive states in expected-value analysis?

A. To guarantee that every model is unbiased

B. To ensure probabilities sum to more than one

C. To prevent the same cost or benefit from being counted more than once

D. To eliminate the need for managerial judgment

Correct Answer: C. To prevent the same cost or benefit from being counted more than once

Question 4

Which measure most directly describes whether predicted probabilities agree with observed event frequencies?

A. Calibration

B. Recall

C. Decision latency

D. Sample size

Correct Answer: A. Calibration

Question 5

What does an average treatment effect estimate?

A. The average causal difference between treatment and control potential outcomes

B. The percentage of model predictions that are correct

C. The probability that a confidence interval is correct

D. The total cost of a treatment program

Correct Answer: A. The average causal difference between treatment and control potential outcomes

Question 6

Which item is normally specified in a power calculation before an experiment?

A. The minimum practically important effect

B. The final observed p-value

C. The post-experiment explanation selected by executives

D. The exact outcome for every participant

Correct Answer: A. The minimum practically important effect

Question 7

A 95% confidence interval for an estimated treatment effect is best interpreted as:

A. A 95% probability that the fixed effect lies in this particular interval

B. An interval generated by a procedure designed to cover the fixed effect in 95% of repeated samples under its assumptions

C. Proof that the treatment caused the effect in every individual

D. A guarantee that the result is practically important

Correct Answer: B. An interval generated by a procedure designed to cover the fixed effect in 95% of repeated samples under its assumptions

Question 8

Which factor is a typical build-versus-buy consideration?

A. Whether the model has a mathematically elegant name

B. Vendor exit rights and total cost of ownership

C. Whether all stakeholders prefer identical dashboards

D. Whether accuracy eliminates the need for governance

Correct Answer: B. Vendor exit rights and total cost of ownership

Question 9

Which action demonstrates meaningful human oversight?

A. Requiring a person to click approve without seeing relevant information

B. Allowing trained reviewers to challenge recommendations and escalate cases

C. Removing all human review after deployment

D. Allowing only the vendor to investigate errors

Correct Answer: B. Allowing trained reviewers to challenge recommendations and escalate cases

Question 10

Which is an example of a lagging indicator in predictive maintenance?

A. Number of alerts generated today

B. Percentage of sensors currently reporting

C. Unplanned downtime hours after deployment

D. Average time for an engineer to view an alert

Correct Answer: C. Unplanned downtime hours after deployment

Machine Learning Basics
Chapter 14 of 15