Problem Framing and the Machine-Learning Lifecycle

Introduction

Machine learning (ML) projects rarely fail because an algorithm cannot be trained. More often, they fail because the wrong problem was selected, the target was poorly defined, the data did not represent the intended population, evaluation was disconnected from organizational value, or a model was deployed without a plan for maintenance. For this reason, effective ML practice begins before model selection and continues after deployment.

Problem framing is the process of translating a business, scientific, public-sector, or operational need into a precise ML task. The ML lifecycle is the connected set of activities through which that task is specified, data are obtained, models are developed and evaluated, systems are deployed, and performance is monitored. These activities are iterative rather than strictly linear: monitoring may reveal a change in the target, data, or operating environment that requires the problem to be reframed.

A useful distinction is between the organizational problem and the prediction problem. An organization may want to reduce customer churn, improve hospital capacity planning, detect fraud, or identify defective products. ML does not directly solve these broad goals. Instead, it estimates a target or supports a decision. The resulting prediction must be integrated with actions, people, policies, and constraints.

Learning Objectives

By the end of this chapter, you should be able to:

1. Translate a business or research need into a well-defined ML task.

2. Distinguish prediction, classification, ranking, clustering, recommendation, and decision-support problems.

3. Define observations, features, targets, prediction horizons, and units of analysis.

4. Identify constraints, assumptions, risks, and appropriate baselines.

5. Select evaluation metrics that reflect both statistical performance and practical value.

6. Explain the major stages of the ML lifecycle, from data collection to monitoring.

7. Recognize leakage, sampling bias, distribution shift, and other common sources of invalid evaluation.

8. Apply responsible AI principles to problem framing, deployment, and governance.

From Organizational Need to ML Task

A well-framed ML problem answers several questions:

• Who or what is being predicted? This defines the unit of analysis, such as a customer, transaction, patient visit, machine, image, or document.
• What is being predicted? This is the target variable, also called the label or outcome.
• When is the prediction made? The prediction time determines which information is legitimately available.
• How far into the future does the prediction apply? This is the prediction horizon.
• What action will follow the prediction? A prediction has value only when it informs a decision or process.
• What constraints apply? Examples include latency, cost, privacy, fairness, interpretability, safety, and regulatory requirements.
• How will success be measured? This requires both model metrics and outcome measures.

A concise problem statement can be written as follows:

> Given information available at time t about an entity or event, estimate target Y over a defined horizon, so that decision A can be improved subject to specified technical, organizational, and ethical constraints.

For example, “use AI to improve retention” is incomplete. A more precise formulation is: “Using customer activity observed during the first 30 days after subscription, estimate whether a customer will cancel during the following 60 days, so that the retention team can prioritize outreach under a fixed monthly budget.” This statement specifies the unit, observation window, target window, action, and resource constraint.

Choosing the ML task

The nature of the target and intended use determines the task type.

Regression estimates a numerical quantity, such as delivery time, energy demand, or house price. Binary classification estimates one of two classes, such as fraud versus legitimate or readmission versus no readmission. Multiclass classification chooses among more than two mutually exclusive classes, such as classifying a support ticket by department. Multilabel classification permits several labels simultaneously, such as assigning multiple topics to a document.

Ranking orders items by relevance, risk, or priority. Search engines and recommendation systems commonly require ranking rather than a single class prediction. Time-series forecasting estimates future values using observations ordered in time. Anomaly detection identifies unusual observations, often when labeled examples of rare failures are limited. Clustering groups observations according to similarity without a predefined target; it is exploratory and does not automatically produce meaningful business segments. Recommendation estimates which items may be useful or attractive to a user. Causal analysis asks what would happen under an intervention, which is different from ordinary prediction. A model that predicts who receives a discount most accurately may not identify who would change behavior because of the discount.

The task should be selected from the decision requirement, not from the availability of a fashionable algorithm. If a fixed rule already performs adequately and is easier to audit, ML may not be justified.

Formalizing the Data and Target

Let an observation be represented by a feature vector \(X\), a target \(Y\), and, where relevant, a decision or action \(A\). A supervised learning dataset consists of examples \((X_i,Y_i)\), for observations \(i=1,\ldots,n\). A model produces a prediction \(\hat{Y}=f(X)\). The notation does not imply that the features cause the target; it only describes an input-output relationship used for estimation.

Unit of analysis and target definition

The unit of analysis must be explicit. “Customer churn” could mean one row per customer, one row per customer-month, or one row per subscription. These alternatives answer different questions and can produce different results. A target should also be operationally measurable. For example, “customer dissatisfaction” might be defined as a survey score below a threshold, a support escalation, or cancellation within a specified period. Each definition has coverage and measurement limitations.

A target definition should specify:

• the event or quantity;
• the population to which it applies;
• the observation and outcome windows;
• the labeling source and procedure;
• treatment of missing, delayed, censored, or conflicting labels;
• the time at which the label becomes available.

For a binary event, one may define

\[

Y_i = \begin{cases}1, & \text{if event } E \text{ occurs for observation } i \text{ during the outcome window},\\0, & \text{otherwise.}\end{cases}

\]

Here, the value 1 indicates occurrence of the specified event, not necessarily a positive or desirable outcome. The definition is valid only when the event and time window are consistently observed.

Features and availability

A feature is an input variable used by the model. A feature may be numerical, categorical, textual, visual, temporal, or derived. The critical question is not simply whether a variable correlates with the target, but whether it is available at prediction time and appropriate for the intended use.

Data leakage occurs when information unavailable at prediction time, or information created using the future outcome, enters the feature set. For example, a hospital discharge code may strongly predict readmission, but it is unavailable when admission decisions are made. Leakage can produce impressive validation results and poor real-world performance.

A feature inventory should record the source, meaning, time of availability, transformation, missingness, sensitivity, and owner of each feature. Data documentation is not administrative overhead; it is part of the model specification.

Prediction horizon and decision threshold

Many models output a score or estimated probability rather than a final action. In binary classification, a model may estimate \(p(X)=P(Y=1\mid X)\). A decision rule classifies an instance as positive when \(p(X)\geq\tau\), where \(\tau\) is a chosen threshold. The threshold should reflect the relative consequences of false positives and false negatives, available resources, and operational capacity. It should not automatically be set to 0.5.

If the cost of a false negative is \(C_{FN}\), the cost of a false positive is \(C_{FP}\), and the predicted probability is calibrated to the true event probability, a simplified cost-sensitive decision chooses positive when

\[

p(X)C_{FN} > (1-p(X))C_{FP}.

\]

Solving gives

\[

p(X) > \frac{C_{FP}}{C_{FP}+C_{FN}}.

\]

This threshold assumes that the action alternatives are binary, costs are appropriately specified, probabilities are calibrated, and other benefits or constraints are ignored. Real decisions may also include intervention cost, capacity limits, and heterogeneous effects.

Constraints, Assumptions, and Baselines

An ML project is bounded by constraints. Technical constraints include memory, inference latency, throughput, availability, and energy consumption. Organizational constraints include staffing, workflow integration, budget, and authority to act. Legal and ethical constraints may govern personal data, automated decisions, retention periods, discrimination, and explainability. Safety-critical systems require stronger evidence and fail-safe procedures than low-risk recommendation tools.

Assumptions should be stated rather than hidden. Common assumptions include:

• training data are sufficiently representative of intended use;
• labels measure the intended concept with acceptable reliability;
• features available at deployment resemble those used during training;
• the relationship between inputs and outcomes remains sufficiently stable;
• users will respond to predictions in a manner compatible with evaluation;
• interventions do not substantially alter the data-generating process.

These assumptions can fail. A fraud model changes attacker behavior; a demand forecast can influence inventory and therefore future demand; a risk score can alter access to services. Such feedback makes monitoring and periodic reassessment essential.

Baselines

A baseline is a reference method against which a new model is compared. A naive baseline may predict the majority class, the previous time-series value, or the historical average. A business baseline may be the current manual process or an existing rules engine. A simple statistical baseline, such as linear or logistic regression, is often valuable because it is interpretable and difficult to outperform meaningfully in small or structured datasets.

A model should not be considered successful merely because it beats a random predictor. It should improve on a relevant baseline while satisfying operational and governance requirements. Baselines also expose target leakage and data errors: implausibly large improvements often require investigation.

Evaluation: Metrics and Experimental Design

Evaluation must reflect the actual purpose of the system. No single metric is universally best.

For binary classification, the confusion matrix counts true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN). Common metrics include:

\[

\text{Accuracy}=\frac{TP+TN}{TP+TN+FP+FN},

\]

\[

\text{Precision}=\frac{TP}{TP+FP}, \qquad

\text{Recall}=\frac{TP}{TP+FN}.

\]

Accuracy measures the fraction of correct classifications. Precision measures the proportion of predicted positives that are truly positive. Recall, also called sensitivity, measures the proportion of actual positives detected. These definitions assume a specified classification threshold and nonzero denominators.

The F1 score is the harmonic mean of precision and recall:

\[

F_1=\frac{2PR}{P+R},

\]

where \(P\) is precision and \(R\) is recall. It is useful when both precision and recall matter, but it ignores true negatives and does not encode business costs.

Specificity is \(TN/(TN+FP)\), the true-negative rate. The receiver operating characteristic (ROC) curve displays true-positive rate against false-positive rate across thresholds; its area under the curve (ROC-AUC) measures ranking discrimination across thresholds. For highly imbalanced data, precision-recall curves may be more informative because they focus on positive predictions.

For regression, common measures include mean absolute error (MAE),

\[

MAE=\frac{1}{n}\sum_{i=1}^{n}|y_i-\hat y_i|,

\]

and mean squared error (MSE),

\[

MSE=\frac{1}{n}\sum_{i=1}^{n}(y_i-\hat y_i)^2.

\]

MAE is expressed in the same units as the target and gives linear penalty to errors. MSE is in squared units and penalizes large errors more strongly. Root mean squared error is \(RMSE=\sqrt{MSE}\), which returns to the target's units. Percentage metrics can be unstable when actual values are zero or near zero.

Metric selection should consider prevalence, costs, calibration, ranking quality, subgroup performance, and operational capacity. A probability model may have good discrimination but poor calibration. Calibration asks whether among cases assigned probability approximately 0.7, about 70% actually experience the event, under comparable conditions.

Beyond model metrics

A model metric is not the same as project success. Evaluation may include:

• reduced processing time;
• lower cost or loss;
• improved safety or service quality;
• increased revenue, subject to causal verification;
• user adoption and appropriate override behavior;
• fairness and accessibility outcomes;
• reliability, latency, and maintenance burden.

If an intervention follows a prediction, a controlled experiment or carefully designed quasi-experiment may be needed to determine whether the complete system improves outcomes. Prediction accuracy alone cannot establish that an action causes benefit.

Splitting data and preventing leakage

A dataset is commonly divided into training, validation, and test sets. The training set is used to fit model parameters; the validation set supports model or hyperparameter choices; the test set provides a final estimate after decisions are complete. The test set should not be repeatedly inspected and used to guide development, because it then ceases to be an unbiased final evaluation.

For independent, identically distributed observations, random splitting may be reasonable. For temporal data, training should generally use earlier periods and validation or testing later periods. For grouped observations, such as multiple records from the same patient or customer, groups should not be split across sets when that would allow identity information to transfer. Cross-validation can provide more stable estimates on limited data, but it must preserve time or group structure where necessary.

The Machine-Learning Lifecycle

1. Discovery and problem framing

Stakeholders identify the decision, affected populations, expected benefit, risks, and success criteria. The team determines whether ML is appropriate and documents the task, target, horizon, constraints, and baseline. Subject-matter experts are essential because data fields often have domain-specific meanings and limitations.

2. Data acquisition and governance

Data may come from operational databases, sensors, surveys, APIs, documents, experiments, or licensed sources. Acquisition requires attention to consent, lawful basis, purpose limitation, access control, retention, provenance, and security. Sampling procedures should reflect the deployment population. Underrepresented groups and rare events require explicit analysis rather than being treated as an afterthought.

3. Data understanding and preparation

Teams inspect distributions, missingness, duplicates, outliers, label quality, temporal coverage, and relationships between sources. Transformations such as encoding, scaling, imputation, and feature construction must be fitted using training data only and then applied to validation and test data. Reproducible pipelines reduce the risk that development and production process data differently.

4. Experimentation and model development

Model selection should follow the task, data, and constraints. Experiments need a clear record of data version, features, algorithm, hyperparameters, random seeds where relevant, metrics, and resource use. Reproducibility is strengthened by version control, automated pipelines, experiment tracking, and documented environments.

Hyperparameter tuning should use validation procedures that match deployment. For example, random cross-validation can yield misleading results when future observations influence past folds. Multiple comparisons also create a risk of selecting a model that performs well by chance; a final untouched test set helps control this risk.

5. Validation and decision review

Validation includes statistical performance, robustness, calibration, subgroup analysis, stress testing, security assessment, and human factors. The question is not only “Is the score high?” but also “Under what conditions does it fail, who bears the cost, and can users recognize or correct errors?” A model card or similar report can document intended use, limitations, evaluation data, and known risks.

6. Deployment and integration

Deployment may be batch, real-time, edge-based, or human-in-the-loop. The system needs interfaces, authentication, logging, fallback behavior, alerting, and a rollback plan. Model outputs should be connected to a defined workflow. A prediction displayed to a user without guidance about interpretation may create automation bias or inconsistent action.

Deployment also requires checking training-serving consistency. A feature that was calculated using a historical batch process may be unavailable or calculated differently in production. Latency, throughput, cost, and availability should be tested under realistic loads.

7. Monitoring and maintenance

Monitoring includes data quality, feature distributions, missingness, prediction distributions, latency, errors, resource use, and, when labels become available, model performance. Data drift refers to changes in input data; concept drift refers to changes in the relationship between inputs and outcomes. A change in class prevalence may affect precision even when ranking ability is stable.

Monitoring thresholds should trigger investigation, not automatic retraining in every case. Retraining can reproduce corrupted labels or amplify a new bias. Maintenance may involve recalibration, threshold adjustment, feature removal, new data collection, model replacement, or retirement. Every change should be versioned and evaluated.

Worked Example: Framing a Support-Ticket Prioritization System

Suppose a software company wants to reduce delays for urgent customer support requests. The initial proposal is “build an AI system that identifies important tickets.” This proposal is too vague.

Background and problem. Support managers currently read incoming tickets and assign priority. Some urgent tickets wait too long, while excessive escalation overloads specialist teams.

Analysis. The team interviews agents and discovers that “important” has several meanings: service outage, security incident, contractual deadline, or severe customer impact. Historical priority labels are inconsistent across teams and may reflect staffing rather than urgency. The decision is not merely to classify text; it is to route tickets and allocate limited attention.

Framing. The unit is an incoming ticket. At ticket creation time, the system uses text and metadata available then. The target is whether the ticket will require specialist intervention within 24 hours according to a documented operational definition. The prediction horizon is 24 hours. The action is to place selected tickets in a specialist queue, whose capacity is limited to 100 tickets per day. A baseline is the existing priority rule and a simple keyword system.

Evaluation. Because missing a genuine urgent ticket is costly, recall is important, but unrestricted escalation is not acceptable. The team reports recall among the top 100 daily tickets, precision at that capacity, false-negative review, subgroup and language performance, and queue latency. A time-based split is used: earlier tickets for training, later tickets for validation and testing. Tickets from the same incident are grouped to prevent near-duplicate leakage.

Approach. The model first produces a probability score. The operations team selects the top 100 tickets rather than using a fixed probability threshold. Human agents can override the ranking, and overrides are logged for later audit. The system is initially deployed in “shadow mode,” generating rankings without changing workflow. This permits comparison with human decisions and identifies data-quality problems.

Outcome. In this illustrative case, the pilot shows that the model improves timely specialist review relative to the keyword baseline, but performance is lower for tickets written in languages with fewer historical examples. The company expands multilingual data collection, adds a language-specific quality monitor, and retains a manual escalation channel. The model is not authorized to close tickets or deny service.

Learning points. The useful target was operationally defined rather than inferred from the word “important.” Capacity changed the appropriate metric from a generic threshold metric to a ranking-at-capacity metric. Historical labels required scrutiny, and deployment included human oversight, shadow testing, monitoring, and a fallback process. The case also demonstrates that fairness and data coverage are part of technical design.

Challenges and Limitations

Label quality and target ambiguity

Labels may be subjective, delayed, incomplete, or affected by previous decisions. A model trained on historical approvals can learn institutional preferences rather than underlying need. Multiple annotators, adjudication rules, agreement analysis, and label audits may be needed.

Sampling and selection bias

Observed data are often generated by existing processes. If only referred patients receive a test, the data do not represent all patients. If only customers who contact support receive labels, silent dissatisfaction is missing. Reweighting or additional data collection may help, but cannot automatically repair all forms of bias.

Imbalance and rare events

When positive events are rare, accuracy can be misleading. A model that always predicts the majority class may appear accurate while detecting no important cases. Evaluation should report prevalence, confusion-matrix counts, precision-recall behavior, and the practical consequences of errors.

Distribution shift and feedback

Economic conditions, user behavior, policies, sensors, and adversarial behavior change. A model can deteriorate without any software defect. Monitoring must be linked to a response plan, and systems with significant consequences require periodic revalidation.

Causal limitations

Predictive association does not establish that changing a feature will change the outcome. A model may identify people likely to fail a course but not identify who would benefit from tutoring. Intervention decisions may require randomized trials, causal inference, or uplift modeling in addition to prediction.

Ethical, Legal, and Governance Considerations

Responsible problem framing asks whether the system should be built, not only whether it can be built. Teams should identify affected stakeholders, potential harms, excluded groups, and avenues for appeal. Sensitive attributes may need protection, but excluding them from modeling does not necessarily remove bias; they may be represented indirectly, and they may be needed to audit subgroup performance.

Important governance practices include data minimization, access controls, documentation, impact assessments, human oversight, incident response, audit logs, and defined accountability. Privacy-preserving techniques can reduce exposure but do not eliminate the need for lawful and ethical purpose. Explainability should be matched to the audience and decision: a developer may need debugging detail, while an affected person may need an understandable reason and a means to contest an outcome.

Regulatory requirements vary by jurisdiction and application. High-impact uses such as employment, credit, education, health, and public services may require transparency, risk management, quality controls, and human rights protections. Compliance is a minimum requirement; technically legal systems can still be harmful or unjust.

Contemporary Developments and Best Practices

Modern ML practice increasingly emphasizes data-centric development, in which improving labels, coverage, and feature quality may produce greater gains than selecting a more complex model. Data contracts define expectations between data producers and consumers, including schemas, freshness, and quality. MLOps applies software engineering and operations practices to reproducible training, deployment, monitoring, and governance.

Foundation models and generative systems introduce additional framing questions: what evidence supports the output, how will hallucinations be detected, what data may be sent to an external service, and where is human verification required? Retrieval, tool use, and automated generation do not remove the need for target definition and evaluation. For generative applications, evaluation may include factuality, groundedness, toxicity, privacy leakage, refusal behavior, and task completion, in addition to latency and cost.

A practical checklist is:

1. Define the decision, user, unit, target, time windows, and action.

2. Confirm that ML is preferable to a rule, process change, or additional data collection.

3. Inventory feature availability and test for leakage.

4. Document assumptions, constraints, risks, and affected groups.

5. Establish naive, business, and simple statistical baselines.

6. Choose metrics and thresholds based on consequences and capacity.

7. Use splits and validation methods that reflect deployment.

8. Evaluate robustness, calibration, subgroup performance, and human interaction.

9. Pilot safely with logging, fallback procedures, and clear ownership.

10. Monitor data, performance, impact, and drift; define when to retrain or retire.

Chapter Summary

Problem framing translates a broad organizational or research need into a precise prediction or analysis task. Effective framing identifies the unit of analysis, target, feature availability, observation window, prediction horizon, action, constraints, and success criteria. Regression, classification, ranking, forecasting, clustering, anomaly detection, recommendation, and causal analysis answer different questions and must not be treated as interchangeable.

Baselines provide meaningful reference points, while metrics such as precision, recall, F1, MAE, and RMSE describe different aspects of performance. Evaluation must account for prevalence, costs, calibration, capacity, subgroup effects, and the difference between model performance and real-world impact. Leakage prevention, time- or group-aware splitting, and disciplined use of validation and test data are essential.

The ML lifecycle includes discovery, data acquisition, preparation, experimentation, validation, deployment, monitoring, and maintenance. It is iterative because changes in data, behavior, policy, or objectives can require reframing. Ethical and governance considerations—including privacy, fairness, transparency, accountability, human oversight, and appeal—are integral to engineering quality. A successful ML system is therefore not merely an accurate model; it is a reliable, useful, maintainable, and responsible decision-support system.

Key Terms

Problem framing: Translating a broad need into a precise ML task and decision context.

Unit of analysis: The entity or event represented by one observation.

Feature: An input variable supplied to a model.

Target variable: The outcome or quantity the model is designed to estimate.

Prediction horizon: The future period to which a prediction applies.

Data leakage: Use of information unavailable at prediction time or derived from the outcome.

Baseline: A reference method used for comparison.

Precision: The proportion of predicted positives that are actual positives.

Recall: The proportion of actual positives detected by the model.

Calibration: Agreement between predicted probabilities and observed frequencies.

Validation set: Data used to guide model or hyperparameter choices.

Test set: Held-out data used for final performance estimation.

Data drift: Change in the distribution of input data over time.

Concept drift: Change in the relationship between inputs and outcomes.

MLOps: Engineering practices for operating ML systems across their lifecycle.

Human-in-the-loop: A system design that includes human review or intervention.

Prediction threshold: A cutoff used to convert a score or probability into an action or class.

Causal analysis: Analysis concerned with effects of interventions, not only prediction.

Model monitoring: Ongoing observation of data quality, performance, behavior, and operational health.

Review Questions

Short Answer Questions

1. What information should a well-defined ML problem statement include?

2. Distinguish a target variable from a feature.

3. Why can accuracy be misleading for rare-event classification?

4. What is data leakage? Give one example.

5. Why is a business baseline useful?

Long Answer Questions

1. Explain how an organizational goal can be translated into an ML task, using customer churn or demand forecasting as an example.

2. Compare precision, recall, F1 score, and calibration, and describe situations in which each is useful.

3. Describe the stages of the ML lifecycle from data acquisition through monitoring.

4. Explain why temporal and grouped data require special train-validation-test splitting strategies.

5. Discuss the ethical and governance issues that should be addressed before deploying a high-impact predictive system.

Analytical / Application Questions

1. A hospital can review only 200 patients daily for a costly intervention. Explain how you would frame the prediction task, choose an evaluation approach, and determine whether ranking or threshold classification is more appropriate.

2. A fraud model has 99% accuracy but detects only 10% of fraudulent transactions. Analyze why the result may be unacceptable and propose more suitable metrics and operational tests.

3. After deployment, a model's feature distributions change substantially, but labels will not be available for three months. Design an interim monitoring and response plan, including what evidence would justify continued use, investigation, recalibration, or suspension.

Multiple Choice Questions with Answers

Question 1

Which element specifies the future period to which an ML prediction applies?

A. Unit of analysis

B. Prediction horizon

C. Baseline

D. Feature inventory

Correct Answer: B. Prediction horizon

Question 2

Which situation is an example of data leakage?

A. Using customer age recorded at prediction time

B. Using an earlier purchase count calculated from historical data

C. Using a discharge code to predict readmission at hospital admission

D. Using a training-only scaling transformation applied to test data

Correct Answer: C. Using a discharge code to predict readmission at hospital admission

Question 3

Recall is defined as:

A. TP divided by TP plus FP

B. TN divided by TN plus FP

C. TP plus TN divided by all cases

D. TP divided by TP plus FN

Correct Answer: D. TP divided by TP plus FN

Question 4

Why is a majority-class baseline particularly important in imbalanced classification?

A. It provides a reference that may achieve high accuracy without detecting rare cases

B. It always gives calibrated probabilities

C. It eliminates the need for a test set

D. It guarantees high recall

Correct Answer: A. It provides a reference that may achieve high accuracy without detecting rare cases

Question 5

Which metric is expressed in the same units as the regression target?

A. Mean squared error

B. Root mean squared error

C. Accuracy

D. ROC-AUC

Correct Answer: B. Root mean squared error

Question 6

For time-series prediction, which split is generally most appropriate?

A. Randomly mixing all observations before splitting

B. Using future observations to train on earlier periods

C. Training on earlier periods and testing on later periods

D. Putting all observations from the same period in every split

Correct Answer: C. Training on earlier periods and testing on later periods

Question 7

What does calibration assess?

A. Whether predicted probabilities correspond to observed event frequencies

B. Whether a model has the lowest possible latency

C. Whether all features have identical units

D. Whether the training set is larger than the test set

Correct Answer: A. Whether predicted probabilities correspond to observed event frequencies

Question 8

A model's input distribution changes over time while the input-output relationship is not yet known to have changed. This is primarily called:

A. Concept drift

B. Data drift

C. Label leakage

D. Overfitting

Correct Answer: B. Data drift

Question 9

Which statement best distinguishes prediction from causal analysis?

A. Prediction always requires randomized experiments

B. Causal analysis asks about effects of interventions, whereas prediction estimates outcomes

C. Prediction cannot use historical data

D. Causal analysis never uses statistical models

Correct Answer: B. Causal analysis asks about effects of interventions, whereas prediction estimates outcomes

Question 10

Why might a system initially be deployed in shadow mode?

A. To allow it to change decisions without logging outcomes

B. To test its outputs and integration without immediately changing operational decisions

C. To avoid defining a target variable

D. To guarantee fairness automatically

Correct Answer: B. To test its outputs and integration without immediately changing operational decisions