Machine Learning Foundations and Problem Formulation

Introduction

Machine learning (ML) has become a central method for building systems that make predictions, recommendations, classifications, and decisions from data. Its applications range from fraud detection and medical diagnosis to language translation, industrial maintenance, scientific discovery, and autonomous systems. Yet successful ML is not primarily a matter of selecting a sophisticated algorithm. It begins with correctly understanding the problem, defining the learning objective, identifying appropriate data, stating assumptions, and designing an evaluation process that reflects real-world use.

This chapter establishes the conceptual foundation for postgraduate study of machine learning. It distinguishes artificial intelligence, machine learning, and statistical learning; compares major learning paradigms; explains essential terminology; and presents a disciplined process for converting a business or scientific question into an ML task. It also examines the assumptions and limitations that determine whether an ML solution is valid and useful.

Learning Objectives

By the end of this chapter, you should be able to:

• distinguish artificial intelligence, machine learning, deep learning, and statistical learning;
• explain supervised, unsupervised, semi-supervised, self-supervised, and reinforcement learning;
• identify observations, features, labels, parameters, hyperparameters, models, and predictions;
• formulate business problems as measurable ML tasks;
• distinguish prediction, estimation, ranking, clustering, recommendation, and decision problems;
• describe the end-to-end ML lifecycle;
• state important assumptions concerning data, targets, distributions, and deployment;
• recognize leakage, bias, distribution shift, and other common threats to validity;
• evaluate ML systems using metrics appropriate to the decision context.

1. Artificial Intelligence, Machine Learning, and Statistical Learning

Artificial intelligence

Artificial intelligence (AI) is the broad field concerned with constructing systems capable of performing tasks that ordinarily require aspects of human intelligence. These aspects may include perception, reasoning, planning, language understanding, learning, and interaction. AI includes symbolic reasoning, search, knowledge representation, robotics, probabilistic modelling, machine learning, and hybrid approaches.

AI is therefore a broad objective or field, not a single algorithm. A rule-based expert system can be considered an AI system even if it does not learn from data. Conversely, a machine-learning model is an AI component when it supports intelligent behaviour, but not every statistical model is necessarily presented as an AI system.

Machine learning

Machine learning is the study and engineering of computational methods that improve their performance on a task through experience, usually represented by data or interaction with an environment. A useful formalization is:

Here, D is the available dataset, A is a learning algorithm, and f̂ is the learned model or predictor. The model is then used to produce outputs for new inputs.

For a supervised prediction task, the model may be written as f̂(x), where x is an input vector. The output could be a class, probability, or numerical value. Learning is meaningful only when performance is assessed on data not used to fit the model.

Statistical learning

Statistical learning emphasizes learning relationships from data while accounting for uncertainty, variability, sampling, and generalization. It includes both predictive and inferential aims. Prediction asks, “How accurately can unseen outcomes be predicted?” Inference asks, “What can be learned about relationships, effects, or mechanisms?”

For a regression model, one possible representation is:

where y is the observed response, x is the input, f is an unknown systematic relationship, and ε is random error. Common assumptions may include E(ε|x) = 0, meaning that the conditional mean of the error is zero. This assumption is not automatically true and must be justified for the intended interpretation.

Machine learning and statistical learning overlap substantially. Both study generalization from data, but machine learning often emphasizes predictive performance, scalability, automation, and deployment, whereas classical statistics may place greater emphasis on sampling designs, parameter interpretation, uncertainty quantification, and causal reasoning. In modern practice, the boundaries are porous.

Deep learning

Deep learning is a family of ML methods based on multi-layer neural networks. It is especially effective when large datasets and computational resources are available, and when inputs such as images, audio, text, or sensor streams have complex structure. Deep learning is not synonymous with AI or ML: it is a subset of ML, which is itself commonly treated as a subset of AI.

2. Core Terminology

A clear vocabulary is essential because imprecise terminology often leads to poorly defined projects.

Let a dataset contain n observations and p features. The feature matrix can be represented as X ∈ ℝ^n×p, and, in a supervised problem, the target vector as y ∈ ℝⁿ for a numerical response. Each row of X corresponds to one observation and each column to one feature. The notation assumes a numerical representation; categorical data require an appropriate encoding or model representation.

3. Learning Paradigms

3.1 Supervised learning

In supervised learning, each training example contains an input and a target: {(x_i,y_i)}_i=1ⁿ. The algorithm learns a function that maps inputs to targets.

Classification predicts a discrete category, such as fraudulent or legitimate. Binary classification has two classes; multiclass classification has more than two. A model may output a predicted class or a probability such as P(Y=1|X=x).

Regression predicts a numerical quantity, such as demand, temperature, or remaining useful life.

A general empirical risk objective is:

Here, F is the chosen model class, L is a loss function, n is the number of training observations, and arg min denotes the model that minimizes average training loss. This formulation is useful when training observations are sufficiently representative of the deployment population and when the loss reflects the application objective. Minimizing training loss alone can cause overfitting.

For numerical prediction, mean squared error is:

where ŷ_i is the prediction for observation i. MSE is expressed in squared units of the target and penalizes large errors strongly. Root mean squared error is RMSE = √MSE, expressed in the original target units.

For binary classification, a confusion matrix distinguishes true positives, true negatives, false positives, and false negatives. Precision and recall are:

These ratios require nonzero denominators. Precision answers how many predicted positives are correct; recall answers how many actual positives were found. The appropriate trade-off depends on the costs of missed cases and false alarms.

3.2 Unsupervised learning

Unsupervised learning uses inputs without target labels. The aim may be to discover groups, reduce dimensions, identify anomalies, or model the structure of the data.

In clustering, an algorithm may partition observations into groups that are internally similar according to a chosen representation and distance measure. Clusters are not automatically “natural” categories; they depend on the features, scaling, distance, algorithm, and number of groups.

In dimensionality reduction, high-dimensional observations are represented using fewer variables while attempting to preserve selected properties, such as variance or neighbourhood structure. Such representations can support visualization or downstream modelling but may reduce interpretability.

3.3 Semi-supervised learning

Semi-supervised learning combines a small labelled dataset with a larger unlabelled dataset. It is useful when obtaining labels requires expert time or costly experiments but collecting raw observations is easy. Its success depends on assumptions about the relationship between labelled and unlabelled data. For example, unlabelled observations must generally be relevant to the same prediction population; otherwise, they may add noise or reinforce bias.

3.4 Self-supervised learning

Self-supervised learning creates training targets from the data itself. A text model might hide part of a sentence and predict the missing token; an image model might predict a transformed or withheld part of an image. The constructed task provides a pretraining signal without human labels. The learned representation can later be adapted to a supervised or other downstream task.

Self-supervision is not the same as unsupervised learning in every technical taxonomy, but both operate without externally supplied labels. Self-supervised learning specifically defines an internally generated supervisory signal.

3.5 Reinforcement learning

Reinforcement learning (RL) concerns an agent that interacts with an environment over time. At time t, the agent observes a state s_t, chooses an action a_t, receives a reward r_t, and transitions to a new state. The objective is usually to maximize expected discounted return:

where G_t is the return from time t, r_t+k+1 is a future reward, and γ satisfies 0 ≤ γ < 1 in the usual discounted formulation. A smaller γ emphasizes immediate rewards; a value closer to one gives greater weight to future rewards. Episodic problems may instead use a finite horizon.

RL differs from ordinary supervised learning because the correct action is not usually supplied for every state, rewards may be delayed, and actions influence future observations. Exploration creates additional risks in safety-critical settings.

4. From Business Question to ML Problem

A business request is rarely an ML problem in its initial form. “Use AI to reduce customer loss” is an objective, not a precise learning specification. Formulation translates the objective into a decision-supported prediction or analysis task.

A disciplined formulation answers six questions:

1. What decision will be made? For example, whether to contact a customer, inspect a machine, or approve a transaction.
2. Who or what is the unit of prediction? The unit might be a customer, account, order, machine, claim, or customer-month.
3. When is the prediction made? The prediction time establishes which information is legally and operationally available.
4. What outcome and horizon matter? The target might be churn within 30 days or failure within the next 72 hours.
5. What action follows each prediction? Predictions have value only when connected to an intervention or decision.
6. How will success and harm be measured? Metrics should include predictive performance, financial value, operational constraints, and fairness or safety requirements.

Target definition and observation window

Suppose a retailer wants to identify customers likely to purchase in the next 30 days. A possible target for customer i is:

Features must be computed using information available at t₀ or earlier. Purchases occurring after t₀ cannot be used as features, even if they are present in the historical database. The distinction between feature window, prediction time, and outcome window prevents temporal leakage.

Prediction versus decision

A high-probability prediction does not automatically imply that action is worthwhile. Let p(x) be predicted probability of an event, B the benefit of a correct intervention, C the intervention cost, and H the cost of an avoidable harmful action. A simplified expected net value of acting can be represented as:

This expression is illustrative and assumes the quantities are defined on a common monetary or utility scale. Real decisions may require treatment effects, capacity constraints, delayed outcomes, and uncertainty. A prediction model estimates risk; it does not by itself estimate whether an intervention causes improvement.

Choosing the task type

A project should not be labelled “classification” merely because the data contain categories. The task is classification when the intended target is categorical and the output supports that decision.

5. The End-to-End Machine-Learning Lifecycle

Problem framing

The lifecycle begins by specifying the decision, stakeholders, constraints, target population, prediction time, and success criteria. A useful project charter identifies what will happen if the model is wrong and who is accountable for the resulting decision.

Data acquisition and governance

Data may come from transactional systems, sensors, surveys, documents, applications, or third-party sources. Before modelling, teams should establish provenance, access rights, consent requirements, retention policies, and data-quality controls. Data dictionaries should define variables, units, timestamps, coding conventions, missing-value meanings, and known changes in collection.

Exploration and preparation

Exploratory analysis examines distributions, missingness, duplicates, outliers, class imbalance, temporal patterns, and relationships between variables. Preparation may include imputation, scaling, encoding, aggregation, and text or image processing. Transformations must be fitted using training data only when they estimate quantities from data; otherwise information can leak from validation or test sets.

Splitting and validation

A random split may be reasonable when observations are independent and identically distributed—or approximately so—and when future deployment resembles random sampling. Time-dependent problems generally require chronological splits. Grouped observations, such as repeated visits by the same patient, require group-aware splitting to prevent the same entity appearing in both training and evaluation data.

Cross-validation estimates performance across several training-validation partitions. It is valuable for model selection, but it does not replace a genuinely untouched final test set. In small or highly structured datasets, uncertainty in the estimate may remain substantial.

Modelling and baseline construction

A baseline is a simple reference method, such as predicting the majority class, the historical mean, or the last observed value. A complex model should demonstrate improvement over a meaningful baseline, not merely over an arbitrary weak method. Model selection should consider performance, interpretability, latency, memory, maintenance, robustness, and governance.

Evaluation

Evaluation must reflect deployment. Accuracy can be misleading under severe class imbalance. A fraud detector that labels every transaction legitimate may achieve high accuracy while finding no fraud. Precision-recall analysis may be more informative when positive cases are rare. Calibration assesses whether predicted probabilities correspond to observed frequencies; for example, among cases assigned probability near 0.7, approximately 70% should experience the event under a suitable calibration population.

Uncertainty should be reported where feasible. A single score hides variation across subgroups, time periods, sites, and random seeds. Confidence or prediction intervals require assumptions and appropriate estimation procedures; they should not be treated as guaranteed bounds.

Deployment and monitoring

Deployment may involve batch scoring, real-time APIs, embedded devices, or human-facing dashboards. Monitoring should cover input quality, missingness, prediction distributions, latency, resource use, outcome performance, calibration, subgroup behaviour, and drift. A model can degrade even when its code is unchanged because user behaviour, policies, markets, sensors, or populations change.

Maintenance and retirement

Models need documented owners, versioning, rollback procedures, retraining criteria, incident handling, and retirement rules. Retraining automatically on newly collected data can amplify errors or feedback loops if labels are biased or outcomes are affected by prior model decisions.

6. Assumptions and Threats to Validity

Representativeness and dependence

Most evaluation arguments assume that deployment examples are sufficiently related to training examples. The strongest textbook assumption is often independent and identically distributed sampling, but real systems frequently violate it. Observations may be correlated over time, clustered by location, or generated by strategic users.

Label quality

Labels may be incomplete, delayed, subjective, or influenced by institutional decisions. A hospital diagnosis label may reflect access to testing rather than true disease status. In such cases, optimizing agreement with historical labels may reproduce existing inequalities.

Missing data

Missingness can be informative. A missing laboratory test may indicate that a clinician did not suspect a condition, not merely a random technical failure. Imputation must therefore be aligned with the data-generating process and evaluated for subgroup effects.

Causality and intervention effects

Predictive association does not establish causation. A feature may predict churn because it is correlated with dissatisfaction, but changing that feature may not reduce churn. If the objective is to choose an intervention, causal inference, randomized experimentation, or treatment-effect modelling may be needed.

Leakage

Leakage occurs when information unavailable at prediction time enters the features, directly or indirectly. Examples include post-outcome billing codes, future sales totals, labels encoded in filenames, and preprocessing performed before data splitting. Leakage often produces impressive validation results and disappointing deployment performance.

Distribution shift and feedback loops

Let P_train(X,Y) denote the training joint distribution and P_deploy(X,Y) the deployment distribution. Performance can change when these distributions differ. Covariate shift changes P(X), label shift changes P(Y), and concept drift changes the relationship P(Y|X). These categories are idealizations; real changes may involve all three.

A deployed model may also alter the data it later observes. For example, a fraud model that blocks selected transactions prevents those transactions from generating future outcomes, making evaluation and retraining more difficult.

7. Worked Formulation Example

A logistics company wants to reduce late deliveries.

Initial request: “Build an ML system to predict late orders.”

Unit: One order scheduled for delivery.

Prediction time: At the time the order is dispatched, because intervention must occur before dispatch or during routing.

Target: y = 1 if delivery occurs more than 24 hours after the promised time; otherwise y = 0. The threshold must be agreed with operations and may need separate treatment for different service contracts.

Features: Distance, route, carrier, weather available at dispatch, warehouse workload, historical carrier performance computed only from prior orders, and package characteristics.

Forbidden features: Actual arrival time, post-dispatch delay codes, and final route changes that were not known when the prediction was required.

Action: Orders above a chosen risk threshold are routed to an expedited carrier or reviewed by a planner. Capacity limits mean that ranking may be more useful than a simple class label.

Evaluation: Use chronological validation. Report recall among the highest-risk orders, precision at the available intervention capacity, calibration, average delay, intervention cost, and performance by region and carrier.

Baseline: Predict late delivery using historical late rates by route or use the overall late rate. The ML model is valuable only if it improves operational outcomes after intervention costs, not merely if it improves an offline score.

This formulation reveals that the project is not just a classification problem. It is a time-indexed ranking and decision-support problem with operational constraints and possible feedback effects.

8. Illustrative Case Study: Predictive Maintenance in Manufacturing

Background

A manufacturer operates industrial pumps in several plants. Unexpected failures interrupt production, while inspecting every pump frequently is expensive. The organization considers an ML system to prioritize inspections.

Problem

The original proposal is “use sensor data to predict failures.” This is incomplete because failure must be defined, the forecast horizon must be specified, and the intervention must be identified.

The team defines the prediction unit as a pump-day. At the start of each day, the system predicts whether the pump will experience a failure requiring an unplanned shutdown within the next seven days. Sensors available at prediction time include vibration, temperature, pressure, operating load, and recent maintenance history.

Analysis

Historical data reveal that some pumps have many more sensor readings than others and that maintenance records are inconsistent across plants. Failures are rare, making accuracy unsuitable as the principal metric. The team also discovers that a maintenance code entered after a failure would have leaked the target and removes it.

Because readings from the same pump are correlated, a random row-level split would overstate generalization. The evaluation uses chronological splits and reports results separately for plants. A simple baseline based on recent threshold violations is retained.

Approach and solution

The team develops a probability model and a ranking interface. Rather than sending every high-risk pump for inspection, operations can inspect only the top 5% each day. The primary evaluation therefore includes precision at 5%, recall within seven days, calibration, false-alarm burden, and avoided downtime per inspection.

A human maintenance engineer reviews explanations based on recent sensor trends and operating context. The system does not automatically shut down equipment; it supports a safety-reviewed inspection decision. Monitoring detects sensor outages, changes in pump populations, and declining calibration.

Outcome

In an illustrative deployment, the model identifies a greater proportion of failures within the inspection capacity than the baseline, but performance varies by plant because sensor quality differs. The organization therefore delays expansion until data collection is standardized and establishes plant-specific monitoring. The case demonstrates that a technically strong model may not be ready for universal deployment if the measurement process and operating context are inconsistent.

Learning points

• Define the target and horizon in relation to an actual intervention.
• Respect temporal availability when constructing features.
• Use grouped and chronological validation when observations are dependent.
• Select metrics according to capacity, cost, and safety.
• Treat data quality and maintenance workflow as part of the ML system.
• Use human oversight when errors can create physical or operational harm.

9. Ethical, Legal, and Governance Considerations

ML systems can distribute benefits and burdens unevenly. Fairness analysis should consider the relevant groups, decision context, base rates, measurement quality, and legally protected characteristics. There is no single fairness metric that resolves every conflict. For example, equal false-positive rates and equal calibration may be difficult to achieve simultaneously when groups have different base rates.

Privacy requires data minimization, purpose limitation, access control, secure storage, and careful handling of sensitive attributes. Removing a protected attribute does not guarantee fairness because proxy variables may remain. In regulated domains, organizations may need explainability, auditability, human review, contestability, documentation, and records of model changes.

Governance should assign responsibility across data owners, model developers, domain experts, risk teams, and decision-makers. Documentation should state intended use, excluded uses, training data, evaluation populations, limitations, known failure modes, and monitoring requirements. Generative and foundation models introduce additional concerns involving data provenance, memorization, hallucination, prompt injection, and uncertain output reliability; these concerns do not eliminate the need for basic problem formulation.

10. Contemporary Directions and Best Practices

Modern ML increasingly combines self-supervised pretraining, foundation models, multimodal data, retrieval systems, and human feedback. These developments can reduce the cost of task-specific modelling but may increase infrastructure, evaluation, privacy, and governance complexity. Transfer learning is useful only when the source representation is relevant to the target population and task.

Effective practice follows several principles:

• begin with the decision and not the algorithm;
• define a prediction time and prevent future information from entering features;
• establish a simple, credible baseline;
• use validation that matches time, groups, and deployment conditions;
• choose metrics linked to operational value and harm;
• inspect performance across relevant subgroups and environments;
• distinguish prediction from causal intervention;
• document assumptions, data provenance, and limitations;
• monitor the complete system after deployment;
• design for human oversight, security, privacy, and accountability.

Chapter Summary

Artificial intelligence is the broad field of intelligent computational systems, while machine learning develops methods that improve through data or interaction. Statistical learning provides a framework for modelling relationships, uncertainty, and generalization. Supervised learning uses labelled examples; unsupervised learning discovers structure without target labels; semi-supervised learning combines labelled and unlabelled data; self-supervised learning constructs targets from the data itself; and reinforcement learning learns through sequential interaction and reward.

An ML project should be formulated around a decision, unit of prediction, prediction time, target, horizon, action, and success criteria. Classification, regression, ranking, clustering, anomaly detection, recommendation, forecasting, and sequential decision-making serve different purposes. The ML lifecycle extends from problem framing and data governance through preparation, validation, modelling, evaluation, deployment, monitoring, and retirement.

Generalization depends on assumptions about representativeness, labels, dependence, missingness, and distribution stability. Leakage, bias, feedback loops, and distribution shift can invalidate apparently strong results. Ethical and regulatory responsibilities require privacy protection, fairness analysis, documentation, human oversight, and accountable governance. The central lesson is that an ML system is a socio-technical decision system, not merely a trained algorithm.

Key Terms

Artificial intelligence: The field of systems that perform tasks associated with intelligent behaviour.

Machine learning: Methods that learn patterns or policies from data or interaction.

Statistical learning: Learning relationships from data while considering uncertainty, sampling, and generalization.

Feature: An input variable supplied to a model.

Label: The observed target associated with a supervised example.

Model: A learned or specified representation mapping inputs to outputs.

Parameter: A quantity estimated during model fitting.

Hyperparameter: A modelling choice set outside ordinary parameter estimation.

Supervised learning: Learning from input-target pairs.

Unsupervised learning: Learning structure from inputs without externally supplied targets.

Semi-supervised learning: Learning from both labelled and unlabelled observations.

Self-supervised learning: Learning with targets generated from the data itself.

Reinforcement learning: Learning actions through interaction and reward.

Generalization: Performance on new data from the intended deployment setting.

Overfitting: Fitting accidental or training-specific patterns that do not generalize.

Data leakage: Use of information unavailable at the prediction time.

Distribution shift: A change between training and deployment data-generating conditions.

Calibration: Agreement between predicted probabilities and observed event frequencies.

Baseline: A simple reference method used for comparison.

Model governance: Processes for accountability, documentation, monitoring, and control of models.

Review Questions

Short Answer Questions

1. How do artificial intelligence, machine learning, and statistical learning differ?
2. What is the distinction between a parameter and a hyperparameter?
3. Why is prediction time important in feature engineering?
4. Define data leakage and give one example.
5. What is the main difference between supervised learning and reinforcement learning?

Long Answer Questions

1. Explain the five major learning paradigms and provide one appropriate application for each.
2. Describe the complete machine-learning lifecycle from problem framing to model retirement.
3. Explain why accuracy may be inappropriate for a rare-event classification problem. Discuss alternative evaluation considerations.
4. Discuss how distribution shift, label quality, and feedback loops can affect deployment performance.
5. Explain why a business objective must be translated into a unit of prediction, target, horizon, action, and success criterion.

Analytical / Application Questions

1. A bank wants to identify customers likely to default within 90 days. Formulate the prediction time, target, allowable features, validation strategy, and at least three evaluation metrics.
2. A hospital has repeated records for the same patients. Explain why a random row-level split may be misleading and design a more appropriate evaluation approach.
3. A company wants to use a churn model to decide whom to contact with retention offers. Explain why high predicted churn risk does not necessarily identify customers who will benefit from the offer, and propose a suitable analytical approach.

Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes machine learning from artificial intelligence?

A. Machine learning is a broad field that includes all forms of reasoning

B. Machine learning is a set of methods that learn from data or interaction, whereas AI is the broader field

C. Artificial intelligence only concerns neural networks

D. Machine learning cannot be used for prediction

Correct Answer: B. Machine learning is a set of methods that learn from data or interaction, whereas AI is the broader field

Question 2

A model predicts whether a transaction is fraudulent or legitimate. What type of supervised task is this?

A. Regression

B. Clustering

C. Binary classification

D. Dimensionality reduction

Correct Answer: C. Binary classification

Question 3

Which situation is an example of data leakage?

A. Scaling features using training-set statistics

B. Using a customer’s post-cancellation information to predict cancellation at an earlier time

C. Evaluating a model on a chronological test set

D. Removing duplicate records before splitting data

Correct Answer: B. Using a customer’s post-cancellation information to predict cancellation at an earlier time

Question 4

What is the principal purpose of a test set?

A. To fit model parameters

B. To choose every hyperparameter repeatedly

C. To provide a final estimate of performance on unused data

D. To create labels automatically

Correct Answer: C. To provide a final estimate of performance on unused data

Question 5

Self-supervised learning differs from ordinary supervised learning because it

A. never uses an objective function

B. creates a training signal from the data itself rather than requiring externally supplied labels

C. can only be used for numerical prediction

D. requires an agent and delayed rewards

Correct Answer: B. creates a training signal from the data itself rather than requiring externally supplied labels

Question 6

In reinforcement learning, the quantity G_t = ∑_k=0^∞ γ^k r_t+k+1 represents

A. the discounted future return from time t

B. the classification accuracy at time t

C. the number of features in a state

D. the training-set mean squared error

Correct Answer: A. the discounted future return from time t

Question 7

If TP = 80 and FP = 20, what is precision?

A. 0.20

B. 0.50

C. 0.80

D. 0.90

Correct Answer: D. 0.90

Question 8

Why is chronological validation usually preferable for many forecasting problems?

A. It guarantees zero prediction error

B. It reproduces the fact that future observations should be predicted from past observations

C. It always creates balanced classes

D. It removes the need for a test set

Correct Answer: B. It reproduces the fact that future observations should be predicted from past observations

Question 9

Which statement about clustering is most accurate?

A. Clusters are always objectively real categories

B. Clusters depend partly on representation, distance measure, algorithm, and other choices

C. Clustering requires a labelled target for every observation

D. Clustering directly estimates causal effects

Correct Answer: B. Clusters depend partly on representation, distance measure, algorithm, and other choices

Question 10

A model has high accuracy because 99% of cases are negative, but it detects no positive cases. Which conclusion is most appropriate?

A. The model is necessarily excellent

B. Accuracy may be misleading, so positive-class metrics and decision costs should be examined

C. The model must be unsupervised

D. The test set should be discarded automatically

Correct Answer: B. Accuracy may be misleading, so positive-class metrics and decision costs should be examined