6. Classification and Probabilistic Prediction

Introduction and Context

Many decisions in data science involve assigning an observation to a category rather than predicting a numerical quantity. A bank may classify a transaction as legitimate or fraudulent, a hospital may classify a patient according to risk, and a telecommunications company may predict whether a customer will churn. These are classification problems. Although the final output is often a label, a useful classification system usually produces more than a label: it estimates the probability of each possible class and supports a decision under particular operational costs.

Classification therefore combines statistical modelling, pattern recognition and decision analysis. A model estimates how features are related to class membership; a decision rule converts estimated probabilities or scores into actions. Separating these two stages is essential. A probability of 0. sixty? No—probabilities must be represented precisely: a predicted probability of 0.60 for default does not automatically mean that a loan must be rejected. The appropriate action depends on the threshold, the costs of errors, regulatory constraints and the capacity of the organisation to investigate cases.

This chapter develops four important classification approaches: logistic regression, nearest-neighbour methods, naïve Bayes and support vector machines. It also examines binary and multiclass classification, probability interpretation, threshold selection, calibration, evaluation and cost-sensitive decision-making. The emphasis is not only on fitting models, but on understanding when their outputs can be trusted and how they should be used in practice.

Learning Objectives

By the end of this chapter, a student should be able to:

1. Distinguish classification from regression and explain the relationship between prediction and decision-making.

2. Formulate binary and multiclass classification problems using suitable notation.

3. Explain and apply logistic regression, k-nearest neighbours, naïve Bayes and support vector machines.

4. Interpret class probabilities, decision scores and classification thresholds.

5. Assess models using confusion matrices, precision, recall, specificity, F-scores, ROC curves and precision–recall curves.

6. Explain calibration and identify when a probability estimate is operationally useful.

7. Select thresholds using asymmetric costs, class prevalence and organisational constraints.

8. Recognise common sources of error, bias, data leakage and inappropriate model use.

Classification as Estimation Followed by Action

Let an observation be represented by a feature vector $x=(x_1,x_2,\ldots,x_p)$ and let $Y$ denote its class. In binary classification, $Y$ commonly takes values 0 and 1. The model estimates either a conditional probability,

$$P(Y=1\mid X=x),$$

or a score that ranks observations according to their likely class. A decision rule then assigns a label, for example:

$$\hat{Y}=1 \quad \text{if} \quad \hat{P}(Y=1\mid x)\geq t,$$

where $t$ is a threshold. The familiar threshold of 0.5 is only one possible choice and is not universally appropriate.

In multiclass classification, $Y$ belongs to one of $K$ classes. A probabilistic model may produce a vector

$$\hat{p}(x)=(\hat{p}_1(x),\hat{p}_2(x),\ldots,\hat{p}_K(x)),$$

where each probability is non-negative and the probabilities sum to one. The maximum-probability rule chooses the class with the largest estimated probability, although other decisions may be preferable when the consequences of errors differ across classes.

A useful distinction is that between a statistical probability and a decision score. Logistic regression and naïve Bayes are naturally probabilistic, although their probabilities may require calibration. A support vector machine principally produces a margin or decision function. Nearest neighbours can estimate probabilities from the proportion of nearby observations, but those estimates depend strongly on the choice of neighbourhood size and distance metric.

Important Terminology

A class is a category to be predicted. The positive class is the class designated as 1 for a binary task; its designation is a modelling and operational choice, not an inherent property of the data. A decision boundary separates regions assigned to different classes. A score ranks observations by evidence for a class. Prevalence is the proportion of cases belonging to a class in a specified population. Class imbalance occurs when some classes are substantially less common than others.

A true positive is a positive case correctly identified; a true negative is a negative case correctly rejected. A false positive is a negative case incorrectly labelled positive, while a false negative is a positive case missed by the classifier. These outcomes form the confusion matrix and provide the basis for many evaluation measures.

Logistic Regression: Modelling Class Probability

Logistic regression is a foundational method for binary classification because it combines a clear probabilistic interpretation with a relatively simple functional form. It models the log-odds of the positive class as a linear function of the predictors:

$$\log\left(\frac{P(Y=1\mid X=x)}{1-P(Y=1\mid X=x)}\right)=\beta_0+\beta_1x_1+\cdots+\beta_px_p.$$

Solving for the probability gives the logistic, or sigmoid, function:

$$P(Y=1\mid x)=\frac{1}{1+e^{-\left(\beta_0+\sum_j\beta_jx_j\right)}}.$$

The coefficients are estimated, commonly by maximum likelihood. A coefficient $\beta_j$ represents the change in log-odds associated with a one-unit increase in $x_j$, holding other predictors constant. The quantity $e^{\beta_j}$ is an odds ratio. For example, if $e^{\beta_j}=1.4$, a one-unit increase in the predictor multiplies the odds of the positive class by 1.4, subject to the model assumptions.

The linear predictor produces a linear boundary in the original feature space. This can be restrictive when class membership depends on interactions or nonlinear patterns. Polynomial terms, interaction terms, transformations or regularisation can improve performance, but they also affect interpretation. L1 regularisation can encourage sparse models by shrinking some coefficients to zero; L2 regularisation reduces coefficient magnitude and can help with multicollinearity and overfitting.

Multiclass logistic regression can be implemented using a multinomial model, which estimates class probabilities jointly, or using one-versus-rest models, in which one classifier is trained for each class. The multinomial formulation is often attractive when the classes are mutually exclusive and a coherent probability distribution is required.

Nearest Neighbours: Local Evidence Rather Than a Global Equation

The k-nearest neighbours (k-NN) method does not first estimate a global parametric relationship. To classify a new observation, it identifies the $k$ training observations closest to it and uses their labels. In a binary problem, the majority class among the neighbours is commonly selected. A local probability estimate can be obtained from the proportion of neighbours in each class:

$$\hat{P}(Y=c\mid x)=\frac{1}{k}\sum_{i\in N_k(x)}I(y_i=c).$$

Small values of $k$ produce flexible boundaries and can capture local structure, but they are sensitive to noise. Large values produce smoother, more stable predictions but may obscure meaningful local patterns. The distance metric is equally important. Euclidean distance is inappropriate when features have very different scales; standardisation is generally required for continuous variables. Categorical variables, missing values and mixed data types require suitable distance definitions or preprocessing.

The computational burden of k-NN is shifted from training to prediction because distances to stored observations must be calculated. For large datasets, indexing structures or approximate-neighbour methods may be needed. k-NN can be effective where similar cases have similar outcomes, but it suffers in high-dimensional spaces. As dimensions increase, observations tend to become relatively equidistant, a phenomenon associated with the curse of dimensionality.

Naïve Bayes: A Generative Probabilistic Classifier

Bayes’ theorem expresses the posterior probability of a class as

$$P(Y=c\mid X=x)=\frac{P(X=x\mid Y=c)P(Y=c)}{P(X=x)}.$$

Naïve Bayes estimates the prior class probability $P(Y=c)$ and the class-conditional feature distribution $P(X\mid Y=c)$. Its defining assumption is conditional independence of features given the class:

$$P(X\mid Y=c)=\prod_{j=1}^{p}P(X_j=x_j\mid Y=c).$$

This assumption is often unrealistic—for example, words in a document are not genuinely independent—but the method can still classify effectively, particularly with sparse text data. Common variants include Gaussian naïve Bayes for continuous features, multinomial naïve Bayes for count data such as word frequencies, and Bernoulli naïve Bayes for binary features.

A practical difficulty arises when a feature value has zero frequency in a class, causing a product of probabilities to become zero. Smoothing, such as Laplace smoothing, assigns small non-zero probabilities. Computation is normally performed on the log scale to avoid numerical underflow:

$$\log P(Y=c\mid x)\propto \log P(Y=c)+\sum_j\log P(X_j=x_j\mid Y=c).$$

Naïve Bayes is fast, requires relatively little training data and provides a useful baseline. Its probability estimates may nevertheless be poorly calibrated, especially when features are correlated, because the independence assumption can make posterior probabilities excessively confident.

Support Vector Machines: Margins and Kernels

A support vector machine (SVM) seeks a separating hyperplane that maximises the margin between classes. For a linear classifier, the decision function is

$$f(x)=w^Tx+b.$$

The sign of $f(x)$ determines the predicted class, while the magnitude indicates distance from the separating hyperplane in the transformed geometric representation. Observations closest to the boundary are the support vectors and exert the greatest influence on the fitted solution.

Real datasets are rarely perfectly separable. The soft-margin SVM permits violations and balances margin width against classification errors through a penalty parameter, usually denoted by $C$. A large $C$ places greater emphasis on avoiding training errors and may produce a less tolerant, more complex boundary; a smaller $C$ allows more violations in exchange for a wider margin.

Kernel functions allow SVMs to represent nonlinear boundaries without explicitly constructing a high-dimensional feature space. The radial basis function kernel is widely used, but its parameters must be tuned carefully. Features should generally be scaled because the geometry of the margin depends on feature magnitudes. Standard SVM outputs are decision scores rather than calibrated probabilities. Probability estimates can be obtained through additional calibration procedures, but they should not be interpreted as direct probabilities merely because software labels them “probability.”

Choosing Between Classification Models

No classifier is uniformly best. Logistic regression is a strong choice when interpretability, linear effects and probability estimates are important. k-NN is attractive when local similarity is meaningful and the dataset is not excessively large or high-dimensional. Naïve Bayes is particularly useful for rapid baselines and text or count-based features. SVMs can perform well in high-dimensional spaces and in problems with complex boundaries, especially when the sample size is moderate.

Model selection should be based on a complete workflow rather than a single performance number. The analyst should define the target and prediction time, split data into training, validation and test sets, preprocess within the training folds, tune hyperparameters using cross-validation, assess discrimination and calibration, and finally examine operational consequences. Class weights, resampling and threshold adjustment may address imbalance, but they solve different problems. Reweighting changes the fitting objective; threshold adjustment changes the action rule after scores have been produced.

Evaluating Classification Performance

The confusion matrix provides the basic accounting framework. For a binary classifier:

• Accuracy is $(TP+TN)/(TP+TN+FP+FN)$.
• Precision, or positive predictive value, is $TP/(TP+FP)$.
• Recall, or sensitivity, is $TP/(TP+FN)$.
• Specificity is $TN/(TN+FP)$.
• F1 score is the harmonic mean of precision and recall: $2PR/(P+R)$.

Accuracy can be misleading under class imbalance. If only 1% of transactions are fraudulent, a model predicting “legitimate” for every transaction achieves 99% accuracy but has zero recall for fraud. Precision is affected by prevalence, whereas recall describes the proportion of actual positives detected. The appropriate metric depends on the decision context.

A receiver operating characteristic (ROC) curve plots true-positive rate against false-positive rate across thresholds. The area under the ROC curve (ROC-AUC) measures ranking ability across thresholds, but it does not establish that probabilities are accurate. In highly imbalanced settings, a precision–recall curve may provide a more informative view of performance on the positive class. Metrics should be reported with confidence intervals or variability estimates where decisions are consequential.

For multiclass tasks, performance can be summarised using a multiclass confusion matrix and macro, weighted or micro averaging. Macro averaging gives each class equal importance; weighted averaging accounts for class frequencies; micro averaging aggregates individual predictions. These summaries can conceal poor performance in a rare or high-risk class, so per-class results remain important.

Probability, Thresholds and Calibration

A classifier can be evaluated for discrimination and calibration. Discrimination concerns whether cases with higher risk tend to receive higher scores than cases with lower risk. Calibration concerns agreement between predicted probabilities and observed frequencies. If a well-calibrated model assigns a probability of 0.70 to many cases, approximately 70% of comparable cases should be positive over repeated groups.

Calibration can be examined using reliability diagrams, calibration curves and proper scoring rules such as log loss and the Brier score. The Brier score for a binary outcome is the mean squared difference between predicted probability and observed outcome:

$$\text{Brier score}=\frac{1}{n}\sum_{i=1}^{n}(\hat{p}_i-y_i)^2.$$

A model can rank cases well while being poorly calibrated. For example, it may consistently assign probabilities that are too extreme. Calibration methods such as Platt scaling and isotonic regression learn a mapping from model scores to probabilities. The calibration mapping must be fitted on data separate from the data used to train the base model, otherwise optimistic estimates and leakage may result.

The threshold should be chosen in relation to the purpose of prediction. If missing a disease case is much more harmful than conducting an unnecessary follow-up test, a lower threshold may be justified. If investigative capacity is limited, the organisation may choose a threshold that produces a manageable number of alerts. Thresholds should be validated under the population and prevalence expected during deployment.

Cost-Sensitive Classification and Decision Theory

Suppose a decision-maker must choose between actions $a_1$ and $a_0$, such as intervention and no intervention. Let $C_{FP}$ be the cost of a false positive and $C_{FN}$ the cost of a false negative. A simple expected-cost rule predicts the positive action when

$$C_{FP}(1-p)<C_{FN}p,$$

where $p=P(Y=1\mid x)$. Rearranging gives the threshold

$$p>\frac{C_{FP}}{C_{FP}+C_{FN}}.$$

This expression assumes that the probabilities are calibrated and that correct decisions have zero cost. Real applications may include intervention costs, benefits, capacity restrictions and multiple actions, but the principle remains: probabilities should be combined with consequences, not treated as decisions by themselves.

Cost estimates are often uncertain. They may include financial loss, clinical harm, customer inconvenience, legal exposure or reputational damage. Sensitivity analysis should therefore examine how decisions change when costs, prevalence or capacity assumptions vary. In regulated settings, a cost-minimising rule cannot override requirements for fairness, explainability, human review or due process.

An Illustrative Case: Prioritising Insurance Fraud Investigations

Background

Consider an illustrative insurance organisation that receives a large number of motor-claim submissions. It cannot investigate every claim, so it wants to rank claims according to the likelihood of fraud and refer a manageable number to investigators. Historical records include claim amount, claimant history, repair characteristics, timing, location and selected text-derived indicators.

Situation and Problem

Fraud is relatively uncommon. A model that maximises accuracy may label nearly every claim as legitimate. Conversely, an aggressive fraud detector may generate too many false alarms, burden investigators and inconvenience legitimate customers. The organisation needs both useful ranking and a defensible intervention policy.

Analysis and Approach

The analytics team first defines the prediction point: only information available when the claim is submitted may be used. Claims resolved after investigation are used as labels, while unresolved or systematically uninvestigated claims are treated cautiously because they may create label bias. The data are divided chronologically so that testing resembles future deployment. Preprocessing, imputation and feature scaling are performed inside cross-validation folds.

Logistic regression is developed as an interpretable baseline. An SVM and a gradient-based benchmark may be considered, but the principal comparison focuses on logistic regression, k-NN and naïve Bayes. The team evaluates precision among the top 5% of ranked claims, recall, calibration and investigator workload rather than accuracy alone. Because k-NN becomes unstable in the high-dimensional feature space, it is not selected. Naïve Bayes provides a fast text-feature baseline but is overconfident when correlated indicators are included. Logistic regression produces slightly weaker ranking than the best alternative but substantially clearer explanations and better calibrated probabilities after validation-based calibration.

The final decision rule is not simply $p\geq0.5$. Investigators can review only 1,000 claims per month, and false negatives are costly, so the organisation selects a threshold and ranking policy that respects this capacity. A human investigator makes the final determination; the model triggers prioritisation rather than automatically denying a claim.

Outcome

The system is monitored using monthly precision, recall, calibration, subgroup performance and changes in claim prevalence. Investigators record reasons for confirmed and rejected referrals, allowing the organisation to detect feedback loops. If investigation is concentrated in particular geographic or demographic groups, the team examines whether this reflects genuine risk, data quality problems or historical enforcement bias.

Learning Points

The case illustrates five principles. First, the prediction target and information availability must be defined before modelling. Second, ranking performance does not guarantee calibrated probabilities. Third, a threshold is an operational choice shaped by costs and capacity. Fourth, class imbalance makes accuracy inadequate as the principal metric. Finally, human oversight and monitoring are essential when model outputs affect customers.

Challenges, Risks and Common Mistakes

Several errors recur in classification projects. Data leakage occurs when a feature contains information created after the prediction point, such as a post-treatment diagnosis or an investigation outcome. Randomly splitting temporally ordered data can also produce overly optimistic results. Preprocessing the full dataset before cross-validation allows information from validation observations to influence the training process.

Another mistake is treating the 0.5 threshold as a scientific constant. It is appropriate only under particular assumptions about probabilities, costs and class priors. Analysts also confuse a high ROC-AUC with useful probability estimates, or report only aggregate performance while hiding failure on a minority class. Oversampling before data splitting can place near-duplicates in both training and test data. Class weights and resampling may improve minority recall but can alter probability interpretation and should be followed by calibration assessment.

Model instability, concept drift and prevalence shift present additional risks. A classifier trained on one population may be poorly calibrated in another. Monitoring should track not only inputs and accuracy, but also score distributions, calibration, subgroup outcomes, false-positive burden and changes in the meaning of the target.

Ethical, Governance and Professional Considerations

Classification systems can reproduce historical discrimination when labels reflect unequal access, enforcement or reporting. Protected attributes may be excluded from the model yet remain inferable through proxies such as location or purchasing patterns. Fairness assessment should therefore examine relevant group-specific error rates, calibration and opportunity outcomes, while recognising that fairness criteria may conflict mathematically and operationally.

High-impact decisions require documentation of the target definition, data provenance, model version, training period, evaluation design, threshold rationale and limitations. Individuals affected by automated decisions may require notice, explanation, appeal or human review under applicable law and organisational policy. Privacy protection includes data minimisation, access control, retention limits and careful treatment of sensitive features. Professional practice requires that analysts communicate uncertainty and avoid presenting predictions as facts.

Current Developments and Best Practice

Modern classification systems increasingly combine classical models with automated feature engineering, representation learning and ensemble methods. Nevertheless, simpler models remain valuable because their calibration, stability and explanations can be easier to govern. Automated machine-learning platforms can compare algorithms and tune hyperparameters, but they do not define a valid target, detect every leakage pathway or decide whether deployment is ethically acceptable.

A sound professional workflow is to:

1. Define the decision, outcome, prediction horizon and affected population.

2. Establish a simple baseline and a meaningful performance criterion.

3. Split data according to deployment conditions and perform all preprocessing within training folds.

4. Compare discrimination, calibration, subgroup performance, cost and operational workload.

5. Select thresholds using explicit costs, benefits and capacity constraints.

6. Validate probability estimates independently of model training.

7. Document assumptions and retain human review where consequences are substantial.

8. Monitor performance and recalibrate or retrain when data or prevalence changes.

Chapter Summary

Classification assigns observations to categories, but responsible classification separates probability estimation from the final decision. Logistic regression models log-odds and offers interpretable coefficients; k-nearest neighbours uses local similarity; naïve Bayes applies Bayes’ theorem with a conditional-independence assumption; and support vector machines optimise separating margins. Each method has distinctive assumptions concerning geometry, feature distributions, dimensionality and interpretability.

Confusion-matrix measures describe different kinds of error, while ROC-AUC and precision–recall analysis assess ranking across thresholds. A probability is useful only when its interpretation is justified; calibration evaluates whether predicted frequencies correspond to observed frequencies. Thresholds should reflect costs, prevalence, capacity and professional obligations rather than default software settings. Data leakage, class imbalance, distribution shift, unfairness and poor monitoring can undermine an apparently accurate model. Effective deployment therefore requires technical evaluation, decision analysis, governance and continuing review.

Key Terms

Classification: Prediction of a categorical outcome or class label.

Binary classification: Classification involving two mutually exclusive classes.

Multiclass classification: Classification involving more than two possible classes.

Log-odds: The logarithm of the odds of an event, commonly modelled linearly in logistic regression.

Logistic regression: A model that estimates binary or multiclass probabilities using a logistic or related link function.

k-nearest neighbours: A method that predicts from the labels of the closest training observations.

Naïve Bayes: A Bayesian classifier that assumes conditional independence of features given the class.

Support vector machine: A classifier that seeks a separating boundary with a large margin.

Decision boundary: The geometric or functional boundary separating predicted classes.

Threshold: A cutoff applied to a probability or score to determine an action or label.

Calibration: Agreement between predicted probabilities and observed event frequencies.

Prevalence: The proportion of a population belonging to a specified class.

Precision: The proportion of predicted positives that are actually positive.

Recall: The proportion of actual positives that are detected.

Specificity: The proportion of actual negatives correctly identified.

F1 score: The harmonic mean of precision and recall.

ROC-AUC: The area under the receiver operating characteristic curve, measuring ranking discrimination across thresholds.

Class imbalance: Unequal representation of classes, especially when one class is rare.

Cost-sensitive classification: Classification in which errors and actions are evaluated using unequal costs or benefits.

Data leakage: Unintended use of information unavailable at prediction time or improperly shared across evaluation partitions.

Review Questions

Short Answer Questions

1. What is the distinction between a classifier’s estimated probability and its final decision?

2. Why is feature scaling important for k-nearest neighbours and support vector machines?

3. What does a logistic regression coefficient represent in terms of odds?

4. Define calibration and explain how it differs from discrimination.

5. Why can accuracy be misleading when classes are highly imbalanced?

Long Answer Questions

1. Explain the assumptions, outputs and practical strengths and weaknesses of logistic regression, k-nearest neighbours, naïve Bayes and support vector machines.

2. Discuss the construction and interpretation of a confusion matrix and the circumstances in which precision, recall, specificity and F1 score are useful.

3. Explain how calibration is assessed and why a high ROC-AUC does not guarantee reliable probabilities.

4. Describe a complete workflow for developing and validating a multiclass classification model.

5. Discuss how data leakage, temporal splitting, preprocessing and class imbalance affect the credibility of model evaluation.

Analytical / Application Questions

1. A disease-screening model has high recall but low precision. Explain the likely operational consequences and propose two threshold or workflow changes.

2. A fraud model assigns probabilities of 0.90 to many claims, but only 0.65 of comparable claims are later confirmed as fraudulent. Diagnose the problem and recommend an evaluation and correction strategy.

3. An organisation can investigate only 2% of cases, while the cost of missing a positive case is five times the cost of investigating a negative case. Explain how you would combine ranking, calibration, threshold selection and capacity constraints in the deployment policy.

22. Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes a classification probability from a classification decision?

A. A probability is always a class label, whereas a decision is always continuous.

B. A probability expresses estimated uncertainty, while a decision applies a rule that may depend on a threshold and costs.

C. A probability can be produced only by an SVM, whereas a decision can be produced by any model.

D. A probability is meaningful only when the classes are perfectly balanced.

Correct Answer: B. A probability expresses estimated uncertainty, while a decision applies a rule that may depend on a threshold and costs.

Question 2

In logistic regression, exp(βj) is interpreted, holding other variables constant, as the change in the:

A. Class prevalence for a one-unit increase in xj.

B. Probability scale for a one-unit increase in xj.

C. Odds of the positive class associated with a one-unit increase in xj.

D. Classification threshold associated with xj.

Correct Answer: C. Odds of the positive class associated with a one-unit increase in xj.

Question 3

A k-nearest neighbours classifier gives probabilities by:

A. Calculating the proportion of neighbouring observations in each class.

B. Estimating a separating hyperplane with maximum margin.

C. Assuming conditional independence of all features.

D. Exponentiating logistic regression coefficients.

Correct Answer: A. Calculating the proportion of neighbouring observations in each class.

Question 4

What is the principal modelling assumption of naïve Bayes?

A. All classes have equal prevalence.

B. Features are conditionally independent given the class.

C. The decision boundary must be linear in the original feature space.

D. Every observation must have exactly the same distance from its neighbours.

Correct Answer: B. Features are conditionally independent given the class.

Question 5

In a soft-margin SVM, increasing C generally places greater emphasis on:

A. Allowing more training violations.

B. Avoiding training classification errors, potentially at the expense of a narrower margin.

C. Calibrating posterior probabilities.

D. Reducing the number of classes.

Correct Answer: B. Avoiding training classification errors, potentially at the expense of a narrower margin.

Question 6

A model has excellent ROC-AUC but systematically predicts probabilities that are too extreme. Which property is deficient?

A. Discrimination.

B. Specificity.

C. Calibration.

D. Feature scaling.

Correct Answer: C. Calibration.

Question 7

If the cost of a false negative is much higher than the cost of a false positive, a rational binary decision rule will generally:

A. Use a lower threshold for predicting the positive action.

B. Use a higher threshold for predicting the positive action.

C. Ignore predicted probabilities.

D. Maximise accuracy regardless of costs.

Correct Answer: A. Use a lower threshold for predicting the positive action.

Question 8

Why may precision–recall analysis be especially informative for a rare positive class?

A. It removes the need for a test set.

B. It focuses on positive predictive performance and detection as the threshold changes.

C. It guarantees calibrated probabilities.

D. It assumes that all classes have equal prevalence.

Correct Answer: B. It focuses on positive predictive performance and detection as the threshold changes.

Question 9

Which practice most directly prevents preprocessing information from leaking from validation data into model training?

A. Scaling the complete dataset before cross-validation.

B. Fitting preprocessing transformations separately within each training fold.

C. Using accuracy as the only evaluation metric.

D. Increasing the number of nearest neighbours.

Correct Answer: B. Fitting preprocessing transformations separately within each training fold.

Question 10

In the illustrative insurance case, why was automatic claim rejection inappropriate?

A. Classification models cannot produce rankings.

B. The model had no features available at submission time.

C. The model was intended to prioritise human investigation, and errors could adversely affect legitimate customers.

D. Fraud is never a classification problem.

Correct Answer: C. The model was intended to prioritise human investigation, and errors could adversely affect legitimate customers.