Model Evaluation and Improvement

Introduction

A machine-learning model is not valuable merely because it fits a training dataset. It is valuable when it makes reliable predictions on appropriate future data, performs acceptably across relevant groups and operating conditions, and supports decisions that are technically and ethically defensible. Model evaluation therefore extends beyond calculating a single accuracy score. It includes the design of data partitions, selection of metrics, comparison with meaningful baselines, control of overfitting, examination of errors, and communication of uncertainty and limitations.

A useful distinction is between model fitting and model evaluation. Fitting estimates parameters from data. Evaluation estimates how the fitted procedure is likely to perform on unseen data. If information from the evaluation data influences preprocessing, feature selection or hyperparameter choice, the evaluation is contaminated and usually optimistic.

This chapter presents a practical framework for robust evaluation and improvement of supervised-learning models. It covers train-validation-test strategies, cross-validation, metrics, baselines, hyperparameter optimisation, feature selection, ensemble methods, error analysis, class imbalance and explainability.

Learning Objectives

By the end of this chapter, you should be able to:

1. Design defensible train, validation and test procedures.
2. Select metrics that reflect the prediction task and business consequences.
3. Use cross-validation and baselines appropriately.
4. Distinguish model parameters from hyperparameters and optimise the latter without leakage.
5. Apply feature selection and ensemble methods critically.
6. Diagnose systematic errors, class imbalance and subgroup failures.
7. Use explainability techniques while recognising their limitations.
8. Document an evaluation process suitable for professional or regulated use.

1. Generalisation and the Evaluation Problem

Let a model be represented by f_θ(x), where x is an input vector and θ contains learned parameters. For a labelled observation (x_i,y_i), a loss function L(y_i,f_θ(x_i)) measures prediction error. The empirical risk on a dataset of size n is:

Here, R̂(θ) is the average observed loss, y_i is the true target, and f_θ(x_i) is the prediction. The objective of learning is commonly to minimise empirical risk, but the practical objective is low expected risk on future observations drawn from the target population:

The equality between training performance and future performance is not expected. A flexible model may memorise idiosyncrasies of the training sample. This is overfitting. A model that is too restricted to represent important structure may suffer underfitting. Evaluation estimates the generalisation gap between observed and future performance.

Evaluation is valid only when the evaluation data represent the intended deployment population and are not used, directly or indirectly, to make modelling decisions. Random splitting is often appropriate for independent observations, but it is inappropriate for many time-series, grouped or spatial datasets.

2. Train, Validation and Test Strategies

2.1 The three-way split

A standard workflow divides labelled data into:

• Training set: used to fit model parameters.
• Validation set: used to select algorithms, features, thresholds and hyperparameters.
• Test set: used once, or very rarely, for final unbiased assessment.

For example, a dataset may be divided into 60% training, 20% validation and 20% test data. The percentages are not universal rules. A small dataset may require cross-validation, whereas a very large dataset may need only modest validation and test samples.

All operations that learn from data must be fitted within the training portion. This includes imputation, scaling, encoding, feature selection and dimensionality reduction. A pipeline expresses this dependency explicitly:

training data → fit preprocessing → fit model → validate
                         ↓
                  transform validation

The test set must remain untouched until the modelling process is frozen. Repeatedly inspecting test results and modifying the model turns the test set into another validation set.

2.2 Stratification, grouping and time

For classification, stratified splitting preserves approximately the class proportions in each partition. It is useful when classes are imbalanced, although it cannot solve imbalance by itself.

If several rows belong to the same person, patient, customer, device or transaction sequence, ordinary random splitting may place related records in both training and test sets. The model may then appear to generalise while actually recognising entities. Group-aware splitting keeps all observations from one group in the same partition.

For forecasting, future observations must not influence past predictions. A time-ordered split or rolling-origin evaluation is appropriate:

train: months 1–12 → validate: month 13
train: months 1–13 → validate: month 14
train: months 1–14 → validate: month 15

The final test period should represent the intended future deployment horizon. Randomly shuffling time-series data is a common and serious form of leakage.

2.3 Data leakage

Data leakage occurs when information unavailable at prediction time enters model development. Examples include:

• scaling the full dataset before splitting;
• using a diagnosis recorded after the prediction date;
• selecting features using all labels before cross-validation;
• duplicating records across partitions;
• imputing missing values with statistics computed from validation or test data.

Leakage can produce high scores and poor real-world performance. A data dictionary and a prediction-time availability audit are valuable safeguards.

3. Cross-Validation

In k-fold cross-validation, the training data are divided into k approximately equal folds. Each fold is used once as validation data, while the remaining folds are used for fitting. If [[MATH{s_j}]] is the score from fold j, the estimated mean score is:

The sample standard deviation of fold scores is:

The mean describes typical performance and the standard deviation describes variation across splits. These are not automatically confidence intervals for population performance; observations may not be independent and additional uncertainty arises from dataset selection and model search.

Common variants include:

• Stratified k-fold: preserves class proportions.
• Repeated k-fold: repeats partitioning with different random seeds.
• Group k-fold: keeps groups intact.
• Leave-one-out: uses one observation for validation at a time; it can be computationally expensive and have high variance.
• Nested cross-validation: uses an inner loop for model selection and an outer loop for performance estimation.

Nested cross-validation is particularly useful when the dataset is small and many hyperparameters or models are compared. The inner loop selects the procedure; the outer loop estimates its performance without allowing the outer validation data to influence selection.

4. Metrics for Model Evaluation

No metric is universally best. Metric choice should reflect the target, class distribution, costs of errors and whether probabilities or decisions are required.

4.1 Classification metrics

For binary classification, a confusion matrix contains true positives (TP), true negatives (TN), false positives (FP) and false negatives (FN).

Accuracy is appropriate when classes and error costs are reasonably balanced. It can be misleading when one class is rare.

Precision answers: among predicted positives, how many are positive?

Recall, also called sensitivity or true-positive rate, answers: among actual positives, how many are detected?

Specificity is the true-negative rate. The false-positive rate is 1 − Specificity.

The F1 score is the harmonic mean of precision and recall:

It is defined when precision and recall are not both zero. F1 gives equal importance to precision and recall; a weighted F-score may be preferable when their importance differs.

A receiver operating characteristic (ROC) curve plots true-positive rate against false-positive rate over decision thresholds. The area under this curve, ROC-AUC, measures ranking ability across thresholds. A precision-recall curve is often more informative for rare positive classes because it focuses on positive prediction quality.

Probability quality should be evaluated with log loss or calibration measures. For binary outcomes with predicted probability p_i and label [[MATH{y_i ∈ {0,1}}]], log loss is:

Probabilities of zero or one require careful numerical handling. Log loss penalises confident incorrect predictions strongly. A calibration curve compares predicted probabilities with observed frequencies. A model can rank cases well but be poorly calibrated.

For multiclass problems, metrics may be macro-averaged, weighted by class support or micro-averaged. Macro averaging gives each class equal importance; weighted averaging gives more influence to common classes.

4.2 Regression metrics

For actual values y_i and predictions ŷ_i, mean absolute error is:

MAE is in the same units as the target and is relatively robust to outliers. Mean squared error is:

Root mean squared error is RMSE = √MSE, also in target units, but it gives greater weight to large errors. The coefficient of determination is:

Here, ȳ is the mean of the observed targets. R² compares the model with a mean-prediction baseline; it can be negative on test data when the model is worse than that baseline. Percentage errors should be used cautiously when targets are zero or close to zero.

5. Baselines and Statistical Comparisons

A baseline is a simple reference against which a model is judged. Examples include majority-class prediction, prior-period forecasting, mean regression prediction, linear regression or an existing operational rule. A sophisticated model that does not outperform a suitable baseline may not justify its cost or complexity.

Performance differences should be interpreted with uncertainty. Report fold-level results, confidence intervals where justified, and practical effect sizes. A statistically detectable improvement may be operationally irrelevant, while a modest average improvement may matter greatly in a high-volume or safety-critical system.

6. Hyperparameter Optimisation

Model parameters are learned during fitting; hyperparameters are specified before fitting. Examples include tree depth, regularisation strength, number of neighbours, learning rate and number of estimators.

A basic search procedure is:

1. Define a pipeline and an evaluation metric.
2. Specify plausible hyperparameter ranges.
3. Use cross-validation on the training data.
4. Select the configuration with the best validation criterion.
5. Refit the selected pipeline on all allowable training data.
6. Evaluate once on the held-out test set.

Grid search evaluates every combination in a finite grid. Random search samples combinations and is often more efficient when only a few hyperparameters strongly affect performance. Bayesian optimisation models the relationship between configurations and scores, selecting promising trials while accounting for previous results.

Search ranges must be scientifically defensible. Excessively broad searches increase computation and the risk of selection overfitting. The optimisation objective should reflect deployment needs, such as recall subject to a maximum false-positive rate, rather than an arbitrary default metric.

7. Feature Selection and Representation

Feature selection can reduce noise, computation, leakage risk and interpretability burden. It may be:

• Filter-based: uses associations such as correlation or mutual information before model fitting.
• Wrapper-based: evaluates subsets using model performance, such as recursive feature elimination.
• Embedded: performs selection during fitting, as with L1 regularisation or tree-based importance.

Selection must occur inside each training fold. Selecting features once using all data leaks validation information. Correlation with the target is not sufficient evidence of usefulness: a feature may be redundant, unstable, unavailable at prediction time or ethically unacceptable.

Feature engineering should be judged by out-of-sample performance and domain plausibility. High-dimensional data may require regularisation, dimensionality reduction or domain-informed representations. Feature selection is not identical to causal discovery; a predictive feature need not cause the outcome.

8. Ensemble Methods

Ensembles combine multiple models. Bagging trains models on resampled data and averages or votes across them, reducing variance. Random forests add random feature selection to bagged decision trees.

Boosting builds models sequentially, with later models focusing on errors or residuals from earlier models. Gradient boosting can achieve strong predictive performance but requires careful regularisation, validation and monitoring.

Stacking trains a meta-model to combine predictions from different base learners. To avoid leakage, meta-model training should use out-of-fold predictions rather than predictions generated on the same data used to fit base models.

Ensembles may improve accuracy but reduce simplicity, increase latency and complicate explanations. The correct choice depends on the cost of errors, governance requirements and maintenance capability.

9. Error Analysis and Class Imbalance

Aggregate metrics conceal important failure patterns. Error analysis should examine:

• false positives and false negatives;
• performance by class, subgroup, geography, device or time period;
• confidence or probability ranges;
• common feature patterns among failures;
• data quality and labelling problems;
• examples near the decision boundary.

For imbalanced classification, possible interventions include class-weighted loss, threshold adjustment, targeted data collection, under-sampling, over-sampling and synthetic methods such as SMOTE. Resampling must be performed inside the training folds. Validation and test sets should retain the natural deployment distribution unless a different evaluation distribution is explicitly justified.

Class weights alter the training objective but do not automatically produce calibrated probabilities. Threshold selection should be based on the relative costs of false positives and false negatives. In a medical screening context, high recall may be prioritised; in an expensive manual-review system, precision may matter more.

Fairness analysis should report subgroup performance and error rates where legally and ethically appropriate. Similar overall accuracy does not guarantee equitable treatment. Protected attributes may be needed for auditing even when they are excluded from prediction, subject to applicable law and governance controls.

10. Explainability and Defensible Use

Explainability techniques answer different questions. Global explanations describe overall model behaviour, while local explanations describe an individual prediction. Feature importance from a tree model, permutation importance, partial dependence, accumulated local effects and SHAP-style additive attributions are common tools.

Permutation importance measures the performance decrease after a feature’s values are randomly permuted. It can be misleading when features are strongly correlated. Partial-dependence plots show average model response as a feature varies, but they may create unrealistic combinations of feature values. Local surrogate explanations approximate a complex model near one instance; they are explanations of an approximation, not necessarily the model itself.

Explanations should be validated for stability, consistency and faithfulness. A visually persuasive explanation is not proof of causality. In regulated settings, documentation should include model purpose, training data, exclusions, metrics, subgroup results, known limitations, human oversight, versioning and monitoring procedures.

11. Illustrative Case: Predicting Customer Churn

Background

A telecommunications provider wants to identify customers likely to cancel within the next 30 days so that retention teams can contact them. The positive class is approximately 8% of customers.

Problem

A random row split would be inappropriate if multiple records per customer exist. In addition, monthly billing information must be restricted to information available before the prediction date.

Analysis

The team creates a customer-level, time-based split: earlier months for training, a later month for validation and the most recent month for testing. Preprocessing, feature selection and class weighting are contained in a pipeline. The majority-class baseline obtains 92% accuracy but zero recall for churners, demonstrating why accuracy alone is inadequate.

Approach

A logistic-regression model provides an interpretable baseline. A gradient-boosted tree model is then tuned using inner cross-validation. The selection criterion is recall subject to a precision requirement, because the retention team has limited capacity. Calibration is assessed before predicted probabilities are used for prioritisation. Error analysis compares performance across contract type and customer tenure.

Outcome

The boosted model improves ranking performance, but its default threshold generates too many calls. A higher threshold produces fewer, more precise interventions while retaining acceptable recall. The final test set is evaluated only after the threshold and model are frozen. Results are reported with confusion matrices, subgroup metrics, calibration plots and a clear statement that the model identifies association, not the cause of churn.

Learning Points

The case illustrates that evaluation design is inseparable from the operational decision. Temporal validity, customer-level grouping, baseline comparison, threshold selection, capacity constraints and subgroup analysis are all necessary for a defensible system.

12. A Professional Evaluation Protocol

A reproducible evaluation report should record:

1. Prediction target, prediction time and unit of analysis.
2. Inclusion and exclusion rules.
3. Data partitioning and leakage controls.
4. Preprocessing and feature-generation procedures.
5. Baselines and chosen metrics.
6. Cross-validation design and random seeds.
7. Hyperparameter search space and selection rule.
8. Test-set results with uncertainty and subgroup analysis.
9. Error examples and known failure modes.
10. Model, data and code versions.
11. Monitoring thresholds and retraining triggers.
12. Human oversight, privacy, fairness and security considerations.

After deployment, evaluation continues. Data drift, concept drift, changes in class prevalence and operational feedback can reduce performance. Monitoring should distinguish changes in input distributions from changes in the relationship between inputs and outcomes. Periodic back-testing with newly labelled data is essential.

Challenges and Limitations

Evaluation estimates performance; it does not guarantee future success. A test set may be unrepresentative, labels may be noisy, and deployment behaviour may change the data-generating process. Cross-validation can be computationally expensive and can still be invalid if the split structure is wrong. Metrics can conflict, and optimising one may harm another. Explainability methods may be unstable or misunderstood. Fairness criteria can conflict mathematically and must be interpreted in context. Finally, a model with strong predictive performance may still be unsuitable if its data collection, use or intervention causes unacceptable privacy or social harms.

Chapter Summary

Robust model evaluation begins with a clear definition of the prediction task and deployment population. Training, validation and test data must be separated according to the data-generating structure, with group-aware or time-aware procedures where necessary. Cross-validation estimates performance more efficiently than a single split, while nested cross-validation reduces selection bias during extensive model comparison.

Metrics must reflect class balance, error costs and whether decisions, rankings or calibrated probabilities are required. Baselines establish practical value. Hyperparameter optimisation, feature selection and preprocessing must occur inside the evaluation procedure to prevent leakage. Ensembles can improve predictive performance but introduce complexity. Error analysis reveals failures hidden by aggregate scores, and imbalance treatments must preserve realistic validation and test distributions. Explainability supports communication and governance but does not establish causality. A defensible system combines quantitative evaluation, subgroup analysis, documentation, monitoring and human accountability.

Key Terms

Generalisation: Performance on unseen data from the target population.

Overfitting: Learning noise or peculiarities that do not generalise.

Underfitting: Using a model too limited to capture important structure.

Data leakage: Unavailable or evaluation information entering model development.

Training set: Data used to estimate model parameters.

Validation set: Data used for model and hyperparameter selection.

Test set: Held-out data used for final assessment.

Cross-validation: Repeated fitting and validation across data folds.

Stratification: Preserving approximate class proportions across partitions.

Baseline: A simple reference model or rule.

Precision: Proportion of predicted positives that are true positives.

Recall: Proportion of actual positives detected.

Calibration: Agreement between predicted probabilities and observed frequencies.

Hyperparameter: A modelling choice set outside ordinary parameter fitting.

Feature selection: Choosing a subset of input variables.

Bagging: Combining models trained on resampled data.

Boosting: Sequentially fitting models to improve previous errors.

Error analysis: Systematic investigation of incorrect predictions.

Class imbalance: Unequal prevalence of target classes.

Explainability: Methods for interpreting model behaviour or predictions.

Review Questions

Short Answer Questions

1. What is the purpose of a held-out test set?
2. Why should preprocessing be fitted within each training fold?
3. Distinguish precision from recall.
4. What is the role of a baseline model?
5. Why can accuracy be misleading for rare-event classification?

Long Answer Questions

1. Explain how you would design an evaluation strategy for a time-series forecasting problem.
2. Compare grid search, random search and Bayesian hyperparameter optimisation.
3. Discuss the advantages and limitations of MAE, MSE, RMSE and R².
4. Explain how class imbalance affects model evaluation and describe suitable responses.
5. Discuss the role and limitations of global and local explainability methods.

Analytical / Application Questions

1. A fraud detector has high accuracy but detects only 10% of fraudulent transactions. Diagnose the problem and propose a revised evaluation and threshold strategy.
2. A hospital dataset contains five records per patient, and records are randomly divided into training and test sets. Explain the likely bias and design a better validation scheme.
3. A model’s ROC-AUC improves while its probability calibration worsens. Explain how both findings can be true and recommend additional evaluation steps.

Multiple Choice Questions with Answers

Question 1

Which dataset should normally remain untouched until the modelling procedure is finalised?

A. Training set

B. Validation set

C. Test set

D. Resampling set

Correct Answer: C. Test set

Question 2

Which metric answers the question: among predicted positives, how many are actually positive?

A. Recall

B. Precision

C. Specificity

D. Accuracy

Correct Answer: B. Precision

Question 3

Why is group-aware splitting important when several records belong to the same customer?

A. It increases the number of features

B. It prevents related records from appearing in both training and evaluation partitions

C. It guarantees perfect calibration

D. It removes all class imbalance

Correct Answer: B. It prevents related records from appearing in both training and evaluation partitions

Question 4

In k-fold cross-validation, what is commonly reported across the fold scores?

A. Only the largest score

B. The mean and variation of the scores

C. The training accuracy only

D. The number of input features

Correct Answer: B. The mean and variation of the scores

Question 5

Which operation is most likely to cause leakage if performed before cross-validation?

A. Storing the random seed

B. Selecting features using all labels

C. Recording the sample size

D. Defining the target variable

Correct Answer: B. Selecting features using all labels

Question 6

Which metric strongly penalises confident incorrect probability predictions?

A. Log loss

B. Accuracy

C. Specificity

D. R²

Correct Answer: A. Log loss

Question 7

What is the main purpose of a baseline model?

A. To guarantee production success

B. To provide a reference for judging whether added complexity is useful

C. To eliminate the need for a test set

D. To replace error analysis

Correct Answer: B. To provide a reference for judging whether added complexity is useful

Question 8

Which method combines models trained on resampled datasets to reduce variance?

A. Bagging

B. Thresholding

C. Calibration

D. Imputation

Correct Answer: A. Bagging

Question 9

For a rare positive class, which curve is often especially informative because it focuses on positive prediction quality?

A. Precision-recall curve

B. Learning curve only

C. Residual plot

D. Normal probability plot

Correct Answer: A. Precision-recall curve

Question 10

What is a major limitation of feature-importance explanations?

A. They always prove causality

B. They may be unstable or misleading when features are correlated

C. They can be computed only for linear models

D. They eliminate the need for validation

Correct Answer: B. They may be unstable or misleading when features are correlated