Predictive Analytics and Machine Learning

Introduction and context

Predictive analytics uses historical data, statistical methods and machine-learning algorithms to estimate future or unknown outcomes. In business, predictions may support credit decisions, sales forecasting, customer retention, inventory planning, fraud detection and workforce scheduling. Machine learning is not a replacement for business judgment: it is a disciplined process for extracting patterns from data and using those patterns to support decisions.

A predictive project normally follows this sequence:

1. Define the business decision and prediction target.
2. Assemble and understand relevant data.
3. Prepare features and the target variable.
4. Split data into training and testing samples.
5. Select and train a suitable model.
6. Evaluate performance using metrics appropriate to the decision.
7. Interpret, validate and deploy the model.
8. Monitor performance, fairness, data quality and drift.

A central principle is that a model must be evaluated on data that were not used to fit it. Otherwise, reported performance may reflect memorisation rather than useful generalisation.

Learning objectives

By the end of this chapter, you should be able to:

• distinguish regression, classification and clustering;
• prepare numerical and categorical features responsibly;
• separate predictors from a target variable and avoid data leakage;
• perform a train-test split and explain its purpose;
• build introductory models using scikit-learn;
• calculate and interpret common evaluation metrics;
• explain overfitting, underfitting and cross-validation;
• interpret model coefficients, feature importance and predictions cautiously;
• identify ethical, governance and operational risks in predictive systems.

1. The predictive modelling framework

In supervised learning, a dataset contains observations with known outcomes. Let the predictors be represented by a feature vector x and the outcome by y. A model estimates a function f such that:

For a new observation, the fitted model produces a prediction y_hat. The approximation is imperfect because business outcomes are affected by unmeasured factors, random variation, data errors and changes in the environment.

In unsupervised learning, outcomes are not supplied. The algorithm searches for structure in the predictors, for example groups of customers with similar behaviour. Clustering is therefore exploratory rather than a direct prediction of a labelled target.

A useful distinction is between a model’s statistical objective and the business decision. A model may predict churn accurately but still be commercially unhelpful if retention offers cost more than the customers saved.

2. Preparing business data

2.1 Defining observations and the target

Each row should represent one clearly defined observation, such as one customer at a particular date or one transaction. The target must be defined before inspecting future information. For example, a churn target might mean “the customer cancels within 30 days after the observation date.” A target defined using the entire customer history can accidentally include information unavailable when the prediction would have been made.

Features are variables available at prediction time. A feature such as “number of support calls in the previous 30 days” may be valid; “cancellation reason” is not valid if it is recorded after churn.

2.2 Data types and missing values

Numerical variables may be continuous, such as revenue, or discrete, such as number of orders. Categorical variables include region, channel and product type. Missingness may arise from non-response, system failure or the fact that a question was not applicable. Missing values should not be replaced blindly. The reason for missingness may itself be informative, but treating missingness as a signal can also encode operational or social bias.

Common approaches include:

• median imputation for skewed or noisy numerical variables;
• mean imputation when a distribution is reasonably symmetric;
• a separate category such as “Unknown” for some categorical variables;
• model-based or time-aware imputation when justified;
• adding a missingness indicator when the absence of a value has meaning.

Imputation values must be learned from the training data only. Calculating a median using the full dataset allows test-set information to influence training and is a form of leakage.

2.3 Encoding categorical variables

Most algorithms require numerical input. One-hot encoding represents a categorical variable with indicator columns. If the variable Channel has categories Online, Store and Partner, one-hot encoding creates binary columns. A model can then estimate separate effects without falsely implying that Store is numerically between Online and Partner.

For a variable with k categories, retaining all one-hot columns with an intercept can create exact linear dependence in ordinary least squares. Dropping one reference category avoids this issue. Many modern implementations can handle redundant columns numerically, but a transparent design still requires awareness of the reference group.

2.4 Scaling

Standardisation transforms a feature using:

where x is an observed value, μ is the training-set mean and σ is the training-set standard deviation. The transformed value has no physical unit. Scaling is especially important for distance-based methods such as k-nearest neighbours and k-means, and for regularised linear models. Tree-based models generally do not require scaling because their splits depend on order rather than distance.

The mean and standard deviation must be fitted on training data and then applied unchanged to validation, test and production data.

2.5 A reproducible preprocessing pipeline

A pipeline connects preprocessing and modelling so that transformations are learned only from the appropriate sample. A typical scikit-learn pattern is:

from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression

numeric_features = ["income", "tenure_months"]
categorical_features = ["region", "channel"]

numeric_pipe = Pipeline([
    ("impute", SimpleImputer(strategy="median")),
    ("scale", StandardScaler())
])

categorical_pipe = Pipeline([
    ("impute", SimpleImputer(strategy="most_frequent")),
    ("encode", OneHotEncoder(handle_unknown="ignore"))
])

preprocess = ColumnTransformer([
    ("num", numeric_pipe, numeric_features),
    ("cat", categorical_pipe, categorical_features)
])

model = Pipeline([
    ("preprocess", preprocess),
    ("classifier", LogisticRegression(max_iter=1000))
])

The option handle_unknown="ignore" prevents a production failure when a previously unseen category appears. It does not guarantee that the resulting prediction is appropriate; the data pipeline still requires monitoring.

3. Training, testing and validation

A train-test split separates data used for fitting from data reserved for final evaluation. If the dataset is randomly divided, a common notation is:

The test set should remain untouched until model selection and tuning are complete. In practice, training data may be divided again into training and validation portions, or cross-validation may be used.

For classification, stratification preserves approximately the same class proportions in each split. For time-dependent data, random splitting can be invalid because it allows future patterns to influence the past. A chronological split is more realistic: earlier observations train the model and later observations test it.

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.20, random_state=42, stratify=y
)

The parameter random_state makes the split reproducible. Reproducibility does not make a split unbiased; it simply allows the same experiment to be repeated.

Cross-validation

In k-fold cross-validation, the training data are divided into k folds. Each fold is used once as validation data while the remaining k − 1 folds are used for training. The average validation score is:

Here score_j is the score from fold j. Cross-validation reduces dependence on one arbitrary validation split, but it does not replace a final untouched test set. For time series, use time-aware folds rather than randomly shuffled folds.

4. Regression

Regression predicts a numerical outcome. Ordinary least squares linear regression assumes a relationship of the form:

Here y_hat is the predicted outcome, β₀ is the intercept, β_j is the coefficient for feature x_j, and p is the number of predictors. The coefficients are estimated by minimising the residual sum of squares:

where n is the number of observations, y_i is the observed outcome and y_hat_i is the prediction for observation i. Squaring gives greater influence to large errors and prevents positive and negative residuals from cancelling.

Important assumptions for conventional inference include linearity in the parameters, independent observations, constant error variance and, for small-sample hypothesis tests, suitable assumptions about error distributions. Prediction can still be useful when some assumptions are imperfect, but coefficient significance and confidence intervals require greater care.

Regression metrics

Mean absolute error is:

It is expressed in the same units as the target and represents average absolute error.

Mean squared error is:

Its units are squared target units. Root mean squared error is:

RMSE is in the target’s units and penalises large errors more heavily than MAE.

The coefficient of determination is:

where y_bar is the mean target in the evaluated sample. Relative to predicting the sample mean, R² measures the proportion of variation explained by the model. It may be negative on test data when the model performs worse than the mean baseline; it is not a percentage of prediction accuracy.

Example: suppose actual sales are 100, 120 and 140, while predictions are 90, 130 and 150. Absolute errors are 10, 10 and 10, so MAE = 10 sales units. Squared errors are 100, 100 and 100, giving MSE = 100 and RMSE = 10 sales units.

from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
import numpy as np

reg = LinearRegression()
reg.fit(X_train, y_train)
pred = reg.predict(X_test)

mae = mean_absolute_error(y_test, pred)
rmse = np.sqrt(mean_squared_error(y_test, pred))
r2 = r2_score(y_test, pred)

A coefficient is not automatically a causal effect. If advertising expenditure and market demand move together, a coefficient may reflect confounding, reverse causality or omitted variables.

5. Classification

Classification assigns observations to categories. Binary classification has two classes, such as default and non-default. A classifier may output a probability, such as P(y = 1|x), and convert it to a class using a threshold.

Logistic regression models the log-odds of a positive class:

Here p is the predicted probability of class 1, and the left side is the log-odds. Solving for probability gives:

where z = β₀ + β₁x₁ + … + β_px_p. A coefficient β_j represents a change in log-odds for a one-unit increase in x_j, holding other included variables constant. e^β_j is the corresponding odds ratio, subject to the model specification and scaling of the feature.

Confusion matrix and metrics

For a binary classifier, predictions can be classified as true positives (TP), false positives (FP), true negatives (TN) and false negatives (FN).

Accuracy is:

Precision is:

Recall, also called sensitivity, is:

The F1 score is the harmonic mean of precision and recall:

These formulas assume the relevant denominators are non-zero. Accuracy can be misleading under class imbalance. For example, identifying 98% of cases as “not fraud” produces 98% accuracy when fraud prevalence is only 2%, even if no fraud is detected. The choice of metric should reflect the relative costs of false positives and false negatives.

ROC-AUC summarises ranking performance across classification thresholds, while precision-recall curves are often more informative for rare positive classes. Neither metric by itself proves that predicted probabilities are calibrated. Calibration asks whether, among cases assigned a probability near 0.7, approximately 70% are positive over a sufficiently large and relevant group.

from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report, roc_auc_score

clf = LogisticRegression(max_iter=1000)
clf.fit(X_train, y_train)
prob = clf.predict_proba(X_test)[:, 1]
pred = (prob >= 0.50).astype(int)

print(classification_report(y_test, pred))
print(roc_auc_score(y_test, prob))

The threshold of 0.50 is a convention, not a law. A bank may choose a lower threshold to detect more potential fraud, accepting more investigations, or a higher threshold when manual review is expensive.

6. Clustering

Clustering groups observations according to similarity without a supplied target. In k-means, the analyst selects k clusters and the algorithm alternates between assigning observations to their nearest centroid and updating each centroid to the mean of its assigned observations. Its objective is:

Here C_j is cluster j, μ_j is its centroid and the norm denotes Euclidean distance. The method assumes that mean-based, relatively compact clusters are meaningful. Results can be sensitive to scaling, outliers, initialisation and the chosen k.

from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler

Z = StandardScaler().fit_transform(customer_features)
km = KMeans(n_clusters=4, n_init=10, random_state=42)
labels = km.fit_predict(Z)

The analyst should profile clusters using original business variables, not only centroid numbers. The silhouette score can assess separation and cohesion, but a mathematically separated cluster may have no actionable business meaning. Segments should therefore be tested for stability, interpretability and commercial usefulness.

7. Overfitting, leakage and model interpretation

Underfitting occurs when a model is too simple to represent important structure. Overfitting occurs when a model learns noise or accidental patterns in training data and performs poorly on new data. Increasing model complexity generally reduces training error, but test error may eventually increase.

Ways to reduce overfitting include:

• using a simpler model;
• regularisation;
• limiting tree depth or the number of leaves;
• selecting features using training data only;
• collecting more representative data;
• using cross-validation for tuning;
• comparing with a simple baseline.

Leakage occurs when information unavailable at prediction time enters the model. Examples include using a post-cancellation field to predict churn, imputing with full-dataset statistics, selecting features using the test target, or randomly splitting a time series. Leakage often produces impressive but deceptive evaluation scores.

Interpretation should match the model and the question. Linear coefficients are conditional associations. Tree feature importance can favour variables with many possible split points. Permutation importance measures performance reduction after shuffling a feature, but correlated predictors can share importance. Partial dependence and local explanation methods can be useful, yet they are approximations and may be unreliable outside the observed data distribution. Explanations should never be treated as proof of causation.

8. Applied case: illustrative customer-retention model

Background

A subscription business wants to identify customers at high risk of cancelling within the next 30 days. The business plans to send a retention offer, but each offer has a cost and excessive contact may annoy loyal customers.

Problem

The analytics team has monthly customer snapshots containing tenure, recent usage, payment history, service contacts, region and subscription plan. The target is cancellation within 30 days after each snapshot. The team must estimate risk without using information recorded after the snapshot.

Analysis and approach

First, the team removes cancellation date, post-cancellation survey responses and any manually created “save outcome” field. It creates lagged features, such as usage in the previous 30 days, and uses a chronological split: earlier months for training and the latest month for testing. This reflects how the model will operate.

Numerical missing values are imputed within a pipeline; categorical variables are one-hot encoded. Logistic regression provides a transparent baseline, while a tree-based model is evaluated as a more flexible alternative. Because only 8% of customers churn, accuracy is not the primary metric. The team reports recall, precision, PR-AUC, calibration and the number of offers generated at candidate thresholds.

Suppose a threshold produces 80 true positives, 120 false positives, 20 false negatives and 780 true negatives. Then:

The model finds 80% of churners, but only 40% of contacted customers are predicted churners who actually cancel. Whether this is acceptable depends on the offer cost and the value of retaining a customer. A profit-based threshold may be more appropriate than a default probability threshold.

Outcome

The business deploys the model only after a controlled pilot. It compares targeted offers with a holdout group to estimate incremental retention. The model is monitored monthly for calibration, missingness, performance by customer group and changes in churn prevalence.

Learning points

The case demonstrates that a model score is not the same as business value; temporal design is essential; leakage can enter through operational fields; imbalanced classification requires suitable metrics; and causal experimentation is needed to establish whether an intervention creates additional retention.

9. Ethics, governance and professional practice

Predictive systems can reproduce historical discrimination. A variable such as postcode may act as a proxy for protected characteristics. Removing protected attributes does not guarantee fairness because other variables may encode similar information. Evaluation should consider legally and ethically relevant group differences, including error rates, access and calibration where appropriate.

Important governance practices include:

• documenting data sources, definitions, assumptions and intended use;
• recording model versions, training dates and random seeds;
• restricting access to personal data;
• applying data minimisation and retention policies;
• checking consent, lawful purpose and relevant privacy requirements;
• providing human review for high-impact decisions;
• enabling challenge, correction and appeal where appropriate;
• monitoring drift, performance and subgroup outcomes;
• testing security against manipulation and unauthorised access.

A model should not be deployed merely because it has a high score. Decision-makers must ask whether the target is appropriate, whether the data represent the population, whether the intervention is beneficial and whether people can be treated fairly. Automated decisions in employment, credit, insurance and public services may have heightened regulatory obligations.

10. Contemporary developments and best practice

Modern analytics increasingly uses gradient boosting, deep learning, automated machine learning and large language models. These technologies can improve performance for suitable data, but they also increase complexity, computational cost and governance requirements. For a diploma-level business application, a transparent baseline should usually be built before a complex model is introduced.

Best practice is to:

1. begin with a meaningful baseline and a clear business metric;
2. define the prediction time and target window explicitly;
3. use pipelines for all learned transformations;
4. choose splits that reflect deployment, especially for time data;
5. tune models using cross-validation inside training data;
6. reserve the test set for final evaluation;
7. report uncertainty, subgroup performance and operational costs;
8. distinguish association from causation;
9. document assumptions, limitations and model version;
10. monitor after deployment and retrain only under controlled procedures.

Chapter Summary

Predictive analytics converts historical data into estimates that support business decisions. Regression predicts numerical outcomes, classification predicts categories or probabilities, and clustering discovers groups without labelled outcomes. Reliable modelling depends on defining observations and targets correctly, preparing features without leakage, using pipelines, and selecting train-test procedures that reflect real deployment. Regression can be evaluated with MAE, MSE, RMSE and R²; classification requires attention to confusion-matrix metrics, class imbalance, thresholds and calibration; clustering requires scaled features and business validation of segments. Overfitting is controlled through appropriate complexity, cross-validation, regularisation and honest testing. Model interpretation describes patterns and associations, not automatically causal effects. Ethical practice requires privacy, fairness assessment, documentation, human oversight and continuous monitoring.

Key Terms

Predictive analytics: The use of data and models to estimate future or unknown outcomes.

Machine learning: Methods that learn patterns from data to make predictions or discover structure.

Feature: An input variable used by a model.

Target: The outcome a supervised model is trained to predict.

Regression: Supervised learning for a numerical outcome.

Classification: Supervised learning for categorical outcomes.

Clustering: Unsupervised grouping of similar observations.

Imputation: Replacing or estimating missing values.

One-hot encoding: Representing categories with binary indicator variables.

Standardisation: Rescaling a variable using its training mean and standard deviation.

Data leakage: Use of information that would not be available at prediction time.

Train-test split: Separation of data for fitting and final evaluation.

Cross-validation: Repeated training and validation across data folds.

Overfitting: Learning noise or sample-specific patterns that fail to generalise.

Precision: The proportion of predicted positives that are actually positive.

Recall: The proportion of actual positives that are detected.

Calibration: Agreement between predicted probabilities and observed frequencies.

Regularisation: A method that discourages excessive model complexity.

Data drift: Change in input distributions or relationships over time.

Model governance: Controls for responsible development, deployment and monitoring.

Review Questions

5 Short Answer Questions

1. Distinguish supervised learning from unsupervised learning.
2. Why should imputation parameters be learned from training data only?
3. Define precision and recall.
4. What is data leakage? Give one business example.
5. Why is scaling important for k-means clustering?

5 Long Answer Questions

1. Explain a complete predictive-modelling workflow from business problem definition to post-deployment monitoring.
2. Compare MAE, RMSE and R², including situations in which each is useful.
3. Explain logistic regression, odds ratios and the role of classification thresholds.
4. Discuss overfitting, cross-validation and the importance of an untouched test set.
5. Explain how ethical and governance considerations affect the deployment of a customer-risk model.

3 Analytical / Application Questions

1. A fraud classifier has high accuracy but detects very few fraudulent transactions. Analyse why accuracy may be misleading and recommend alternative metrics and threshold decisions.
2. Design a leakage-resistant modelling process for predicting next-month sales from monthly business data. Justify the split strategy and feature timing.
3. A company obtains four customer clusters using k-means. Propose a procedure for deciding whether the clusters are stable, interpretable and useful for marketing action.

Multiple Choice Questions with Answers

Question 1

Which task is a supervised learning problem?

A. Grouping customers without a target variable

B. Predicting next-month revenue from historical features and known revenue outcomes

C. Reducing dimensions for visualisation

D. Finding unusual records without labels

Correct Answer: B. Predicting next-month revenue from historical features and known revenue outcomes

Question 2

Which practice is most likely to cause data leakage when preparing a test set?

A. Applying a median learned from the training set

B. Using a pipeline for imputation and modelling

C. Calculating the imputation median from the complete dataset before splitting

D. Using an untouched test set for final evaluation

Correct Answer: C. Calculating the imputation median from the complete dataset before splitting

Question 3

For actual values 100, 120 and 140 and predictions 90, 130 and 150, what is the MAE?

A. 3.33

B. 10

C. 100

D. 300

Correct Answer: B. 10

Question 4

What does recall measure in binary classification?

A. The proportion of predicted positives that are correct

B. The proportion of actual positives that are detected

C. The proportion of all predictions that are correct

D. The proportion of actual negatives that are detected

Correct Answer: B. The proportion of actual positives that are detected

Question 5

Why is accuracy often unsuitable as the only metric for rare fraud detection?

A. Accuracy cannot be calculated for binary outcomes

B. Accuracy ignores both true positives and true negatives

C. A model can achieve high accuracy by predicting the majority non-fraud class

D. Accuracy is measured in squared currency units

Correct Answer: C. A model can achieve high accuracy by predicting the majority non-fraud class

Question 6

Which split is generally most appropriate for evaluating a model that predicts future monthly demand?

A. A chronological split with earlier months for training and later months for testing

B. A split using only the most recent month for training and older months for testing

C. A random split that deliberately mixes future and past observations

D. A split based only on alphabetical customer names

Correct Answer: A. A chronological split with earlier months for training and later months for testing

Question 7

In standardisation, what does σ represent in z = (x − μ)/σ?

A. The target probability

B. The training-set standard deviation

C. The model intercept

D. The number of clusters

Correct Answer: B. The training-set standard deviation

Question 8

What is the main purpose of an untouched test set?

A. To fit preprocessing transformations repeatedly

B. To select features until the score is maximised

C. To provide a final estimate of performance on unseen data

D. To replace the need for a business baseline

Correct Answer: C. To provide a final estimate of performance on unseen data

Question 9

Which statement about a logistic-regression coefficient is correct?

A. It is always a causal effect

B. It represents a change in log-odds for a one-unit feature increase, holding included variables constant

C. It is always measured in the target’s original units

D. It directly equals classification accuracy

Correct Answer: B. It represents a change in log-odds for a one-unit feature increase, holding included variables constant

Question 10

Which issue is especially important when applying k-means clustering?

A. The target must contain exactly two classes

B. Features should never be scaled

C. The chosen number of clusters and the business meaning of the resulting groups must be assessed

D. The method automatically proves that clusters are causal groups

Correct Answer: C. The chosen number of clusters and the business meaning of the resulting groups must be assessed