Predictive Analytics Fundamentals

Introduction

Predictive analytics uses historical and current data to estimate future outcomes or the likelihood of events. It is a central component of business analytics because organisations rarely make decisions only about what has happened; they also need to decide what is likely to happen next. Examples include forecasting sales, estimating customer lifetime value, identifying customers who may leave, assessing credit risk, predicting equipment failure and prioritising marketing leads.

Predictive analytics does not provide certainty. A model produces an estimate based on patterns in data and assumptions about how those patterns may continue. A professionally useful model therefore requires more than a high accuracy score. Analysts must define the business problem, prepare appropriate data, select suitable variables, evaluate performance on unseen data, communicate uncertainty and assess whether predictions improve decisions.

This chapter introduces two foundational model families: regression, which predicts numerical outcomes, and classification, which predicts categories or event probabilities. It also explains feature selection, training and test data, model assumptions, performance measures, overfitting and business usefulness.

Learning Objectives

By the end of this chapter, you should be able to:

1. Explain the purpose and workflow of predictive analytics.
2. Distinguish regression from classification problems.
3. Interpret a simple linear regression model and its coefficients.
4. Prepare training and test data appropriately.
5. Explain feature selection and common model assumptions.
6. Calculate and interpret introductory accuracy measures.
7. Recognise overfitting, underfitting and data leakage.
8. Evaluate whether a model is useful for a business decision.
9. Discuss ethical, governance and operational issues in predictive modelling.

1. The Predictive Analytics Workflow

A predictive project begins with a decision, not with an algorithm. A typical workflow is:

1. Define the decision and target. Specify what action the organisation may take and what outcome is to be predicted.
2. Collect and understand data. Identify sources, time periods, units, missing values, duplicates and possible biases.
3. Prepare the data. Correct errors, encode categorical variables, handle missing values and construct meaningful features.
4. Split the data. Use training data to estimate the model and test data to assess performance on observations not used for fitting.
5. Select and fit a model. Begin with a transparent baseline and use a more complex method only when justified.
6. Evaluate performance. Use measures appropriate to the target, costs of errors and business context.
7. Interpret and validate. Check assumptions, stability, fairness and whether relationships make business sense.
8. Deploy and monitor. Integrate predictions into a process and monitor data quality, performance and changing conditions.

The target, also called the response or dependent variable, is the outcome being predicted. Features, predictors or independent variables are the inputs used by the model. For an observation indexed by i, a prediction may be written as ŷ_i, while the observed target is y_i. The prediction error, or residual for that observation, is:

Here, e_i and y_i have the same units. This definition applies to both regression predictions and numerical comparisons of classification probabilities, although classification uses additional measures.

2. Regression and Classification

2.1 Regression

Regression predicts a numerical quantity, such as monthly revenue, delivery time, demand, salary or house price. The target can be continuous or, in some applications, a count. A simple linear regression with one feature is:

where:

• ŷ_i is the predicted value for observation i;
• x_i is the feature value;
• β₀ is the intercept, the predicted target when x_i = 0;
• β₁ is the slope, the expected change in the target for a one-unit increase in x_i.

The slope has units of target per unit of feature. Its interpretation is meaningful only within the observed data range and under the model’s assumptions. An intercept may have little practical meaning if a feature value of zero is impossible or far outside the data range.

With several features, multiple linear regression is:

The coefficient β_j represents the expected change in the target for a one-unit increase in feature j, holding the other included features constant. This is a conditional association, not automatically a causal effect.

Ordinary least squares estimates coefficients by minimising the sum of squared residuals:

Here, n is the number of observations and SSE is the residual sum of squares. Squaring gives greater influence to large errors and ensures positive and negative errors do not cancel. This method is appropriate when a linear conditional mean is a reasonable approximation and when extreme errors do not make the estimate unusably sensitive.

Worked Example: Simple Regression

Suppose a retailer estimates weekly sales from advertising expenditure. The fitted model is:

Sales are measured in units and advertising in monetary units. For advertising expenditure of 300 monetary units:

The slope suggests that an additional monetary unit of advertising is associated with 4.5 additional sales units, within the observed range and conditional on the model. It does not prove that increasing advertising causes exactly 4.5 additional sales units; seasonality, price, promotions and competitor activity may also affect sales.

2.2 Classification

Classification predicts a category, such as approve/decline, churn/not churn, fraud/not fraud or low/medium/high risk. Binary classification has two classes. Multiclass classification has more than two classes.

Many classifiers first estimate a probability. For a binary outcome, let p_i be the predicted probability that observation i belongs to the positive class. A threshold converts the probability into a class prediction:

Here, t is the chosen threshold, commonly 0.50 but not necessarily the best business threshold. A lower threshold may be appropriate when missing a positive case is very costly, such as failing to identify a serious safety fault.

Logistic regression is a widely used, interpretable classification model. It models the log-odds of the positive class:

The probability can be recovered using:

The log-odds formulation ensures predicted probabilities remain between 0 and 1. A coefficient is additive on the log-odds scale. Its exponentiated value, exp(β_j), is an odds ratio for a one-unit increase in feature j, holding other features constant. Odds ratios should not be confused with probability differences.

3. Data Preparation and Feature Selection

Predictive performance depends substantially on data quality. Important preparation tasks include:

• Handling missing values: determine whether missingness is random, systematic or informative. Imputation must be learned from training data and then applied to test data.
• Encoding categories: convert categories into suitable numerical representations, such as one-hot encoding, while avoiding inconsistent categories between training and deployment.
• Scaling: standardise features when the algorithm is sensitive to measurement scale. A common transformation is:

where μ and σ are the mean and standard deviation calculated from training data only. Scaling is often useful for distance-based and regularised models but is not essential for every tree-based method.

• Outliers: investigate whether extreme values are errors, rare genuine cases or evidence of a different process. Removing them automatically can discard important business information.
• Time variables: create features such as month, day of week, recency or trend only using information available at prediction time.

Feature Selection

Feature selection is the process of choosing variables to include in a model. It can improve interpretability, reduce data collection costs, shorten computation and sometimes improve generalisation. Selection should be guided by business knowledge, data quality, predictive evidence and ethical considerations.

Three broad approaches are:

A strong predictor may be unsuitable if it is unavailable at decision time, legally restricted, highly unstable or a proxy for protected characteristics. Feature selection must therefore consider operational and governance criteria, not merely statistical association.

4. Training, Validation and Test Data

A model must be evaluated on data that represent future use and were not used to fit or select the final model. The training set is used to estimate parameters. A validation set may be used to compare models, tune hyperparameters or choose a threshold. The test set is reserved for final, relatively unbiased performance assessment.

A common introductory split is 70–80% training data and 20–30% test data, but the appropriate choice depends on sample size, data volume and the cost of holding out observations. For time-dependent data, a random split may be inappropriate because it can allow future information to influence a model intended for the past. A chronological split is often more realistic: earlier observations train the model and later observations test it.

Cross-validation repeatedly divides training data into folds. In k-fold cross-validation, the training data are divided into k parts; each part is used once for validation while the others are used for fitting. Average validation performance supports model comparison. The final test set should not be repeatedly consulted during development.

Data leakage occurs when information unavailable at prediction time, or information derived from the outcome, enters the model. Examples include using a cancellation code created after a customer has already churned, calculating an imputation mean from the full dataset before splitting, or randomly mixing future and past records in a forecasting task. Leakage can produce impressive but invalid results.

5. Model Assumptions and Diagnostics

Assumptions are simplifications that support interpretation or reliable estimation. They do not all apply identically to every algorithm.

For ordinary least squares regression, commonly examined assumptions include:

1. Linearity: the conditional mean of the target is adequately represented by a linear combination of predictors.
2. Independent errors: residuals are not systematically related across observations. This is especially important for repeated measurements and time series.
3. Constant error variance: residual variability is approximately similar across fitted values, known as homoscedasticity.
4. Limited multicollinearity: predictors are not so strongly related that individual coefficient estimates become unstable.
5. Reasonable residual behaviour: influential observations and extreme residuals should be investigated.

Normally distributed errors are useful for small-sample confidence intervals and hypothesis tests, but normality is not required for obtaining a prediction in every practical setting. A high R² does not establish that assumptions are satisfied.

For classification, relevant checks include class imbalance, probability calibration, threshold choice, separability, stability across groups and changes in the base rate. A model can rank cases well but produce poorly calibrated probabilities. Calibration means that among cases assigned a probability of approximately 0.70, roughly 70% should be positive in a sufficiently large comparable group.

Multicollinearity can make coefficient interpretation unstable even when overall predictions remain adequate. Correlation matrices, variance inflation diagnostics and domain knowledge can reveal redundant predictors. Removing a variable should not be automatic: two related variables may represent distinct business concepts.

6. Performance Measures

No single metric is best for every problem. The metric must reflect the target type and consequences of errors.

6.1 Regression Measures

Mean absolute error is:

MAE is in the same units as the target and represents the average absolute prediction error. It is relatively less sensitive to extreme errors than squared-error measures.

Mean squared error is:

MSE is expressed in squared target units, which can make direct communication difficult, but it penalises large errors strongly.

Root mean squared error is:

RMSE returns to the target’s units and remains more sensitive to large errors than MAE.

The coefficient of determination is:

where ȳ is the mean observed target in the evaluation sample. R² compares the model’s squared-error performance with predicting the sample mean. It can be negative on test data when the model performs worse than that baseline. It is not a percentage of accuracy and does not demonstrate causation.

6.2 Classification Measures

A binary classifier produces a confusion matrix:

Accuracy is:

Precision is:

It answers: among predicted positives, what proportion are actually positive? Recall, also called sensitivity or true positive rate, is:

It answers: among actual positives, what proportion were identified? Specificity, or true negative rate, is:

The F1 score is the harmonic mean of precision and recall:

These formulas require non-zero denominators. Accuracy can be misleading with imbalanced classes. For example, if only 2% of transactions are fraudulent, predicting “not fraud” for every transaction gives 98% accuracy but detects no fraud. In such a situation, precision, recall, specificity, the precision–recall trade-off and financial costs are more informative.

A cost-based evaluation can assign different costs to errors. If C_FP is the cost of a false positive and C_FN the cost of a false negative, a simplified total error cost is:

This excludes operating costs and benefits unless they are added explicitly. It illustrates why threshold selection should reflect the decision, not only a default probability threshold.

Worked Example: Classification Evaluation

Suppose a churn model produces TP = 36, FP = 9, FN = 24 and TN = 131. There are 200 customers. Accuracy is:

Precision is 36 ÷ (36 + 9) = 0.80, or 80%. Recall is 36 ÷ (36 + 24) = 0.60, or 60%. Thus, the model’s churn alerts are correct four times out of five, but it identifies only 60% of customers who actually churn. Whether that is acceptable depends on the cost of contacting customers and the value of recovering a potential churner.

7. Overfitting, Underfitting and Generalisation

Overfitting occurs when a model learns random noise or peculiarities of the training data instead of general patterns. It usually has very strong training performance but materially weaker test performance. Causes include excessive complexity, too many features relative to observations, repeated tuning against the test set and leakage.

Underfitting occurs when a model is too simple to represent important structure. It performs poorly on both training and test data. A useful model balances bias and variance: overly simple models have high bias, while highly sensitive models have high variance.

Ways to reduce overfitting include:

• use a simpler model or fewer features;
• collect more representative data;
• apply regularisation;
• use cross-validation for tuning;
• control tree depth or the number of model parameters;
• remove leakage and duplicate records;
• monitor performance on genuinely future data.

Regularised linear models add a penalty to the fitting objective. Ridge regression, for example, uses:

where λ ≥ 0 controls the penalty and p is the number of features. Larger λ generally shrinks coefficients more strongly. The intercept is commonly excluded from the penalty. The penalty does not automatically solve poor data quality, leakage or biased sampling.

A performance gap between training and test data is a warning, not a complete diagnosis. Analysts should inspect learning curves, validation results, feature stability, subgroup performance and changes in the operating environment.

8. Business Usefulness and Interpretation

A predictive model is valuable only when it supports a better decision. Business usefulness depends on:

• the quality and timeliness of predictions;
• the action taken after a prediction;
• the costs of data collection and intervention;
• the value of correct decisions;
• the costs of false positives and false negatives;
• operational capacity, such as the number of customers a team can contact;
• legal, ethical and reputational consequences;
• whether the model remains effective over time.

A model with slightly lower statistical performance may be preferable if it is transparent, stable, inexpensive and easy to integrate. Conversely, a complex model may be justified in high-volume decisions when its incremental benefit is demonstrated and governance is adequate.

Interpretation should distinguish association from causation. If high support-call frequency predicts churn, contacting customers with many calls may help, do nothing or worsen dissatisfaction. A prediction identifies risk; it does not prove which intervention will change the outcome. Causal experiments or carefully designed quasi-experimental methods are needed to estimate intervention effects.

Predictions should be communicated with appropriate uncertainty. A point estimate such as projected demand of 10,000 units should not be presented as a guarantee. Prediction intervals, scenario ranges or confidence information may be more useful for planning, although their interpretation depends on the model and assumptions.

9. Illustrative Case: Predicting Subscription Churn

Background

A fictional online learning company has 80,000 active subscribers. Monthly churn is costly because acquiring a new subscriber is more expensive than retaining an existing one. The company wants to identify subscribers who may cancel in the next 30 days so that a retention team can offer relevant support.

Problem

The target is binary: churn within 30 days, coded 1, or remain subscribed, coded 0. Candidate features include days since last login, number of lessons completed, payment failures, subscription age, customer-service contacts and plan type.

Analysis

The data team first defines a prediction date for each subscriber. Only information available on that date is included. It removes duplicate accounts and investigates missing payment information. Because the records are time ordered, the team trains on January–September observations, uses October for validation and tests on November observations. This design approximates future deployment better than a random split.

The team begins with a simple baseline that predicts no churn for every subscriber. Since only 8% churn, this baseline has 92% accuracy but zero recall for churners. A logistic regression model is then fitted, with categorical plan type encoded and numerical features prepared using training data only. A decision threshold is selected using the retention team’s capacity and the cost of false alerts.

The model shows test precision of 0.31 and recall of 0.68. These values may appear modest, but they are more informative than accuracy alone. The team estimates the financial value of contacting a correctly identified churner, the cost of contacting a subscriber who would have stayed, and the cost of missing a churner. It also checks whether performance differs substantially by plan type, region and accessibility-related customer characteristics.

Approach and Solution

The company uses the model to rank subscribers by predicted churn probability rather than treating all predictions as certain. The retention team contacts the highest-risk group that it can serve. Messages are tested through a controlled experiment, because prediction alone cannot show whether the intervention reduces churn. The model is retrained monthly, while data drift and calibration are monitored weekly.

Outcome

In the first trial, the model increases the proportion of contacted customers who subsequently renew compared with a generic campaign. The company does not claim that the model caused every renewal; the experimental comparison estimates the incremental effect of the retention action. Governance staff document features, threshold logic, access controls and an appeal process for affected customers.

Learning Points

1. The business target and decision were defined before modelling.
2. A high-accuracy imbalanced baseline was shown to be operationally weak.
3. Time-based splitting reduced the risk of future information entering training.
4. Threshold selection reflected team capacity and error costs.
5. Predictions were separated from causal claims about retention offers.
6. Fairness, monitoring and documentation were part of deployment rather than afterthoughts.

10. Ethical, Governance and Regulatory Considerations

Predictive models can reproduce historical discrimination. If past lending decisions were biased, a model trained on those outcomes may learn and amplify that bias. Removing a protected attribute does not guarantee fairness because other variables may act as proxies.

Responsible practice includes documenting the purpose, data sources, target definition, feature rationale, model version, evaluation population and limitations. Organisations should restrict access to sensitive data, protect personal information, retain audit logs and establish procedures for correction or appeal. Data protection obligations commonly require a lawful purpose, data minimisation, appropriate security and transparency, although exact legal requirements depend on jurisdiction and application.

Fairness cannot be reduced to one universal metric. Possible checks include comparing selection rates, false-positive rates, false-negative rates or calibration across relevant groups. These measures can conflict, particularly when group base rates differ. The organisation must define the relevant harm, involve affected stakeholders and justify its chosen standard.

Human oversight must be meaningful. A human who simply approves every automated recommendation without reviewing evidence does not provide effective oversight. Staff need training, clear escalation rules and authority to override a model when circumstances warrant it.

11. Contemporary Developments and Best Practice

Modern predictive analytics increasingly uses automated machine learning, cloud platforms, real-time scoring and explainability tools. These technologies can accelerate development but do not replace problem definition, validation or governance. Generative AI may assist with feature documentation, code drafts or natural-language explanations, but analysts must verify outputs, protect confidential data and avoid treating generated content as evidence.

Best practice for introductory predictive projects is to:

• establish a meaningful baseline;
• define the prediction time and avoid leakage;
• use a split that reflects deployment;
• select metrics based on consequences;
• compare training, validation and test performance;
• inspect errors and subgroup outcomes;
• prefer interpretable models when performance is similar;
• document assumptions and limitations;
• test the intervention rather than assuming prediction creates value;
• monitor drift, calibration, fairness and business outcomes after deployment.

Chapter Summary

Predictive analytics estimates future numerical outcomes or event categories from data. Regression predicts quantities, while classification predicts categories or probabilities. Linear regression expresses a target as a combination of features, and logistic regression models the log-odds of a binary outcome.

A credible predictive workflow begins with a business decision and includes data preparation, feature selection, training, validation, testing, interpretation and monitoring. Training data estimate model parameters; validation data support development; and test data provide a final assessment on unseen observations. Time-aware splits are important when future information must not influence past predictions.

Regression can be assessed using MAE, MSE, RMSE and R². Classification should use a confusion matrix and measures such as accuracy, precision, recall, specificity and F1, with threshold and error costs considered explicitly. Overfitting harms generalisation, while underfitting reflects excessive simplicity. Leakage can make performance appear better than it really is.

A model’s business value depends on the decisions, costs, benefits, operational capacity, fairness and governance surrounding it. Statistical association does not establish causation, and deployment requires monitoring because data and behaviour change over time.

Key Terms

Predictive analytics: Analysis that estimates future outcomes or event probabilities from data.

Target variable: The outcome a model is designed to predict.

Feature: An input variable used to generate a prediction.

Regression: A modelling task involving a numerical target.

Classification: A modelling task involving categorical outcomes.

Linear regression: A model representing the conditional mean as a linear combination of predictors.

Logistic regression: A classification model that models binary log-odds as a linear function of features.

Coefficient: An estimated parameter describing a feature’s relationship with the modelled outcome.

Residual: The difference between an observed value and its prediction.

Training data: Observations used to estimate model parameters.

Validation data: Observations used during development to compare or tune models.

Test data: Held-out observations used for final performance assessment.

Feature selection: Choosing which variables to include in a model.

Data leakage: Use of information that would not be available at prediction time.

Overfitting: Learning noise or idiosyncrasies that reduce performance on new data.

Underfitting: Using a model too simple to capture important structure.

Precision: The proportion of predicted positives that are actually positive.

Recall: The proportion of actual positives correctly identified.

Calibration: Agreement between predicted probabilities and observed frequencies.

Concept drift: Change over time in the relationship between inputs and outcomes.

Review Questions

5 Short Answer Questions

1. What is the difference between a target variable and a feature?
2. Distinguish regression from classification with one business example of each.
3. Why should a test set not be repeatedly used during model development?
4. What does recall measure in binary classification?
5. Explain why a high training score and a low test score may indicate overfitting.

5 Long Answer Questions

1. Describe the complete predictive analytics workflow from business problem definition through post-deployment monitoring.
2. Explain ordinary least squares regression, including the meaning of the intercept, slope and residual sum of squares.
3. Compare MAE, RMSE and R², including situations in which each may be useful.
4. Explain how data leakage can occur and describe three practical methods for preventing it.
5. Discuss why ethical governance and fairness assessment are necessary when predictive models influence customers or employees.

3 Analytical / Application Questions

1. A retailer’s demand model has low training error but much higher test error. Diagnose possible causes and recommend a development plan.
2. A fraud classifier has 99% accuracy but detects only 10% of fraudulent transactions. Explain why the accuracy is misleading and recommend more suitable evaluation measures.
3. Design a predictive modelling approach for employee attrition. Define the target, candidate features, data split, evaluation measures, operational action and governance checks.

Multiple Choice Questions with Answers

Question 1

Which task is a regression problem?

A. Predicting whether a customer will churn

B. Predicting next month’s sales revenue

C. Classifying invoices as valid or invalid

D. Assigning products to risk categories

Correct Answer: B. Predicting next month’s sales revenue

Question 2

In the model ŷ = 500 + 3x, what does the coefficient 3 represent?

A. The predicted target when x equals zero

B. The average target value in the dataset

C. The predicted change in the target for a one-unit increase in x

D. The squared prediction error

Correct Answer: C. The predicted change in the target for a one-unit increase in x

Question 3

Which dataset should normally be reserved for the final assessment of a model?

A. Training data

B. Validation data

C. Test data

D. Imputation data

Correct Answer: C. Test data

Question 4

What does precision measure?

A. The proportion of actual positives that are identified

B. The proportion of predicted positives that are actually positive

C. The proportion of actual negatives that are identified

D. The proportion of all predictions that are correct

Correct Answer: B. The proportion of predicted positives that are actually positive

Question 5

A model predicts every transaction as non-fraudulent when 2% are fraudulent. What is its accuracy?

A. 2%

B. 50%

C. ว่า 98%

D. 100%

Correct Answer: C. ว่า 98%

Question 6

Which situation is an example of data leakage?

A. Scaling features using training-set means

B. Using a customer status recorded after cancellation to predict cancellation

C. Removing duplicate records before splitting data

D. Testing a model on later chronological observations

Correct Answer: B. Using a customer status recorded after cancellation to predict cancellation

Question 7

Which measure is expressed in the same units as the regression target?

A. MAE

B. MSE

C. R²

D. Log-odds

Correct Answer: A. MAE

Question 8

If false negatives are much more costly than false positives, which action may be appropriate?

A. Always use a threshold of zero

B. Lower the classification threshold to identify more positives

C. Ignore recall and optimise only accuracy

D. Remove all positive cases from the data

Correct Answer: B. Lower the classification threshold to identify more positives

Question 9

What is the main concern when a model performs very well on training data but poorly on test data?

A. Underfitting

B. Overfitting

C. Calibration certainty

D. Perfect generalisation

Correct Answer: B. Overfitting

Question 10

Which statement about a predictive model is most appropriate?

A. A strong association proves that an intervention will cause the outcome

B. A prediction identifies likely outcomes but does not by itself establish causation

C. A high accuracy score guarantees business value

D. Removing protected attributes guarantees fairness

Correct Answer: B. A prediction identifies likely outcomes but does not by itself establish causation