Classification and Business Prediction

Introduction

Classification is a supervised learning task in which a model assigns an observation to one of a set of predefined categories. In business analytics, the categories may represent whether a customer will churn, whether a transaction is fraudulent, whether a borrower will default, or whether a campaign recipient will respond. Classification differs from regression because the target is categorical, although many classification models produce probabilities that support financial and managerial decisions.

A classification project normally follows this sequence:

1. define the business decision and target outcome;
2. obtain and prepare labelled historical data;
3. divide data into training, validation and test sets;
4. fit one or more models;
5. evaluate discrimination, calibration and business cost;
6. select an operating threshold;
7. deploy, monitor and periodically review the model.

The central principle is that a statistically accurate model is not automatically a useful business model. A model must produce reliable predictions, operate within policy and regulatory constraints, and support an action whose benefits exceed its costs.

Learning objectives

By the end of this chapter, you should be able to:

• distinguish classification from regression and describe the classification workflow;
• interpret probabilities, odds, log-odds and decision boundaries;
• build and interpret logistic regression, decision-tree and k-nearest-neighbour models;
• calculate and interpret confusion-matrix measures and threshold-dependent costs;
• address imbalance, overfitting, leakage and probability calibration;
• apply classification to churn, credit risk, fraud and response prediction;
• evaluate ethical, governance and operational implications of predictive decisions.

1. Classification foundations

Let the target variable be Y, with possible classes such as 0 and 1. The predictor vector is X=(X₁,X₂,...,X_p). A classifier estimates either a class label \hat{Y} or a conditional probability such as P(Y=1|X). A threshold converts probability into an action:

Here, \hat{p} is the estimated probability of class 1 and t is the selected threshold. The default threshold of 0.5 is not universally appropriate. If missing a fraud case is much more costly than investigating a legitimate transaction, a lower threshold may be rational.

Classification can be binary, multiclass or multilabel. Binary classification has two mutually exclusive outcomes. Multiclass classification has more than two mutually exclusive outcomes, such as low, medium and high risk. Multilabel classification permits several labels simultaneously, for example products that a customer may be interested in.

Business framing

A target must be defined precisely. “Churn” may mean cancellation within 30 days, no renewal at contract expiry, or no activity for 90 days. The observation date, prediction horizon and information available at prediction time must be specified. Otherwise, the model may learn from information that would not have been available when the decision was made.

A useful project specification identifies:

2. Logistic regression

Logistic regression is a strong baseline for binary classification because it is interpretable, computationally efficient and capable of producing probabilities. It models the log-odds of class 1 as a linear function of predictors:

where p=P(Y=1|X), \beta₀ is the intercept, \beta_j is the coefficient for predictor X_j, and the log function is the natural logarithm. Solving for probability gives the logistic or sigmoid function:

The output is always between 0 and 1. The model assumes that predictors have a linear relationship with the log-odds, not necessarily with the probability itself. It also assumes independent observations unless dependence is explicitly modelled, and it can be affected by severe multicollinearity, influential observations and omitted nonlinear effects.

Interpretation

A coefficient \beta_j represents the change in log-odds associated with a one-unit increase in X_j, holding other predictors constant. Exponentiating the coefficient gives an odds ratio:

For example, if \beta_j=0.40, the odds ratio is approximately e^0.40=1.49. A one-unit increase in the predictor multiplies the odds by about 1.49, or increases them by approximately 49%, subject to the model assumptions. This is not the same as a 49% increase in probability.

For a binary predictor, the odds ratio compares the odds for the group coded 1 with the reference group coded 0. For a continuous predictor, the unit of measurement matters; rescaling income from dollars to thousands of dollars changes the coefficient but not the substantive prediction.

Worked example: probability from a logistic model

Suppose a churn model is:

A customer has two complaints and a month-to-month contract, coded 1. Thus:

Therefore:

The estimated probability of churn is approximately 21.4%. With a threshold of 0.20, the customer is classified as a likely churner; with a threshold of 0.50, the customer is not. The threshold must therefore be linked to the intervention decision.

Decision boundaries

For a threshold of 0.5, the logistic decision boundary occurs where p=0.5, equivalently where z=0. With two predictors, this is a straight line; with more predictors, it is a hyperplane. Nonlinear terms, interactions or transformations can produce curved or more flexible boundaries. A coefficient sign indicates the direction of association, but causal conclusions require a suitable research design.

3. Decision trees

A decision tree recursively partitions observations into increasingly homogeneous groups. An internal node applies a rule such as “income < 40,000”; a branch records the result; and a terminal leaf provides a class prediction or estimated class probability.

For a node containing class proportions p₁,...,p_K, the Gini impurity is:

For binary classes, impurity is zero when all observations belong to one class and is largest when the classes are equally represented. Entropy is an alternative:

A split is selected by reducing weighted impurity. If a parent node has impurity I(P) and children C₁,...,C_m, the reduction is:

where n_r is the number of observations in child r and n_P is the number in the parent. The formula applies when child impurities are weighted by their share of the parent.

Trees are easy to explain, accommodate nonlinear relationships and require little feature scaling. However, an unrestricted tree can memorize training data. Control complexity using maximum depth, minimum observations per leaf, minimum split size, pruning, cross-validation and, where appropriate, ensembles such as random forests or gradient boosting.

A leaf probability is usually the proportion of training observations in that leaf. Small leaves can produce unstable probabilities, so probability smoothing or calibration may be necessary. A tree’s variable importance is not proof that a variable is causal, and importance can be biased toward variables with many possible split points.

4. k-nearest neighbours

The k-nearest-neighbours (kNN) method predicts an observation using the labels of the k most similar training observations. For numeric features, the common Euclidean distance between observations x and z is:

For binary classification, a simple estimated probability is the fraction of the neighbours in class 1:

Here, N_k(x) is the set of k nearest observations and I(·) is an indicator equal to 1 when its condition is true and 0 otherwise.

Feature scaling is essential when variables use different units. Standardisation commonly uses:

where \bar{x}_j is the training mean and s_j is the training standard deviation. Scaling parameters must be calculated using the training set only, then applied to validation and test data.

Small k values create flexible, noisy boundaries; large values create smoother boundaries but may overlook local patterns. kNN can be computationally expensive at prediction time and performs poorly with many irrelevant or high-dimensional features. Categorical variables require an appropriate distance measure or encoding; arbitrary numerical codes can create misleading distances.

5. Comparing models and selecting a threshold

A confusion matrix compares predicted and actual classes:

Important measures include:

Accuracy is appropriate only when error costs and class frequencies make it meaningful. Recall, also called sensitivity or true-positive rate, is important when missing a positive case is costly. Precision is important when interventions or investigations are expensive. Specificity measures the proportion of actual negatives correctly rejected.

The receiver operating characteristic (ROC) curve plots recall against false-positive rate as the threshold varies. The area under this curve (ROC-AUC) measures ranking discrimination across thresholds. The precision–recall curve is often more informative when the positive class is rare. Neither AUC guarantees good probability calibration. Calibration asks whether cases assigned probability 0.7 experience the positive outcome approximately 70% of the time, usually within comparable groups.

For a business decision, expected error cost can be written as:

where each C is the cost or value associated with an outcome. If benefits and costs differ by customer, transaction or segment, use individual-level expected utility rather than one aggregate cost.

Threshold selection should use validation data or an explicitly specified cost function, not the test set. The test set should provide an approximately unbiased final estimate after model and threshold decisions are complete.

6. Data preparation and reliable validation

Classification quality depends heavily on data design. Important practices include:

• define the outcome window before examining predictors;
• remove duplicates and resolve inconsistent labels;
• impute missing values within the resampling pipeline;
• encode categorical variables using training data only;
• scale features for distance-based methods;
• inspect outliers and impossible values;
• use stratified splits when class proportions must be preserved;
• use time-based splits when future performance is the real objective;
• apply cross-validation inside the training data for tuning.

Data leakage occurs when information unavailable at the prediction time enters the model. Examples include a cancellation code recorded after churn, a fraud investigation outcome, or a repayment status measured after the credit decision. Leakage can produce excellent validation scores and poor operational performance.

Class imbalance occurs when one class is uncommon. Remedies include class-weighted learning, carefully designed resampling, anomaly-detection methods and threshold adjustment. Random oversampling must be performed within training folds; otherwise duplicated observations can appear in both training and validation data. Performance should be reported by class and, where relevant, by customer segment.

7. Applications in business

Churn prediction

A telecommunications company may predict cancellation within the next 30 days using tenure, service interruptions, payment history, contract type and recent complaints. The model should be evaluated against the incremental retention value: contacting every high-risk customer may be wasteful if many would remain anyway. A controlled experiment can test whether interventions generated by the model reduce churn rather than merely identifying likely churners.

Credit risk

Credit classification may predict default within a defined period. Predictors can include repayment history, affordability measures and debt utilisation, subject to law and policy. Probability calibration is particularly important because expected loss depends on estimated probability, exposure and loss severity. A simplified expected-loss relationship is:

where PD is probability of default, EAD is exposure at default and LGD is loss given default. This equation applies as a simplified expected-value calculation; real credit systems may model time, discounting, recoveries and dependence separately.

Fraud detection

Fraud is usually rare, labels may be delayed, and behaviour changes as offenders adapt. Precision at the investigation capacity, recall at a chosen false-positive rate and monetary value recovered may be more useful than accuracy. Models should support analyst review, preserve an audit trail and be monitored for changing fraud patterns.

Response prediction

Marketing models estimate the probability that a customer responds to an offer. The relevant business quantity is often uplift: the difference between expected response with treatment and expected response without treatment. A high response probability does not prove that the campaign caused the response; randomised treatment and control groups are needed to estimate incremental impact.

8. Illustrative case: retention analytics at Northstar Mobile

Background and problem

Northstar Mobile, a fictional subscription provider, experienced rising monthly cancellations. Management wished to target a retention offer, but the offer cost $18 per customer and reduced contribution margin by $10 when accepted. The analytics team defined the target as cancellation within 60 days of the monthly prediction date and excluded information recorded after that date.

Analysis and approach

The team assembled 24 months of customer-month records containing tenure, contract type, complaints, payment failures, data usage change and service interruptions. Because future deployment was the objective, the most recent three months were held out as a time-based test set. Logistic regression was selected for interpretability, while a pruned decision tree and kNN were benchmarked.

The logistic model showed higher churn odds for month-to-month contracts, recent payment failures and increasing complaints. Continuous variables were scaled only for kNN. Missing values were imputed inside cross-validation. Class weights were considered because only 12% of customer-month records represented churn.

The team compared ROC-AUC, recall, precision, calibration and expected campaign value. A threshold of 0.5 generated too few contacts. A lower threshold selected more customers, but the additional contacts had lower expected value. The chosen threshold maximised estimated net value on validation data, subject to a call-centre capacity limit.

Outcome

In a subsequent controlled pilot, eligible high-scoring customers were randomly assigned either the retention offer or business-as-usual treatment. The model identified a high-risk group, but the experiment showed that the offer reduced churn substantially only among customers with a particular combination of service interruptions and contract flexibility. Northstar therefore used the model for prioritisation and refined the intervention policy using treatment-effect evidence.

Learning points

The model’s purpose was not simply to predict churn. It was to allocate a scarce intervention. Time-aware validation reduced leakage risk, threshold selection reflected capacity, and experimentation distinguished prediction from causation. Monitoring was established for score distribution, churn prevalence, calibration, intervention acceptance and performance across customer groups.

9. Governance, ethics and contemporary practice

Classification can reproduce historical discrimination when labels reflect unequal access, biased enforcement or past institutional decisions. Fairness analysis should examine error rates, selection rates, calibration and subgroup performance where legally and ethically appropriate. These measures can conflict mathematically, especially when base rates differ; no single fairness metric resolves every policy question.

Organisations should document the purpose, data sources, features, training period, validation design, threshold, limitations and responsible owner. High-impact decisions may require explainability, human review, adverse-action reasons, data protection controls and an appeal process. Sensitive attributes may be excluded from prediction but still needed for fairness auditing, subject to legal safeguards.

Modern practice includes random forests, gradient-boosted trees, calibrated ensembles, interpretable additive models and representation-learning methods. More complex models may improve ranking performance, but complexity increases requirements for monitoring, explanation, security and governance. Automated machine learning can accelerate comparison, but it does not replace target definition, leakage review, cost analysis or professional judgement.

After deployment, monitor data drift, concept drift, missingness, latency, calibration, subgroup outcomes and realised financial value. A model should be retrained or retired when its assumptions no longer hold. Version control, reproducible pipelines and access controls are part of analytical quality, not optional technical extras.

Best-practice checklist

Before deployment, confirm that:

• the target, horizon and decision are unambiguous;
• every feature is available at scoring time;
• the split reflects operational use;
• preprocessing is fitted within training folds;
• the selected metric reflects business costs;
• probabilities have been checked for calibration;
• threshold and capacity decisions are documented;
• performance is reported by relevant segments;
• human oversight and appeal procedures exist where needed;
• monitoring and a model-retirement process are assigned to named owners.

Chapter Summary

Classification predicts categorical outcomes and often supplies probabilities for business action. Logistic regression models linear log-odds and offers interpretable odds ratios. Decision trees produce transparent rule-based partitions but require complexity control. k-nearest neighbours uses local similarity and therefore depends strongly on feature scaling, distance choice and the value of k.

Confusion-matrix measures answer different questions: precision concerns the reliability of positive alerts, recall concerns the detection of positives, specificity concerns rejection of negatives, and F1 balances precision and recall. ROC-AUC measures ranking discrimination, while calibration assesses whether predicted probabilities correspond to observed frequencies. Thresholds should be selected using costs, benefits and operational capacity rather than habit.

Reliable classification requires sound target definition, leakage prevention, appropriate validation, imbalance management and monitoring. In churn, credit, fraud and response prediction, predictive accuracy is only one part of value. Ethical governance, fairness, explainability, experimentation and human accountability determine whether a model is safe and useful in practice.

Key Terms

Classification: Predicting a categorical outcome.

Binary classification: Classification with two possible classes.

Probability threshold: Cut-off used to convert a predicted probability into a class or action.

Logistic regression: A model that represents binary-outcome log-odds as a linear predictor.

Odds ratio: Multiplicative change in odds associated with a predictor change.

Decision tree: A recursive set of feature-based rules ending in prediction leaves.

Gini impurity: A measure of class heterogeneity in a node.

Entropy: An information-based measure of uncertainty in a node.

k-nearest neighbours: A method predicting from the labels of the closest training observations.

Decision boundary: The region or surface separating predicted classes.

Confusion matrix: Table of true and false positive and negative predictions.

Precision: Proportion of predicted positives that are actual positives.

Recall: Proportion of actual positives that are detected.

Specificity: Proportion of actual negatives correctly identified.

Calibration: Agreement between predicted probabilities and observed outcome frequencies.

Class imbalance: Unequal prevalence of classes in a dataset.

Data leakage: Use of information unavailable at the time of prediction.

Concept drift: Change in the relationship between predictors and outcomes over time.

Uplift: Incremental effect of an intervention compared with a suitable control.

Expected loss: Probability-weighted financial loss, often represented by PD × EAD × LGD.

Review Questions

Short Answer Questions

1. What distinguishes classification from regression?
2. Why is a probability threshold needed in binary classification?
3. Interpret an odds ratio of 1.5 in logistic regression.
4. Why must features be scaled for many kNN applications?
5. What is data leakage, and why does it threaten model validity?

Long Answer Questions

1. Explain logistic regression, including log-odds, probability interpretation and key assumptions.
2. Compare decision trees and kNN in terms of interpretability, flexibility, preprocessing and overfitting.
3. Explain precision, recall, specificity, F1 and calibration, and identify situations in which each is useful.
4. Design a validation and threshold-selection strategy for a fraud-classification system.
5. Discuss ethical, governance and monitoring requirements for a credit-risk classifier.

Analytical / Application Questions

1. A retention model produces probabilities for customers, but the intervention budget covers only 8% of customers. Explain how you would select and validate an operating threshold.
2. A fraud model has 98% accuracy but detects only 20% of fraud cases. Diagnose why accuracy is misleading and recommend better evaluation measures.
3. A marketing model identifies customers likely to purchase. Explain why this does not establish that contacting them caused the purchase, and design an experiment to estimate incremental response.

Multiple Choice Questions with Answers

Question 1

Which quantity does logistic regression model directly for a binary outcome?

A. The raw predictor mean

B. The log-odds of the positive class

C. The squared classification error only

D. The Euclidean distance to a centroid

Correct Answer: B. The log-odds of the positive class

Question 2

A predicted probability is 0.214. Under a threshold of 0.20, what class is assigned?

A. Positive class

B. Negative class

C. Both classes

D. No class can be assigned

Correct Answer: A. Positive class

Question 3

If a logistic-regression coefficient is 0.40, its odds ratio is approximately:

A. 0.40

B. 0.67

C. 1.49

D. 2.40

Correct Answer: C. 1.49

Question 4

Which measure is the proportion of predicted positives that are actually positive?

A. Recall

B. Specificity

C. Precision

D. Accuracy

Correct Answer: C. Precision

Question 5

Which method is most directly affected by predictors being measured in very different units?

A. k-nearest neighbours

B. Logistic regression with no interactions

C. A confusion matrix

D. A probability threshold

Correct Answer: A. k-nearest neighbours

Question 6

What is the main purpose of pruning a decision tree?

A. To increase the number of leaves without limit

B. To reduce overfitting and improve generalisation

C. To eliminate the target variable

D. To guarantee perfect calibration

Correct Answer: B. To reduce overfitting and improve generalisation

Question 7

For a rare fraud class, which evaluation view is often more informative than accuracy?

A. Precision–recall performance

B. The sample mean of every feature

C. The number of tree branches only

D. The intercept alone

Correct Answer: A. Precision–recall performance

Question 8

Which example is data leakage in a churn model?

A. Customer tenure measured at scoring time

B. Complaints recorded before scoring

C. A cancellation code entered after the customer churned

D. Contract type known at scoring time

Correct Answer: C. A cancellation code entered after the customer churned

Question 9

In the simplified formula Expected loss = PD × EAD × LGD, what does PD represent?

A. Payment duration

B. Probability of default

C. Predicted debt

D. Portfolio difference

Correct Answer: B. Probability of default

Question 10

Why is a randomised control group useful in response prediction?

A. It guarantees every prediction is correct

B. It estimates incremental treatment impact rather than association alone

C. It removes the need for a target definition

D. It makes class imbalance impossible

Correct Answer: B. It estimates incremental treatment impact rather than association alone