Business Problem Framing and Analytics Processes

Introduction

Predictive analytics does not begin with an algorithm. It begins with a decision. Organisations collect large volumes of data, but data alone does not identify what should be predicted, why the prediction matters, or how it will change an action. Business problem framing connects organisational objectives with measurable analytical questions.

A well-framed analytics problem identifies the decision-maker, the decision to be improved, the population affected, the prediction time, the target variable, operational constraints, and the value of success. Poor framing can produce a technically accurate model that is commercially useless—for example, predicting customer churn after the customer has already cancelled.

This chapter introduces the CRISP-DM process and complementary problem-framing tools. It explains how to convert broad business challenges into predictive questions, define units of analysis and target variables, establish success criteria, estimate costs and benefits, and govern the use of predictive models.

Learning Objectives

By the end of this chapter, you should be able to:

1. Distinguish a business problem from an analytics problem and a predictive question.
2. Explain the phases of CRISP-DM and their iterative nature.
3. Define the unit of analysis, observation window, prediction horizon, and target variable.
4. Translate business objectives into measurable model and business success criteria.
5. Identify constraints, risks, costs, benefits, and ethical considerations.
6. Evaluate whether a proposed predictive analytics project is feasible and valuable.

From Business Challenge to Predictive Question

A business challenge is usually expressed in broad language: “sales are declining,” “customers are leaving,” or “fraud is increasing.” Such statements are important but not yet suitable for modelling. A predictive question specifies what outcome will be estimated, for whom, when, and for what decision.

A useful transformation has four stages:

The final question is more precise because it defines a population (active customers), a target (cancellation), a horizon (30 days), and a decision (offer allocation).

A predictive question normally contains the following elements:

• Entity or unit: customer, transaction, store, machine, employee, claim, or account.
• Prediction time: the point at which available information is frozen.
• Prediction horizon: the future period covered by the prediction.
• Target event or value: what is to be predicted.
• Action: what the organisation will do with the prediction.
• Constraints: budget, timing, privacy, capacity, fairness, and technical limitations.

The question should also distinguish prediction from explanation. “Which customers are likely to leave?” is predictive. “Why do customers leave?” may require explanatory, qualitative, or causal analysis. A model’s predictive association does not automatically demonstrate that changing a variable will cause the outcome to change.

CRISP-DM as an Analytics Process

CRISP-DM—Cross-Industry Standard Process for Data Mining—is a widely used framework for organising analytics work. Its six phases are Business Understanding, Data Understanding, Data Preparation, Modelling, Evaluation, and Deployment. The process is iterative rather than strictly linear.

Figure: CRISP-DM is an iterative process centred on a business decision.

1. Business Understanding

This phase clarifies the organisational objective, stakeholders, decision process, scope, risks, and definition of success. Questions include:

• What decision is currently being made?
• Who makes it and how frequently?
• What happens if no analytical intervention is introduced?
• Which outcomes matter financially or operationally?
• What actions are possible after a prediction is produced?

A project charter should record the objective, assumptions, deliverables, timeline, responsibilities, dependencies, and approval authority. The charter prevents “scope drift,” in which a project gradually changes from solving a business problem to producing interesting analyses.

2. Data Understanding

Data understanding involves identifying available sources, assessing their meaning, examining quality, and checking whether the data represents the intended population. Analysts should inspect missing values, duplicates, unusual values, changes in definitions, sampling bias, and the timing of records.

A variable labelled “customer income,” for example, may actually mean self-reported income at registration, estimated household income, or the income used at the latest credit assessment. These interpretations lead to different analytical conclusions.

3. Data Preparation

Data preparation includes selecting records, joining sources, correcting errors, transforming variables, constructing features, and creating training and testing datasets. The preparation must respect time. Information that was unavailable at the prediction time must not enter the model. This error, called data leakage, can create impressive test results and poor operational performance.

Preparation should be reproducible. A documented data pipeline is preferable to undocumented spreadsheet manipulation because it enables auditing, updating, and consistent scoring.

4. Modelling

Modelling selects and trains one or more methods appropriate to the target and decision. The simplest adequate model is often a useful benchmark. For a binary outcome, classification methods may be used; for a numerical outcome, regression methods may be appropriate. Model choice should consider accuracy, interpretability, latency, maintainability, fairness, and implementation cost—not accuracy alone.

The modelling phase should use a baseline. A baseline may be the current business rule, the historical average, or a simple statistical model. A complex model is valuable only if it improves the decision relative to a credible baseline.

5. Evaluation

Evaluation asks two separate questions:

1. Is the model statistically reliable on data representing future use?
2. Would using it improve the business decision?

A model may rank customers accurately but deliver little value if the organisation cannot contact enough customers, if interventions are ineffective, or if the cost of action exceeds the benefit. Evaluation should therefore include technical metrics, operational feasibility, financial value, and ethical review.

6. Deployment

Deployment may involve a dashboard, an automated batch score, an application programming interface, an alert, or a revised workflow. Deployment also requires ownership, documentation, monitoring, retraining rules, access controls, and procedures for handling errors and appeals.

Deployment is not the end of the process. Data distributions, customer behaviour, regulations, and business policies change. This phenomenon is often described as data drift or concept drift, depending on whether the input distribution or the relationship between inputs and outcomes changes.

Defining the Analytical Object

Unit of Analysis

The unit of analysis is the entity represented by one modelling record. Examples include one customer, one invoice, one loan application, one store-day, or one machine-hour. It must match the decision being supported.

Suppose a retailer wants to predict whether a customer will make a purchase in the next 14 days. A customer-level model may be suitable for targeted marketing. A transaction-level model would answer a different question and may create multiple records for the same customer. Confusing the unit of analysis can lead to duplicated information, incorrect measurement, and invalid evaluation.

The unit should be stated explicitly: “One row represents one active customer as at the weekly scoring date.”

Observation Window and Prediction Horizon

The observation window is the historical period from which predictors are calculated. The prediction horizon is the future period in which the target is measured.

For example, a bank may use transactions from 1 January to 31 March to predict whether an account will become delinquent between 1 April and 30 June. The prediction time is 31 March; the observation window ends there; the prediction horizon begins afterwards.

The temporal boundary prevents future information from being used accidentally. It also makes the model’s intended use clear.

Target Variable

The target variable is the outcome to be predicted. It may be:

• Binary: churn within 30 days: yes/no.
• Multiclass: complaint category: billing, service, delivery, or other.
• Count: number of support contacts next month.
• Continuous: expected sales value next quarter.
• Time-to-event: time until machine failure.

A target must have an operational definition. “Churn” might mean account closure, no purchase for 90 days, or cancellation of a subscription. Each definition produces a different target and potentially different actions.

For a binary target, a common representation is:

Here, y_i is the target for entity i. The definition assumes that the event can be observed reliably and that the horizon is specified. Ambiguous or inconsistently recorded outcomes reduce both model quality and business trust.

Features and Availability

A feature is an input variable available at prediction time. Features should be classified as available, conditionally available, or unavailable. For example, a customer’s cancellation reason may predict churn well but is unavailable before cancellation; using it would be leakage.

Feature selection should consider business meaning, stability, privacy, fairness, and actionability. A feature can be predictive without being useful for intervention. It may also act as a proxy for a protected characteristic, creating fairness risks even when the protected attribute itself is excluded.

Success Criteria and Evaluation Measures

Success must be defined at three levels.

The choice of model metric depends on the decision. For binary classification:

Here, TP is true positives, TN true negatives, FP false positives, and FN false negatives. Accuracy is appropriate only when class proportions and error costs make it meaningful. Precision measures the proportion of predicted positives that are truly positive. Recall measures the proportion of actual positives identified. These formulas assume a defined classification threshold and mutually exclusive outcomes.

A model that identifies fraud may prioritise recall because missed fraud is costly. A marketing campaign with limited capacity may prioritise precision because contacting unsuitable customers is expensive. Threshold selection should therefore reflect costs and capacity rather than defaulting to 0.5.

For a numerical target, mean absolute error is often interpretable:

Here, n is the number of observations, yᵢ is the observed value, and ŷᵢ is the prediction. MAE has the same units as the target and treats errors linearly. It assumes that the average absolute deviation is a meaningful measure of performance.

Costs, Benefits, and Expected Business Value

Prediction creates value only when it changes an action and improves an outcome. A simple decision-value model compares the benefits and costs of intervention.

For an intervention applied to a group:

This is a conceptual accounting relationship rather than a universal model. Each component must be defined in monetary or comparable units.

A more explicit expected-value calculation for individual case i is:

where pᵢ is the estimated probability that the desired outcome will occur, Bᵢ is the benefit if it occurs, and Cᵢ is the intervention cost. This expression assumes that the intervention benefit is proportional to the estimated probability and that costs and benefits are measured on the same time and monetary basis. If the intervention changes the probability rather than merely identifying likely outcomes, the relevant quantity is the incremental probability caused by the intervention, not the raw predicted probability.

For example, if a retention offer costs £4 per customer, has a benefit of £30 when it prevents a cancellation, and the estimated probability of a preventable cancellation is 0.20, then:

EV = 0.20 × £30 − £4 = £2.

The expected value is £2 per contacted customer before considering staff capacity, targeting accuracy, and whether the offer actually causes retention. A high-risk customer may not be persuadable; therefore, uplift or treatment-effect modelling may be more appropriate than ordinary churn prediction when the decision is about whom to treat.

Constraints and Feasibility

Common constraints include:

• Time: predictions must arrive before the decision deadline.
• Budget: data acquisition, software, staff, and interventions have costs.
• Capacity: a call centre may contact only 500 customers per day.
• Data access: legal, contractual, or technical restrictions may apply.
• Privacy: personal data must be collected and used for legitimate purposes.
• Explainability: decisions affecting employment, credit, insurance, or welfare may require understandable reasons.
• Reliability: a model must continue to operate when data sources fail.
• Fairness: performance and outcomes should be examined across relevant groups.

Constraints should be identified before modelling. A model that requires data unavailable in production is not feasible, even if its validation score is excellent.

A Structured Problem-Framing Template

A practical project statement can use the following structure:

For [decision-maker], we will support [decision] by predicting [target] for [unit of analysis] at [prediction time], using information available during [observation window], over [prediction horizon]. Success means [business outcome], subject to [constraints]. The prediction will trigger [action], and value will be assessed using [financial and operational measures].

Illustration:

For the customer-retention team, we will support weekly offer allocation by predicting whether each active customer will cancel within 30 days, using information available at the weekly scoring date. Success means reducing avoidable cancellations by 5% without exceeding the £20,000 monthly offer budget, while monitoring performance across customer groups.

This statement is sufficiently specific to guide data collection, modelling, testing, deployment, and evaluation.

Illustrative Case: Predicting Late Deliveries

Background

A fictional online retailer, Northstar Home, receives complaints about late deliveries. Management wants to improve customer communication and allocate limited service-recovery resources.

Problem

The initial request is: “Build a model to predict late orders.” This is incomplete. The team clarifies that one row will represent one order at the time it is dispatched. The target is whether delivery occurs more than two days after the promised date. The prediction horizon extends from dispatch to the promised delivery date plus two days.

Analysis

The business decision is whether to send an early warning message and prioritise the order for logistics review. Available features include destination region, carrier, dispatch weekday, package weight, warehouse, weather category available at dispatch, and historical carrier performance calculated only from previous orders.

The team rejects actual delivery scan events as features because these occur after dispatch and would not be available when the decision is made. It also examines whether carrier history is calculated using future records; if so, the feature would leak information.

Success is defined at several levels:

• Business: reduce complaints about late delivery by 8% in a pilot.
• Operational: produce a score within 15 minutes of dispatch.
• Capacity: logistics staff can review 300 orders per day.
• Model: achieve useful precision in the highest-risk 300 orders, compared with the existing rule.

Approach and Solution

The team compares three approaches: the existing rule, a simple logistic classification model, and a tree-based model. The test data is selected by time rather than randomly, so that later orders simulate future use. The model is evaluated on precision at the operational capacity of 300 orders, calibration, subgroup performance, and complaint reduction in a controlled pilot.

The model is not automatically deployed merely because it has the highest technical score. The pilot compares model-supported operations with the existing process. Staff receive explanations such as “carrier-region combination has historically experienced delays,” rather than unexplained scores.

Outcome

In this illustrative case, the pilot shows fewer late-delivery complaints and no increase in unnecessary interventions. The organisation adopts the model with weekly monitoring, monthly data-quality checks, and a review rule if precision falls below the agreed threshold for two consecutive weeks.

Learning Points

1. The unit of analysis and prediction time determine which data is legitimate.
2. Operational capacity affects the relevant metric.
3. A time-based test better represents future deployment than an indiscriminate random split.
4. Business impact must be measured through a pilot or other evaluation of the action, not only through model metrics.
5. Monitoring and ownership are part of deployment, not optional additions.

Ethics, Governance, and Responsible Use

Predictive analytics can distribute opportunities, prices, attention, or sanctions. Governance should therefore cover purpose limitation, lawful data use, data minimisation, security, retention, access control, transparency, and accountability. Depending on jurisdiction and context, privacy and data-protection laws may require a lawful basis, notices, rights of access, correction, objection, or human review.

Fairness cannot be established by a single metric. Analysts should examine representation, error rates, calibration, treatment rates, and outcomes across relevant groups. Metrics can conflict: improving equal recall may reduce calibration equality, for example. The appropriate standard depends on the decision, legal requirements, stakeholder impact, and available evidence.

Human oversight must be meaningful. A human who merely approves an automated recommendation without time, information, or authority to challenge it is not providing effective oversight. Documentation should state the model’s purpose, population, training data, limitations, intended use, prohibited use, performance, and monitoring arrangements.

Common Failure Modes and Best Practices

Failure Modes

• Starting with a preferred algorithm instead of a decision.
• Defining the target after examining convenient data.
• Using post-outcome information through leakage.
• Optimising accuracy while ignoring intervention costs.
• Treating correlation as evidence of causation.
• Randomly splitting time-dependent data without justification.
• Ignoring changes in data definitions or operational processes.
• Deploying a model without a named owner or monitoring plan.

Best Practices

1. Write the decision and action before selecting a model.
2. State the unit, prediction time, observation window, and horizon explicitly.
3. Create a data dictionary and record variable availability.
4. Establish business, operational, and model success criteria together.
5. Compare with a credible baseline.
6. Use validation that reflects future deployment.
7. Quantify costs, benefits, capacity, and uncertainty.
8. Involve domain experts, data specialists, legal advisers, and affected stakeholders.
9. Pilot the decision process where possible.
10. Monitor performance, drift, fairness, data quality, and unintended consequences.

Chapter Summary

Business problem framing is the foundation of predictive analytics. A broad challenge must be converted into a decision-oriented predictive question that identifies the entity, prediction time, horizon, target, action, constraints, and success criteria. CRISP-DM provides an iterative process covering business understanding, data understanding, preparation, modelling, evaluation, and deployment.

The unit of analysis determines what each record represents. The observation window defines historical information available for predictors, while the prediction horizon defines the future period in which the target is measured. Precise target definitions prevent ambiguity and leakage. Model metrics such as precision, recall, accuracy, and MAE must be selected according to decision costs and operational capacity.

A technically accurate model is not automatically valuable. Expected business value depends on the benefit of improved decisions, intervention costs, error costs, capacity, and implementation feasibility. Responsible practice also requires privacy, fairness, explainability, security, human oversight, and monitoring. The most effective analytics projects connect a measurable prediction to a feasible action and evaluate whether that action improves outcomes.

Key Terms

Business objective: The organisational result an analytics project seeks to improve.

Business problem: A situation requiring an organisational decision or intervention.

Predictive question: A precise question about a future outcome for a defined population and time horizon.

CRISP-DM: An iterative process consisting of Business Understanding, Data Understanding, Data Preparation, Modelling, Evaluation, and Deployment.

Unit of analysis: The entity represented by one record in an analytical dataset.

Prediction time: The point at which information is available and a prediction is generated.

Observation window: The historical period used to calculate predictor variables.

Prediction horizon: The future period during which the target outcome is measured.

Target variable: The outcome a model is designed to predict.

Feature: An input variable used by a predictive model.

Data leakage: Use of information unavailable at prediction time or derived from the future outcome.

Baseline: A simple or existing method used for comparison with a proposed model.

Precision: The proportion of predicted positive cases that are actually positive.

Recall: The proportion of actual positive cases identified by the model.

Accuracy: The proportion of all classifications that are correct.

Expected value: The anticipated benefit of a decision after considering probabilities and costs.

Data drift: Change over time in the distribution of model input data.

Concept drift: Change over time in the relationship between inputs and the target.

Model governance: Policies and controls supporting responsible development, use, monitoring, and review of models.

Uplift: The incremental change in outcome caused by an intervention compared with not receiving it.

Review Questions

Short Answer Questions

1. What is the difference between a business problem and a predictive question?
2. Define the unit of analysis and give one example.
3. What is data leakage, and why does it damage model evaluation?
4. Distinguish precision from recall.
5. Name the six phases of CRISP-DM.

Long Answer Questions

1. Explain how a broad business challenge can be converted into a measurable predictive question.
2. Discuss why the observation window, prediction time, and prediction horizon must be documented.
3. Explain why technical model performance and business value are different forms of success.
4. Describe the role of evaluation and deployment in an iterative CRISP-DM process.
5. Discuss the main ethical and governance issues in using predictive analytics for customer decisions.

Analytical / Application Questions

1. A university wants to identify students at risk of withdrawing. Propose a unit of analysis, target definition, observation window, prediction horizon, action, and two success criteria. Explain one potential leakage risk.
2. A call centre can contact only 1,000 customers each week. Explain how this capacity constraint should influence model evaluation and threshold selection.
3. A retailer predicts which customers are likely to buy after receiving a discount. Explain why ordinary purchase prediction may be insufficient for deciding whom to discount, and identify a more decision-relevant analytical approach.

Multiple Choice Questions with Answers

Question 1

Which statement best describes a predictive question?

A. Sales are lower than last year.

B. Which active customers are likely to cancel within 30 days before an offer is sent?

C. The organisation should improve retention.

D. Customer service is an important function.

Correct Answer: B. Which active customers are likely to cancel within 30 days before an offer is sent?

Question 2

In a customer churn project, what is the unit of analysis if each row represents one customer at the weekly scoring date?

A. One customer-week scoring record

B. One product category

C. One call-centre employee

D. One month of company revenue

Correct Answer: A. One customer-week scoring record

Question 3

Which CRISP-DM phase focuses primarily on clarifying objectives, stakeholders, constraints, and success criteria?

A. Data Preparation

B. Modelling

C. Business Understanding

D. Deployment

Correct Answer: C. Business Understanding

Question 4

A feature records a customer’s cancellation reason, but the model is generated before cancellation. Using this feature would most likely create:

A. Class imbalance

B. Data leakage

C. Underfitting

D. Random sampling

Correct Answer: B. Data leakage

Question 5

Which metric is the proportion of predicted positive cases that are actually positive?

A. Recall

B. Accuracy

C. Precision

D. Mean absolute error

Correct Answer: C. Precision

Question 6

A model identifies 80 true positive cases and 20 false positive cases. What is its precision?

A. 20%

B. 25%

C. 75%

D. 80%

Correct Answer: D. 80%

Question 7

Why might a time-based test split be preferable to a random split for a forecasting application?

A. It always produces a higher accuracy.

B. It better represents future deployment conditions.

C. It removes the need for a target variable.

D. It guarantees that no data is missing.

Correct Answer: B. It better represents future deployment conditions.

Question 8

If an intervention costs £4, provides a £30 benefit when successful, and the relevant estimated probability is 0.20, what is the expected value per intervention before other costs?

A. £2

B. £6

C. £26

D. £34

Correct Answer: A. £2

Question 9

Which statement about model deployment is most accurate?

A. Deployment ends the analytics process permanently.

B. Only model accuracy matters after deployment.

C. Deployment requires ownership, monitoring, and procedures for change or failure.

D. A deployed model never needs retraining.

Correct Answer: C. Deployment requires ownership, monitoring, and procedures for change or failure.

Question 10

Why may ordinary churn prediction be insufficient when deciding whom to target with a retention offer?

A. Churn prediction cannot use customer data.

B. The highest-risk customers may not be persuadable by the offer.

C. Churn is always a continuous variable.

D. Retention offers have no measurable costs.

Correct Answer: B. The highest-risk customers may not be persuadable by the offer.