Business Problem Formulation and Analytics Workflow

Introduction

Supervised and unsupervised learning are not ends in themselves. They are means of supporting decisions, improving processes, allocating resources, managing risk, and discovering patterns. The quality of an analytics project therefore depends less on selecting an impressive algorithm than on formulating the right business problem and connecting model outputs to actions.

A strategic question such as “How can we improve customer retention?” is not yet a machine-learning problem. It may become a prediction problem if the organization wants to estimate which customers are likely to churn, a classification problem if customers must be assigned to high-, medium-, or low-risk groups, a ranking problem if a limited retention budget must be allocated to the most promising customers, or a causal decision problem if management wants to know which intervention will reduce churn. Similarly, “How can we understand our customers?” may lead to clustering, association discovery, dimensionality reduction, or descriptive segmentation.

This chapter presents a decision-oriented workflow for translating business questions into analytics tasks. It combines CRISP-DM with related ideas from the data-to-value chain, model risk management, design thinking, and operations management. Particular attention is given to the unit of analysis, target-variable definition, constraints, baselines, business value, validation, and deployment.

Learning Objectives

By the end of this chapter, you should be able to:

1. Translate strategic and operational questions into prediction, classification, ranking, clustering, or discovery tasks.
2. Define the unit of analysis, observation window, prediction horizon, and target variable.
3. Distinguish prediction from causal explanation and decision optimization.
4. Use CRISP-DM and related frameworks to structure an analytics project.
5. Establish meaningful business, operational, and analytical baselines.
6. Specify constraints, success criteria, and evaluation metrics that reflect business value.
7. Identify leakage, bias, governance, privacy, and implementation risks.
8. Evaluate an analytics proposal through a complete applied case.

1. From Business Questions to Analytics Tasks

1.1 The translation process

A useful formulation moves through five levels:

Business objective → decision → analytical task → data design → evaluation criterion

For example:

• Business objective: reduce avoidable loan losses.
• Decision: determine which applications require additional review.
• Analytical task: estimate default risk and rank applications for review.
• Data design: one row per application, with information available at application time.
• Evaluation criterion: identify as many true high-risk applications as possible within the review team’s capacity, while controlling unnecessary referrals.

This translation prevents a common error: treating a model metric as the business objective. A high classification accuracy is not automatically valuable if the model is used on a highly imbalanced population, produces too many false alarms, or cannot be integrated into the workflow.

1.2 Common analytical task types

These categories can overlap. A churn model may generate probabilities, classify customers using a threshold, rank customers for contact, and support a budget allocation decision. The appropriate formulation depends on the decision, not merely on the available algorithm.

1.3 Prediction is not explanation or causation

A predictive model estimates an outcome from available information. It need not identify the causes of that outcome. A variable may be highly predictive of employee turnover without being an appropriate intervention target. For example, a recent transfer may predict resignation, but eliminating transfers may be infeasible or harmful.

Causal questions ask what would happen under an intervention. “Which customers will churn?” is predictive. “Which customers will remain if offered a discount?” is closer to a treatment-effect question. The second question requires information about interventions, outcomes under comparable conditions, and assumptions supporting causal inference. Confusing these questions can lead to ineffective or discriminatory decisions.

2. Defining the Analytical Object

2.1 Unit of analysis

The unit of analysis is the entity represented by one observation in the modeling dataset. It may be a customer, transaction, account-month, store-day, machine, employee, claim, or product-week.

The unit must match the decision. If the business contacts customers individually, a customer-level unit may be appropriate. If staffing decisions are made by store and day, a store-day unit may be more useful. Choosing the wrong unit creates aggregation bias, duplicated records, invalid comparisons, or operational outputs that cannot be acted upon.

A unit-of-analysis specification should state:

• the entity being observed;
• the time at which the observation is created;
• the features available at that time;
• the outcome observation window;
• the population to which the analysis applies; and
• the action associated with a model output.

2.2 Observation window and prediction horizon

Suppose a retailer wants to predict whether a customer will churn in the next 90 days. A defensible record might contain customer behavior during the 180-day observation window ending on the scoring date, with churn measured during the subsequent 90-day prediction horizon.

The temporal structure can be represented as:

Features must be measured no later than the scoring date. The target must be measured after the scoring date. If information from the prediction horizon is used as a feature, the model suffers from target leakage and its apparent performance will be overstated.

2.3 Target-variable definition

A target variable is the outcome the model is intended to estimate or predict. It must be operationally defined rather than described vaguely.

For example, “customer churn” might mean any of the following:

• account cancellation;
• no purchase for 90 consecutive days;
• failure to renew a subscription;
• reduction in usage below a contractual threshold.

Each definition produces a different population, label rate, cost structure, and intervention strategy. A target specification should include its unit, coding, time horizon, inclusion rules, exclusion rules, and treatment of missing or censored outcomes.

For binary classification, a target may be represented as:

Here, y_i is the target for entity i. This notation applies only when the event is mutually exclusive and the observation window is sufficiently defined to determine whether the event occurred.

2.4 Population and sampling frame

The modeling population must be distinguished from the available sample. A model trained on customers who have supplied extensive digital data may perform poorly for customers who interact primarily through physical channels. Similarly, a credit model developed from approved applicants may not generalize to rejected applicants because the outcomes of rejected applications are unobserved.

Before modeling, analysts should document eligibility, coverage, exclusions, missingness, and changes in the population over time. Generalization is an empirical question, not an assumption.

3. Formulating Tasks for Supervised and Unsupervised Learning

3.1 Supervised learning

In supervised learning, historical examples contain input variables and an observed outcome. The objective is to learn a relationship that can estimate outcomes for new cases.

Regression predicts a numeric quantity, such as revenue, delivery time, or loss amount. Classification predicts a category or event, such as fraud versus non-fraud. Ranking orders cases by a score, often for a constrained intervention. Forecasting is a time-dependent form of prediction in which temporal ordering and future information are especially important.

The target form should reflect the decision. If a manager can review only 500 cases, ranking may be more appropriate than producing a yes/no classification for every case. If different errors have materially different consequences, a probability score may be more informative than a fixed class label.

3.2 Unsupervised learning

Unsupervised learning is used when a target outcome is not supplied or is not considered adequate for the question. Clustering can identify groups with similar behavior, but a mathematical cluster is not automatically a meaningful business segment. A useful segment should be stable, interpretable, sufficiently large, and connected to a possible action.

Discovery tasks also include association analysis, anomaly detection, and representation learning. These methods are exploratory and can generate hypotheses. They do not, by themselves, establish causality or business relevance.

A clustering project should therefore specify:

• the entities being grouped;
• the variables and transformations used;
• the similarity or distance concept;
• the desired level of granularity;
• stability and validation tests; and
• the action that might follow from each segment.

3.3 Ranking and prioritization

Ranking is often the most realistic formulation when resources are scarce. A sales team may contact only 1,000 leads per week; an inspection agency may inspect only a fixed number of sites; a hospital may have limited appointment capacity.

The ranking problem should specify the eligible population, capacity constraint, scoring frequency, and outcome used to assess the top-ranked cases. A model that performs well across the entire population may be inferior at the top of the list if the business acts only on the highest-scored cases.

4. CRISP-DM and Related Frameworks

CRISP-DM—Cross-Industry Standard Process for Data Mining—organizes analytics work into six iterative phases:

1. Business understanding: clarify objectives, stakeholders, decisions, constraints, and value.
2. Data understanding: identify sources, inspect quality, examine distributions, and assess data-generating processes.
3. Data preparation: select, integrate, clean, transform, and document data.
4. Modeling: choose techniques, construct models, and tune parameters.
5. Evaluation: assess technical validity and business suitability before deployment.
6. Deployment: operationalize outputs, monitor performance, and capture learning.

The phases are not strictly linear. Evaluation may reveal that the target is poorly defined; data understanding may expose an infeasible business objective; deployment may reveal that a model’s outputs arrive too late to support action.

A related decision-oriented workflow adds two explicit elements: intervention design and measurement after implementation. These are important because a predictive model creates value only when its output changes a decision and that decision improves an outcome.

4.1 The analytics workflow as a control loop

A mature workflow can be described as:

This is a control loop rather than a one-time project. Data distributions change, customer behavior adapts, policies alter outcomes, and model performance can deteriorate. Monitoring and revision are therefore part of the analytical design.

5. Baselines, Constraints, and Success Criteria

5.1 Baselines

A baseline is a reference against which an analytical method is judged. Suitable baselines may include:

• current business practice;
• random selection;
• a majority-class rule;
• historical average;
• seasonal or naïve forecast;
• simple linear or logistic model;
• last-period performance; or
• a rule created by domain experts.

Without a baseline, “improvement” has no defensible meaning. A complex model should justify its additional cost, risk, and maintenance burden by outperforming a credible simple alternative.

5.2 Constraints

Constraints may be statistical, operational, legal, ethical, or financial. Examples include:

• limited staff or contact capacity;
• response-time requirements;
• minimum precision or recall for a protected process;
• prohibited features;
• interpretability requirements;
• data-retention rules;
• budget limits;
• fairness constraints; and
• restrictions on automated decisions.

Constraints should be written before model selection where possible. Otherwise, a team may optimize a metric that later proves unusable.

5.3 Business value and expected value

A simple decision-value model can clarify whether prediction is worthwhile. Suppose an intervention is applied to selected cases. Let N be the number of selected cases, p the probability that a selected case produces a successful incremental outcome, B the benefit per success, and C the cost per intervention. A simplified expected net value is:

This formula assumes that probabilities are meaningful, benefits and costs are measured on a comparable monetary basis, cases are sufficiently independent for planning purposes, and the intervention effect is not already captured in the baseline. In practice, p should ideally represent the incremental probability caused by the intervention, not merely the probability of the outcome.

For a retention campaign, a model may identify customers likely to churn, but the financially relevant quantity is often the expected value of contacting each customer. A high-risk customer who would leave regardless of the offer may be less valuable than a moderate-risk customer who is persuadable.

5.4 Decision-oriented metrics

Metric selection should follow the decision. Common measures include:

• Accuracy: proportion of correct classifications. It can be misleading under class imbalance.
• Precision: proportion of predicted positives that are actual positives.
• Recall or sensitivity: proportion of actual positives identified.
• Specificity: proportion of actual negatives correctly identified.
• Mean absolute error: average absolute prediction error for numeric outcomes.
• Root mean squared error: square root of average squared error; it gives greater weight to large errors.
• Lift: performance in a selected group relative to a baseline group.
• Calibration: agreement between predicted probabilities and observed event frequencies.
• Top-k value: business value generated by acting on the highest-ranked k cases.

For ranking, top-k evaluation and cumulative gains may be more relevant than overall accuracy. For clustering, internal measures such as silhouette width can help assess separation, but managerial usefulness and stability must also be evaluated.

6. Data Design and Leakage Prevention

6.1 Feature availability

Each feature must be available at the moment the decision is made. A useful data dictionary should include the variable name, definition, source, time stamp, unit, missingness, permissible use, and known limitations.

Leakage can occur when:

• a feature is recorded after the target event;
• a data-cleaning process uses future information;
• a post-outcome status code is mistaken for a predictor;
• records from the future enter the training set; or
• the sampling process depends on the outcome.

Randomly splitting time-dependent data can also create leakage because observations from the future may influence evaluation of past-like cases. Temporal validation is usually more appropriate when deployment predicts future events.

6.2 Data quality and measurement error

A model cannot recover information that was never measured reliably. Analysts should examine missingness mechanisms, inconsistent definitions, duplicate entities, changing coding practices, outliers, and delayed updates. Missingness may itself be informative, but treating every missing value as a meaningful signal without investigating the process can create unstable or unfair predictions.

6.3 Reproducibility and documentation

A professional project should preserve the data extraction logic, feature definitions, code version, model parameters, training dates, validation design, assumptions, and approval decisions. Reproducibility is not merely a research ideal; it supports auditability, incident investigation, and maintenance.

7. Governance, Ethics, and Responsible Formulation

Analytics projects can create harm even when their predictions are statistically accurate. Risks include privacy violations, discriminatory outcomes, excessive surveillance, manipulation, and denial of opportunities.

Responsible formulation asks:

• Is the decision legitimate and proportionate?
• Are individuals informed where required?
• Is the data collected for an appropriate purpose?
• Could protected groups experience disparate error rates or impacts?
• Is human review available for consequential decisions?
• Can the organization explain and challenge an outcome?
• Are retention, access, and security controls adequate?

Fairness cannot be reduced to a single universal metric. Different fairness criteria may conflict, especially when base rates differ across groups. The appropriate analysis depends on the legal setting, decision context, harm model, and institutional responsibilities. Governance should be designed at the problem-formulation stage, not added after model development.

Generative AI and automated feature construction may accelerate exploration, but they do not remove the need for target definition, provenance, validation, privacy controls, and human accountability. An automatically generated feature can encode sensitive information or leakage just as a manually created feature can.

8. Illustrative Case: Retention Prioritization at Northstar Telecom

The following is a fictional illustrative case.

Background

Northstar Telecom serves two million mobile customers. Its retention team can make approximately 40,000 targeted contacts each month. Management observes increasing cancellations and asks the analytics team to “build a churn model.”

Problem formulation

The team first clarifies the decision. The goal is not simply to predict churn; it is to prioritize customers for an intervention that may prevent avoidable cancellations.

The proposed unit is a customer-month: one record for each eligible active customer at the monthly scoring date. Features include recent usage, payment history, service incidents, tenure, plan changes, and contact history, provided they are available by the scoring date.

The target is cancellation within the following 60 days. Customers who are already in a cancellation process at the scoring date are excluded because they are not eligible for the proposed intervention. The target is therefore a binary variable with value 1 for cancellation during the 60-day horizon and 0 otherwise.

Baseline and constraints

The current process contacts customers selected by service managers. A second baseline randomly selects eligible customers. Capacity is fixed at 40,000 contacts per month. The retention offer costs £8 per contact, and the estimated contribution margin preserved by a successful incremental retention is £120. These estimates require financial review and should not be treated as established facts merely because they appear in a planning model.

The team agrees that performance will be assessed using:

• precision and recall among the top 40,000 ranked customers;
• calibration of predicted churn probabilities;
• incremental retention measured through a controlled experiment;
• net value after contact and offer costs; and
• performance and error analysis across relevant customer groups.

Analysis and modeling approach

The data is divided by time: earlier months are used for development and later months for validation. This reflects the intended future-use setting. A simple logistic-regression model and a gradient-boosting model are compared with the manager-selection baseline. Features created after the scoring date are removed. In particular, a “cancellation-request status” field is excluded because it directly reveals the outcome process.

The model score is used for ranking rather than applying an arbitrary probability threshold. This matches the fixed capacity constraint. The team also examines whether the highest-ranked customers are persuadable. A customer with a high predicted cancellation risk may not benefit from an offer if cancellation is caused by an unavoidable relocation or a permanently discontinued service.

Experimental evaluation

Northstar uses a randomized controlled test among eligible high-ranked customers. Some receive the retention intervention and others receive the ordinary service process. This estimates incremental impact more directly than comparing contacted customers with non-contacted customers, because contact decisions may otherwise be confounded by customer risk.

Suppose, illustratively, that 20,000 high-ranked customers are randomly divided equally between treatment and control. The treatment group has a 14% cancellation rate and the control group has a 16% rate. The estimated absolute reduction is:

Thus, the estimated incremental retention rate is 2 percentage points under the assumptions of valid randomization, adequate implementation, and comparable follow-up. If the treated group contains 10,000 customers, the estimated number of incremental retained customers is:

If the contribution margin per retained customer is £120 and the contact cost is £8 per treated customer, the illustrative net value is:

The intervention would therefore have negative net value under these assumed figures, despite the model successfully identifying a group with elevated churn risk. This result demonstrates the distinction between predictive performance and decision value. Management might redesign the offer, target a more persuadable subgroup, reduce cost, or discontinue the intervention.

Outcome and learning points

The case shows why the correct objective is “maximize incremental retention value under contact capacity,” not “maximize churn-model accuracy.” It also shows why causal evaluation is needed when the business action itself changes outcomes. A ranking model can be technically strong and still fail commercially if the intervention is ineffective or too expensive.

Key lessons include:

1. Define the decision before selecting the algorithm.
2. Establish the unit, time windows, and eligibility rules explicitly.
3. Prevent leakage by enforcing feature availability at scoring time.
4. Compare with operational and simple analytical baselines.
5. Evaluate the action, not only the prediction.
6. Use experimentation when incremental impact is central to value.
7. Monitor performance, costs, fairness, and behavioral adaptation after deployment.

9. Common Failure Modes and Best Practices

Failure modes

Starting with a method. Selecting deep learning or clustering before defining the decision often produces technically elaborate but unusable work.

Vague targets. Terms such as “engagement,” “quality,” and “risk” conceal multiple possible operational definitions.

Wrong unit of analysis. Aggregating transactions to customers can hide temporal variation; using transaction-level records for a customer-level intervention can duplicate customer decisions.

Ignoring capacity. A model may identify many high-risk cases when the organization can act on only a small fraction.

Optimizing a proxy. A measurable outcome may not represent the value or harm that management actually cares about.

Treating correlation as intervention guidance. Predictive variables are not automatically effective levers.

Skipping deployment design. A model that cannot be delivered at the required frequency, latency, or level of explanation is not operationally complete.

Best practices

• Write a one-page problem statement before modeling.
• Identify the decision owner and affected stakeholders.
• Specify the unit, timing, target, population, and action.
• Define a baseline and minimum acceptable improvement.
• Make constraints explicit and test them early.
• Separate exploratory analysis from confirmatory evaluation.
• Use temporal validation when predicting future outcomes.
• Document assumptions, data provenance, and limitations.
• Measure incremental business outcomes where interventions are involved.
• Establish monitoring, retraining, escalation, and retirement criteria.

Chapter Summary

Business analytics begins with problem formulation, not algorithm selection. A strategic question must be translated into a decision, an analytical task, a data structure, and a measurable success criterion. Prediction, classification, ranking, clustering, anomaly detection, causal inference, and optimization answer different questions and require different evidence.

The unit of analysis, observation window, prediction horizon, target definition, and eligible population determine what a model means and where it can be used. CRISP-DM provides a practical iterative structure through business understanding, data understanding, preparation, modeling, evaluation, and deployment. Baselines, operational constraints, and decision-oriented metrics prevent technical performance from being mistaken for business value.

Effective analytics evaluates not only whether a model predicts well, but whether its output supports a feasible, ethical, and valuable action. Leakage prevention, temporal validation, governance, fairness assessment, experimentation, monitoring, and documentation are therefore integral to the workflow. The central principle is simple: formulate the decision carefully, then build only the analytical capability required to improve it.

Key Terms

Unit of analysis: The entity represented by one observation in an analytical dataset.

Observation window: The period during which predictor information is collected.

Prediction horizon: The future period over which an outcome is predicted.

Target variable: The defined outcome a supervised-learning model estimates.

Supervised learning: Learning from examples containing inputs and observed outcomes.

Unsupervised learning: Discovering structure in data without a supplied target outcome.

Classification: Assigning observations to categories or estimating category probabilities.

Regression: Predicting a numeric quantity.

Ranking: Ordering cases according to a score or priority.

Clustering: Grouping observations according to similarity.

Anomaly detection: Identifying observations that differ substantially from expected patterns.

CRISP-DM: An iterative framework covering business understanding, data understanding, preparation, modeling, evaluation, and deployment.

Baseline: A reference method or current practice used for comparison.

Data leakage: Use of information unavailable at the time a real prediction would be made.

Calibration: Agreement between predicted probabilities and observed event frequencies.

Constraint: A requirement limiting feasible analytical or operational solutions.

Incremental effect: The change in outcome caused by an intervention relative to an appropriate comparison condition.

Concept drift: Change over time in relationships between inputs, outcomes, or populations.

Decision rule: The procedure that converts an analytical output into an action.

Governance: Policies and controls supporting lawful, ethical, secure, and accountable analytics.

Review Questions

Short Answer Questions

1. Why must a business objective be translated into a decision before an analytical task is selected?
2. What is the difference between an observation window and a prediction horizon?
3. Define data leakage and give one example.
4. Why might ranking be preferable to binary classification when organizational capacity is limited?
5. What is the purpose of a baseline in an analytics project?

Long Answer Questions

1. Explain how the choice of unit of analysis can affect model interpretation, validation, and operational usefulness.
2. Compare prediction, causal inference, and optimization, using a business example to distinguish them.
3. Describe the six phases of CRISP-DM and explain why the process is iterative rather than strictly linear.
4. Discuss how technical metrics and business-value metrics can lead to different conclusions about an analytics project.
5. Explain the principal ethical and governance issues that should be addressed before deploying a consequential model.

Analytical / Application Questions

1. A bank can manually review only 2,000 loan applications per week. Formulate an analytics problem that uses ranking, and specify the unit of analysis, available features, target, prediction horizon, baseline, constraints, and evaluation metrics.
2. A retailer wants to create customer segments but has no agreed definition of a valuable customer. Design a responsible clustering workflow, including variable selection, validation, interpretation, and the conditions under which the segments should be used.
3. A hospital develops a model predicting readmission and plans to provide additional discharge support to high-risk patients. Explain why predictive accuracy alone is insufficient and design an evaluation approach that measures incremental clinical and economic value.

Multiple Choice Questions with Answers

Question 1

Which sequence best represents a decision-oriented analytics workflow?

A. Algorithm selection → data collection → business objective → deployment

B. Business objective → decision → analytical task → data design → evaluation

C. Data cleaning → dashboard design → algorithm selection → business objective

D. Deployment → target definition → sampling → stakeholder identification

Correct Answer: B. Business objective → decision → analytical task → data design → evaluation

Question 2

What is the unit of analysis?

A. The algorithm used to generate predictions

B. The time period used for model validation

C. The entity represented by one observation in the dataset

D. The metric used to evaluate classification

Correct Answer: C. The entity represented by one observation in the dataset

Question 3

A feature records whether a customer submitted a cancellation request after the scoring date. Using it to predict cancellation at the scoring date is an example of:

A. Regularization

B. Data leakage

C. Clustering

D. Calibration

Correct Answer: B. Data leakage

Question 4

Which task is most appropriate when a sales team can contact only 500 leads and needs to prioritize them?

A. Ranking

B. Random sampling

C. Unconstrained clustering

D. Dimensionality reduction only

Correct Answer: A. Ranking

Question 5

Which CRISP-DM phase focuses on clarifying stakeholders, objectives, decisions, and business value?

A. Data preparation

B. Modeling

C. Business understanding

D. Deployment

Correct Answer: C. Business understanding

Question 6

If a model predicts a 0.30 probability for a group of cases, calibration concerns whether:

A. The model uses exactly 30 features

B. Approximately 30% of those cases experience the event

C. The model has 30% classification accuracy

D. The model ranks exactly 30 cases

Correct Answer: B. Approximately 30% of those cases experience the event

Question 7

In the illustrative retention case, 10,000 treated customers experience a 2-percentage-point incremental retention effect. How many incremental retained customers are estimated?

A. 20

B. fifty

C. 200

D. 2,000

Correct Answer: C. 200

Question 8

Why is a predictive variable not necessarily an appropriate intervention target?

A. Prediction always requires randomized data

B. Association with an outcome does not establish that changing the variable will cause the outcome to change

C. Predictive variables cannot be measured numerically

D. Intervention targets must never be included in a model

Correct Answer: B. Association with an outcome does not establish that changing the variable will cause the outcome to change

Question 9

Which method is generally most appropriate for evaluating whether a retention intervention causes incremental improvement?

A. Comparing contacted customers with all non-contacted customers without adjustment

B. A randomized controlled experiment

C. Selecting the model with the highest training accuracy

D. Removing the outcome variable from evaluation

Correct Answer: B. A randomized controlled experiment

Question 10

Why should temporal validation often be used for future-outcome prediction?

A. It guarantees perfect accuracy

B. It prevents the model from using any features

C. It better reflects the ordering and information conditions of deployment

D. It eliminates the need for a baseline

Correct Answer: C. It better reflects the ordering and information conditions of deployment