2. Data Science Life Cycle and Problem Formulation

Introduction: From Organizational Need to Analytical Work

Data science projects rarely fail because a statistical algorithm is unavailable. They more often fail because the team solves the wrong problem, uses an unsuitable definition of success, misunderstands the decision context, or produces an output that cannot be adopted by the organization. A data science life cycle provides a disciplined way to move from an organizational need to a useful, responsible, and maintainable analytical solution.

The life cycle is not merely a sequence of technical tasks. It connects business strategy, domain knowledge, data, analytical methods, technology, human judgment, and organizational change. A project may begin with a business concern such as rising customer churn, delayed hospital appointments, fraudulent transactions, or inefficient inventory. These concerns are not yet data science problems. They must be translated into precise analytical questions, measurable outcomes, feasible data requirements, and decisions that an organization can actually take.

This chapter examines the Cross-Industry Standard Process for Data Mining (CRISP-DM), related life-cycle approaches, and the practice of problem formulation. It emphasizes the reasoning required before model development begins and the feedback loops that continue after deployment.

Learning Objectives

After studying this chapter, the learner should be able to:

1. Explain the purpose and stages of a data science life cycle.

2. Apply CRISP-DM to a realistic organizational problem.

3. Distinguish a business problem, analytical question, prediction target, and decision rule.

4. Formulate hypotheses, requirements, assumptions, constraints, and success metrics.

5. Assess data, technical, organizational, ethical, and operational feasibility.

6. Design a practical project plan with milestones, risks, and iteration points.

7. Evaluate common problem-formulation errors and propose improvements.

8. Explain how governance, deployment, monitoring, and stakeholder adoption affect project success.

Why Problem Formulation Comes Before Modelling

An organizational problem describes an undesirable condition or an opportunity. For example, “customer churn is too high” expresses concern but does not specify what must be predicted, for whom, when, or for what action. A well-formulated data science problem narrows this concern into a decision-support task.

A useful chain is:

Organizational objective → decision → analytical question → data and method → measurable outcome.

Suppose a subscription company wants to reduce cancellations. The relevant decision might be whether to offer a retention intervention to selected customers. The analytical question could be: “Which active customers are likely to cancel within the next 30 days, using information available at the time an intervention can be made?” The target is then a defined cancellation event within a defined time window. The decision rule may involve contacting customers whose predicted risk exceeds a threshold and whose expected intervention benefit exceeds its cost.

This formulation exposes issues that a vague business statement hides. What counts as cancellation? Are suspended accounts included? Is the prediction made weekly or daily? Is the objective to maximize prevented cancellations, revenue retained, or profit after incentives? Which customers may legally or ethically be targeted? These questions determine the data, labels, evaluation metrics, and deployment design.

A technically accurate model can still be a poor organizational solution. A model with high accuracy may identify customers who cannot be influenced, while a less accurate model may be more valuable if it identifies customers who are both at risk and responsive to intervention. Thus, prediction quality and decision value are related but not identical.

CRISP-DM as a Practical Organizing Framework

CRISP-DM consists of six iterative phases: business understanding, data understanding, data preparation, modelling, evaluation, and deployment. It is widely used because it is understandable across industries and because it treats modelling as only one part of the project. The phases are not strictly linear. Findings in data preparation may change the business question; evaluation may reveal that the original success criterion was unsuitable; deployment may expose data-quality or workflow problems that require earlier phases to be revisited.

Business Understanding

Business understanding establishes why the project exists and what decision it should support. The team identifies stakeholders, organizational objectives, decision owners, constraints, available resources, and the consequences of success or failure.

A strong business-understanding document normally includes:

• the organizational problem and its context;
• the decision that analysis will inform;
• the population, unit, and time period of interest;
• desired outcomes and unacceptable outcomes;
• assumptions, constraints, and dependencies;
• project scope and explicit exclusions;
• risks, governance requirements, and stakeholders;
• technical, operational, and economic feasibility;
• criteria for continuing, revising, or stopping the project.

The team should distinguish the problem owner from the data owner, model user, and technical implementer. A marketing manager may own the retention decision, a data governance team may control customer records, and a software team may implement scoring. If these roles are not identified, important requirements may be discovered too late.

Data Understanding

Data understanding begins with an inventory of potential sources and proceeds to exploratory analysis. The team examines data structure, provenance, granularity, completeness, missingness, distributions, anomalies, definitions, and relationships between tables.

This phase asks not only whether data exist, but whether they represent the phenomenon of interest. A transaction table may record completed purchases but omit abandoned baskets. A complaints database may reflect reporting behavior rather than the true frequency of service failures. Historical labels may be affected by earlier policies, creating a feedback loop.

Data understanding should also identify leakage. Leakage occurs when information unavailable at prediction time, or information caused by the future outcome, enters the training data. For example, using a “recovery action completed” field to predict whether an account will default may produce excellent historical performance but cannot support an early warning decision.

Data Preparation

Data preparation transforms source data into an analytical dataset. Tasks may include joining sources, correcting types, handling missing values, resolving duplicates, constructing variables, defining the observation period, and creating labels. Preparation must be reproducible; undocumented manual changes make later validation and maintenance difficult.

The analytical unit must be explicit. It might be one customer, customer-month, loan application, machine-hour, medical encounter, or product shipment. Mixing units can create invalid comparisons and duplicated information. Time must also be respected. For a forecasting problem, training observations should precede validation and test observations chronologically. Random splitting may create an unrealistically easy task when the future resembles the past less closely than randomly selected records do.

Modelling

Modelling includes selecting methods, training models, tuning parameters, establishing baselines, and comparing alternatives. The simplest useful baseline should be defined first. A baseline may be a historical average, current business rule, majority class, or existing operational process. Without a baseline, a complex model cannot be judged against the value already being delivered.

Model selection should reflect the decision context. Interpretability may be essential in regulated lending or clinical settings. Prediction latency may matter in fraud detection. The cost of false positives and false negatives may be highly asymmetric. A model should therefore be selected not solely for a single score, but for its performance, stability, explainability, computational demands, and compatibility with organizational use.

Evaluation

Evaluation asks two different questions: Does the model perform adequately on data representing future use? And does the proposed solution meet the original organizational objective? These questions should not be collapsed.

For classification, useful measures may include precision, recall, specificity, F1 score, area under the ROC curve, precision-recall performance, calibration, and performance by subgroup. For regression, measures may include mean absolute error, root mean squared error, and error distributions. For ranking or recommendation, top-k performance and business conversion may be relevant.

A threshold is a decision choice, not an intrinsic property of a model. It should reflect intervention capacity, cost, risk tolerance, and the relative consequences of errors. Evaluation should also test robustness across time, geography, customer segments, and plausible changes in data quality.

Deployment

Deployment means making analytical output available for use, not simply placing a model on a server. It may involve a dashboard, an application programming interface, a batch score, an alert, a report, or integration into an operational workflow. The deployment plan should specify who receives the output, when it is generated, what action follows, how exceptions are handled, and how outcomes are recorded.

Post-deployment monitoring should cover data drift, concept drift, missingness, latency, model performance, fairness indicators, intervention outcomes, and system failures. A model may remain statistically accurate while its business value declines because customer behaviour, policy, prices, or competitors have changed. Deployment is therefore a new stage of learning rather than the end of the project.

Related Life-Cycle Perspectives

CRISP-DM is not the only useful framework. The Team Data Science Process (TDSP) emphasizes project structure, repositories, work items, reproducibility, and collaboration. The data science life cycle used in many modern organizations adds explicit stages for data engineering, experimentation, productionization, and monitoring. Agile approaches organize work into short iterations and stakeholder reviews, allowing assumptions to be tested early. MLOps extends this concern to versioning, automated testing, continuous integration, deployment, observability, and model governance.

These frameworks are complementary. CRISP-DM gives a broad reasoning structure; Agile supports iterative delivery; TDSP supports team coordination and reproducibility; MLOps supports reliable operation of models in production. A project should adopt the elements appropriate to its risk, scale, regulatory environment, and technical maturity rather than treating any framework as a rigid recipe.

From Vague Concern to Formal Analytical Specification

Problem formulation benefits from separating several related statements.

A business problem identifies an organizational difficulty or opportunity. An analytical objective states what information or estimate is needed. A target variable defines the outcome to be predicted or explained. A decision rule specifies how an analytical result will influence action. A success metric evaluates whether the result is useful. A constraint limits what may be done. A requirement describes a necessary capability or condition.

For example:

• Business problem: too many deliveries arrive late.
• Decision: prioritize shipments for intervention before promised delivery time.
• Analytical question: which shipments are likely to miss the promised time using information available at dispatch?
• Target: late delivery, defined as arrival after the promised date and time.
• Decision rule: route high-risk shipments to an approved intervention workflow.
• Technical metric: recall at the capacity-supported alert threshold.
• Business metric: reduction in late deliveries without unacceptable additional logistics cost.
• Constraint: customer address and protected attributes cannot be used for prohibited discriminatory targeting.

This structure prevents the common error of treating “build a model” as the project objective.

Hypotheses and Assumptions

A hypothesis is a proposition that can be examined with evidence. It may concern a causal mechanism, a predictive relationship, or a process condition. For example, “longer resolution times increase the probability of service cancellation” is a testable hypothesis, whereas “the dashboard will improve performance” requires an intervention design and outcome measurement.

Assumptions should be recorded separately. Examples include the stability of a business definition, the availability of a field at scoring time, the reliability of a historical label, or the willingness of staff to act on alerts. An assumption register should identify how each assumption will be tested and what contingency applies if it fails.

Requirements and Acceptance Criteria

Requirements may be functional or non-functional. Functional requirements describe what the solution must do, such as generating a weekly risk list or allowing authorized users to inspect contributing factors. Non-functional requirements concern latency, availability, security, privacy, scalability, accessibility, auditability, and maintainability.

Acceptance criteria convert expectations into observable conditions. “The system should be accurate” is weak. “At the agreed operating threshold, recall should exceed the baseline by a specified amount, scores should be available by 08:00 each business day, and every score should be traceable to a model version” is more actionable. Acceptance criteria should be agreed before results are known to reduce post hoc justification.

Measuring Success at Multiple Levels

A mature project uses a hierarchy of metrics. Business metrics describe organizational value, such as profit, retention, service time, safety incidents, or customer satisfaction. Analytical metrics describe model or analytical performance. Operational metrics describe system behaviour, including availability, latency, data freshness, and failure rates. Adoption metrics indicate whether intended users receive, understand, and act on the output.

Metrics must be connected to a measurement design. If a retention model recommends offers, comparing contacted customers with all historical customers may be misleading because previous targeting was not random. A controlled experiment, quasi-experimental design, or carefully constructed comparison may be needed to estimate incremental impact. Prediction identifies risk; it does not automatically establish that an intervention causes improvement.

A project should also define guardrail metrics. For example, a hospital scheduling system may seek to reduce waiting time while ensuring that no patient group experiences a substantial deterioration in access. A fraud system may reduce losses while keeping legitimate-customer declines below an approved level.

Feasibility and Project Planning

Feasibility assessment should be conducted early and revisited throughout the life cycle. Four dimensions are particularly important:

1. Data feasibility: Are relevant, sufficiently detailed, timely, and legally usable data available?

2. Technical feasibility: Can the solution be developed, integrated, secured, and maintained with available infrastructure and skills?

3. Organizational feasibility: Will users trust and adopt the output, and can the process support the recommended actions?

4. Economic and ethical feasibility: Do expected benefits justify costs, and are risks to individuals and society acceptable?

A project plan should contain discovery activities, data-access milestones, exploratory analysis, baseline construction, modelling iterations, validation, stakeholder review, pilot deployment, and monitoring design. Each milestone should produce a tangible artifact such as a problem statement, data dictionary, analytical dataset, baseline report, evaluation protocol, or deployment decision.

A useful planning practice is to identify a minimum viable analytical product. This might be a transparent report or baseline rule that can be tested with users before a sophisticated model is built. Early delivery reduces technical risk and reveals whether the proposed decision is valuable at all.

Illustrative Case Study: Predicting Missed Outpatient Appointments

Background

The following is an illustrative case. A regional clinic network reports that missed appointments reduce service capacity and delay care. Managers ask a data science team to “build a no-show prediction model.”

Situation and Problem Formulation

The team first interviews clinic managers, scheduling staff, clinicians, and privacy officers. It learns that appointments are scheduled through several systems, reminder messages are sent at different times, and some patients lack reliable digital access. Staff can make reminder calls for only a limited number of appointments.

The business objective is refined as: reduce avoidable missed appointments while protecting equitable access and avoiding stigmatizing patients. The decision is whether to offer an additional, approved reminder or transport-support intervention to selected appointments. The analytical question becomes: which appointments have an elevated probability of being missed, based only on information available before the intervention deadline?

The target is defined as failure to attend without an approved cancellation within the appointment window. The team excludes emergency appointments and records the prediction time. It proposes two success measures: reduction in missed appointments among eligible appointments receiving intervention, and maintenance of acceptable access and fairness indicators across relevant patient groups. Recall at the clinic’s daily call capacity is selected as an initial model metric, but it is not treated as the final business outcome.

Analysis and Data Understanding

Potential data include appointment type, lead time, prior attendance history, reminder status, clinic location, scheduling changes, and transport-support eligibility. The team discovers that a field indicating a completed reminder call is recorded after the prediction time and is therefore excluded. It also finds that historical no-show labels differ between clinics. A data dictionary and common label definition are created.

Exploration shows that appointment lead time and prior attendance are predictive, but performance differs across clinics. Missing contact information is not randomly distributed. Rather than automatically imputing it as ordinary missingness, the team treats contactability as an operational issue and assesses whether the proposed intervention is feasible for each group.

Approach and Solution

A transparent baseline based on recent attendance rates is compared with logistic regression and a tree-based model. The team uses time-based validation and evaluates calibration, recall at the available call capacity, and subgroup performance. Because the model will influence service access, scores are not used to deny appointments or label patients as irresponsible. They are used only to prioritize supportive outreach.

A pilot is designed in which eligible high-risk appointments are randomly assigned, where ethically and operationally acceptable, to usual reminders or an additional intervention. The pilot records whether the intervention was delivered, whether the patient attended, and any unintended effects. Staff receive an explanation of the purpose and limitations of the score, together with a process for overriding it.

Outcome

The pilot demonstrates whether targeted outreach produces an improvement beyond standard practice. If the model identifies risk accurately but outreach does not change attendance, the project should not claim success. The team may then investigate transport barriers, scheduling policies, or appointment suitability rather than simply tuning the model.

Learning Points

The case illustrates that the central work was not algorithm selection. It involved defining the target, respecting prediction-time information, aligning the model with intervention capacity, distinguishing prediction from causal impact, and designing safeguards. It also shows why deployment, adoption, and fairness must be considered during formulation rather than added after modelling.

Common Failure Modes and Professional Risks

A frequent failure is solution-first thinking: selecting a fashionable algorithm before establishing the decision and outcome. Another is metric substitution, in which a convenient technical score replaces the business objective. Teams may also use data because they are available rather than because they are valid representations of the problem.

Other risks include target leakage, sampling bias, survivorship bias, inconsistent labels, unexamined missingness, temporal leakage, overfitting, and evaluation on data that do not resemble deployment conditions. Organizational risks include unclear ownership, insufficient intervention capacity, lack of training, incentives that conflict with the model, and failure to document assumptions.

Ethical and governance concerns include privacy, informed use, data minimization, security, fairness, explainability, contestability, and accountability. Sensitive attributes may be needed for auditing fairness even when they are not used for prediction; this requires careful governance. Automated recommendations should not remove appropriate human review in high-impact contexts. Data protection, sector-specific regulation, records retention, and audit requirements must be considered according to jurisdiction and application.

A responsible team documents data provenance, access permissions, model versions, evaluation datasets, known limitations, intended use, prohibited use, and escalation procedures. It also plans for model retirement. A model that is no longer valid should be withdrawn rather than allowed to operate indefinitely.

Current Developments in Life-Cycle Practice

Modern projects increasingly treat data and models as versioned, testable products. Data contracts define expected schemas and quality conditions between producing and consuming systems. Feature stores and reusable pipelines can improve consistency, although they introduce governance and operational complexity. MLOps platforms automate testing, deployment, monitoring, and rollback.

Generative artificial intelligence has expanded the range of possible analytical assistants, including tools for code generation, documentation, natural-language querying, and unstructured-data extraction. These uses create additional requirements for validation, confidentiality, provenance, prompt and output monitoring, and human oversight. A language model output is not automatically a verified analytical result.

Responsible AI frameworks increasingly emphasize impact assessment, transparency, risk classification, human control, and continuous monitoring. The practical implication is that problem formulation must include not only “Can this be predicted?” but also “Should this be predicted, for whom, under what conditions, and with what safeguards?”

Best Practices for Data Science Project Initiation

A sound initiation process should:

• begin with a decision and stakeholder, not an algorithm;
• define the population, unit of analysis, outcome, timing, and intervention;
• document assumptions, constraints, risks, and exclusions;
• establish a baseline and agree on success and guardrail metrics;
• test data availability and quality before promising delivery;
• preserve temporal order and prevent leakage;
• involve domain, technical, legal, privacy, and operational expertise;
• plan for adoption, monitoring, retraining, and retirement;
• use iterative prototypes and obtain evidence before scaling;
• record decisions so that the project remains reproducible and auditable.

Chapter Summary

The data science life cycle connects organizational objectives with analytical work and operational decisions. CRISP-DM organizes this connection through business understanding, data understanding, data preparation, modelling, evaluation, and deployment, while allowing iteration between phases. Problem formulation requires precise definitions of the business problem, decision, analytical question, target, hypotheses, requirements, constraints, and success metrics. Good projects distinguish technical performance from business impact, use baselines and guardrail measures, assess data and organizational feasibility, and plan for adoption and monitoring. Ethical governance, privacy, fairness, documentation, and human accountability must be considered throughout the life cycle. The most important professional discipline is to ensure that a model solves a meaningful, feasible, and responsibly governed problem.

Key Terms

Business problem: An organizational difficulty or opportunity requiring a decision or intervention.

Analytical question: A precisely stated question that data analysis is intended to answer.

CRISP-DM: An iterative framework comprising business understanding, data understanding, preparation, modelling, evaluation, and deployment.

Target variable: The outcome that a predictive or explanatory analysis seeks to estimate.

Decision rule: A specified condition linking analytical output to an organizational action.

Hypothesis: A testable proposition about a relationship, mechanism, or expected outcome.

Requirement: A condition or capability that a proposed solution must satisfy.

Constraint: A limitation affecting available data, methods, resources, actions, or outcomes.

Baseline: A simple reference method used to judge whether a more advanced approach adds value.

Data leakage: Use of information in analysis that would not be available at the intended prediction time.

Analytical unit: The entity represented by one observation, such as a customer, order, or appointment.

Success metric: A measure used to determine whether project objectives have been achieved.

Guardrail metric: A measure used to ensure that improvement in one objective does not cause unacceptable harm elsewhere.

Data drift: Change in the distribution or quality of input data over time.

Concept drift: Change in the relationship between inputs and the outcome being predicted.

Deployment: The process of making analytical output available within an operational workflow.

MLOps: Practices and technologies for reliable development, deployment, monitoring, and governance of machine-learning systems.

Model calibration: The degree to which predicted probabilities correspond to observed event frequencies.

Data provenance: The documented origin, transformation history, and ownership of data.

Adoption: The extent to which intended users receive, understand, trust, and act on analytical output.

Review Questions

Short Answer Questions

1. Why is a business problem not automatically an analytical question?

2. Name the six phases of CRISP-DM.

3. What is data leakage, and why does it threaten evaluation validity?

4. Distinguish a technical metric from a business metric.

5. What is the purpose of a baseline model or rule?

Long Answer Questions

1. Explain how a vague organizational concern can be transformed into a decision, analytical question, target, and success metric.

2. Discuss the activities and outputs of the business-understanding and data-understanding phases of CRISP-DM.

3. Compare CRISP-DM, Agile, TDSP, and MLOps in terms of the problems each helps an organization manage.

4. Explain why deployment should be treated as an ongoing learning and monitoring stage rather than the end of a project.

5. Discuss the role of requirements, assumptions, constraints, and acceptance criteria in planning a data science project.

Analytical / Application Questions

1. A bank wants to “use AI to improve loan quality.” Formulate a more precise business problem, analytical question, target, decision rule, two success metrics, and two ethical constraints.

2. A retailer evaluates a demand model by randomly splitting five years of transactions into training and test sets. Analyze the risks of this design and propose a better evaluation approach.

3. A hospital model accurately predicts readmission but staff have no capacity to intervene. Explain why the project may have low practical value and redesign the problem formulation.

22. Multiple Choice Questions with Answers

Question 1

Which statement best represents a properly formulated analytical question?

A. Build a machine-learning model for customer churn.

B. Improve customer satisfaction using data.

C. Which active customers are likely to cancel within 30 days using information available before an intervention?

D. Find interesting patterns in customer records.

Correct Answer: C. Which active customers are likely to cancel within 30 days using information available before an intervention?

Question 2

In CRISP-DM, which phase primarily establishes the organizational objective, decision owner, scope, and constraints?

A. Business understanding

B. Data preparation

C. Modelling

D. Deployment

Correct Answer: A. Business understanding

Question 3

A model uses a field recorded only after a loan default to predict default risk. This is an example of:

A. Calibration

B. Data leakage

C. Concept drift

D. Underfitting

Correct Answer: B. Data leakage

Question 4

Why is a baseline important in a data science project?

A. It guarantees that the final model will be unbiased.

B. It provides a reference for judging whether a complex method adds value.

C. It eliminates the need for business metrics.

D. It automatically selects the best decision threshold.

Correct Answer: B. It provides a reference for judging whether a complex method adds value.

Question 5

Which item is a guardrail metric for a hospital scheduling system?

A. Number of model features

B. Training-set accuracy

C. Reduction in waiting time with no unacceptable deterioration for a patient group

D. Number of software commits

Correct Answer: C. Reduction in waiting time with no unacceptable deterioration for a patient group

Question 6

A fraud model has high accuracy because fraudulent transactions are rare. Which additional evaluation is most appropriate?

A. Examine precision, recall, and the costs of false positives and false negatives.

B. Ignore the class distribution and report accuracy only.

C. Use the number of model parameters as the main metric.

D. Evaluate only on the majority class.

Correct Answer: A. Examine precision, recall, and the costs of false positives and false negatives.

Question 7

Which situation most clearly illustrates concept drift?

A. A column changes from text format to numeric format.

B. The distribution of customer ages changes slightly.

C. The relationship between transaction patterns and fraud changes after a new fraud strategy appears.

D. A duplicate record is removed during preparation.

Correct Answer: C. The relationship between transaction patterns and fraud changes after a new fraud strategy appears.

Question 8

A team discovers that its historical no-show labels have different definitions across clinics. What should it do first?

A. Increase model complexity.

B. Create and document a consistent outcome definition before comparing models.

C. Delete all records from the smallest clinic.

D. Deploy the model and correct labels later.

Correct Answer: B. Create and document a consistent outcome definition before comparing models.

Question 9

Which statement best distinguishes prediction from causal evaluation?

A. A prediction model estimates risk, whereas causal evaluation examines whether an intervention changes outcomes.

B. Prediction always proves that an intervention works.

C. Causal evaluation does not require outcome data.

D. Prediction and causal evaluation are identical when accuracy is high.

Correct Answer: A. A prediction model estimates risk, whereas causal evaluation examines whether an intervention changes outcomes.

Question 10

Why should deployment include monitoring and a retirement plan?

A. Models never change after deployment.

B. Input data, outcome relationships, system conditions, and organizational value may change over time.

C. Monitoring replaces the need for initial validation.

D. Retirement is required only for models with low accuracy during training.

Correct Answer: B. Input data, outcome relationships, system conditions, and organizational value may change over time.