Data Science Lifecycle and Problem Framing

Introduction

Data science projects rarely fail because a model cannot be trained. They more often fail because the organization has not defined the right problem, the available data cannot support the intended decision, success has not been measured, or the final analysis is not adopted by users. A data science lifecycle provides a disciplined way to move from an organizational need to a useful, responsible and feasible analytical product.

This chapter introduces the lifecycle of a data science project, with particular emphasis on CRISP-DM (Cross-Industry Standard Process for Data Mining). It explains how to translate broad business concerns into analytical questions, hypotheses, measurable success criteria, deliverables and project plans. Although CRISP-DM is widely used, it should not be treated as a rigid sequence. Data science is iterative: findings from data understanding may change the business question, and model evaluation may reveal that the original success criteria were inappropriate.

Problem framing is therefore both a technical and organizational activity. It requires communication with decision-makers, subject experts, data owners, users and governance specialists. A well-framed project specifies what decision will be improved, who will use the result, what evidence is required, and what constraints must be respected.

Learning Objectives

By the end of this chapter, you should be able to:

1. Explain the purpose and phases of a data science lifecycle.
2. Describe CRISP-DM and compare it with other lifecycle perspectives.
3. Convert an organizational need into an analytical question and testable hypotheses.
4. Define business, analytical and operational success criteria.
5. Identify appropriate deliverables, stakeholders, risks and constraints.
6. Prepare a feasible data science project plan.
7. Recognize ethical, legal, privacy and deployment considerations during problem framing.

Why a Lifecycle Is Necessary

A data science project transforms resources into a decision-support capability. Its resources may include data, software, infrastructure, domain expertise, analyst time and stakeholder attention. Its output may be a report, dashboard, prediction service, recommendation, classification system or policy evaluation.

A lifecycle helps a team answer five fundamental questions:

• Purpose: What organizational decision or process requires improvement?
• Evidence: What data and analysis could inform that decision?
• Feasibility: Can the problem be solved with available data, skills, time and technology?
• Value: How will success and benefit be measured?
• Use: How will the result be communicated, deployed, monitored and maintained?

A technically accurate model can still be unsuccessful. For example, a model predicting customer churn may achieve high accuracy but provide no useful intervention if the organization cannot contact customers, offer an appropriate retention service or act before the customer leaves. Project value depends on the connection between analysis and action.

The CRISP-DM Framework

CRISP-DM contains six broad phases:

1. Business understanding
2. Data understanding
3. Data preparation
4. Modelling
5. Evaluation
6. Deployment

The phases are iterative rather than strictly linear. A team may return from modelling to data preparation, from evaluation to business understanding, or from deployment to data understanding when monitoring reveals data quality problems.

Business Understanding

Business understanding establishes the organizational context and project purpose. The team identifies the problem, relevant stakeholders, available resources, constraints, risks and expected benefits.

A useful business understanding statement includes:

• the current situation;
• the decision or process that needs improvement;
• the groups affected;
• the proposed analytical contribution;
• constraints such as budget, time, privacy or regulation;
• the expected organizational benefit.

A vague statement such as “use data science to improve sales” is not an adequate project definition. A stronger statement might be: “The online retailer wants to prioritize customer-retention actions for customers at high risk of cancelling within the next 30 days, subject to contact, privacy and budget constraints.”

Data Understanding

Data understanding examines what data exists, how it was generated, whether it represents the target population and whether it is suitable for the question. Activities include data collection, description, exploration and quality assessment.

Important questions include:

• Who or what does each record represent?
• What period and population are covered?
• Which variables are measured, and with what definitions?
• Are missing values systematic?
• Are there duplicates, outliers or inconsistent units?
• Was the data collected before or after the outcome of interest?
• Could a variable contain information that would not be available when making the real decision?

This last issue is called data leakage. For example, using a customer’s cancellation date to predict whether the customer will cancel is inappropriate because the information becomes available only after the event is effectively known.

Data Preparation

Data preparation converts raw data into an analysis-ready form. It may include data integration, cleaning, handling missing values, correcting types and units, constructing variables, encoding categories and splitting data into training, validation and test sets.

Preparation should be documented and reproducible. A transformation that is applied to training data must be applied consistently to future data. Decisions about removing records or imputing values should be justified because they can change the population represented by the analysis.

Modelling

Modelling selects and trains analytical methods appropriate to the question and data. Possible methods include descriptive statistics, regression, classification, clustering, time-series analysis and recommendation methods.

Model choice should reflect the decision context. A slightly less accurate model may be preferable if it is easier to explain, cheaper to operate or more stable across groups. Modelling should include a baseline, such as a simple rule or existing process, so that the team can determine whether the proposed approach adds value.

Evaluation

Evaluation asks whether the results satisfy both analytical and business requirements. It is not enough for a model to perform well on a technical metric. The team must ask whether the model addresses the original problem, whether the test data fairly represent future use, and whether risks are acceptable.

Evaluation may include:

• predictive performance;
• error analysis;
• fairness across relevant groups;
• robustness under changing conditions;
• usability and interpretability;
• cost-benefit analysis;
• stakeholder review;
• compliance and privacy assessment.

Deployment

Deployment means making the result available for use. It may involve a report, dashboard, batch scoring process, application programming interface, workflow integration or decision-support tool. Deployment also includes documentation, training, access control, monitoring and maintenance.

A deployed model should be monitored for data drift, changes in user behaviour, changes in the target outcome and deterioration in performance. Deployment is not the end of the lifecycle; it begins an operational feedback loop.

From Organizational Need to Analytical Question

Problem framing is the process of converting a broad organizational concern into a precise analytical task. This conversion should preserve the connection between the analysis and the decision.

A useful chain is:

Organizational objective → decision → analytical question → data → method → action → measurable outcome

For example:

• Objective: reduce delivery delays.
• Decision: identify shipments requiring intervention.
• Analytical question: which shipments are likely to arrive more than one day late?
• Data: route, carrier, origin, destination, weather, dispatch time and historical delivery outcomes.
• Method: probability model or classification model.
• Action: contact the customer or reroute the shipment.
• Outcome: reduction in late deliveries and avoidable intervention costs.

The analytical question should define the unit of analysis, population, time horizon and outcome. “Which shipments” defines the unit; “likely to arrive” defines the prediction; “more than one day late” defines the outcome; and the planned intervention determines when the prediction must be made.

Descriptive, Diagnostic, Predictive and Prescriptive Questions

Data science questions can be classified as follows:

These categories are related but not interchangeable. A predictive model may identify risk without proving causation. If the organization wants to know whether a discount causes retention, an experimental or causal design may be required rather than a standard prediction model.

Hypotheses and Analytical Assumptions

A hypothesis is a proposition that can be examined with evidence. In applied data science, hypotheses guide data selection, analysis and interpretation.

A good hypothesis identifies a population, variables, expected relationship and time context. For example: “Among customers active during the last six months, customers with a decline in purchase frequency are more likely to cancel during the following 30 days.”

Hypotheses may be:

• Associational: two variables are related.
• Predictive: information available now can improve estimates of a future outcome.
• Causal: changing one factor would change an outcome, under stated assumptions.
• Operational: a proposed intervention can improve a measurable process outcome.

The distinction between association and causation is essential. If customers who use support services cancel more often, support use may be a warning sign rather than the cause of cancellation. Confounding variables, selection effects and reverse causation may explain the observed relationship.

Hypotheses should be accompanied by assumptions. Examples include:

• the outcome is measured consistently;
• the training data represent the future population;
• important predictors are available at decision time;
• records can be linked without unacceptable error;
• the proposed intervention is feasible;
• the measurement period is long enough to observe outcomes.

Success Criteria and Metrics

A project should define success before extensive modelling begins. Three layers are useful.

Business Success Criteria

These describe organizational value, such as reducing average handling time, increasing timely deliveries, lowering fraud losses or improving student retention. They should be measurable and tied to a decision.

Analytical Success Criteria

These describe the quality of the analytical output. For a binary classifier, common measures include precision, recall and F1 score.

Let:

• TP = true positives;
• FP = false positives;
• FN = false negatives;
• TN = true negatives.

Precision is:

Recall is:

The F1 score is the harmonic mean of precision and recall:

These formulas apply to binary classification and require the relevant denominators to be non-zero. Precision answers, “Among predicted positives, how many were positive?” Recall answers, “Among actual positives, how many were detected?” They do not by themselves establish business value.

For imbalanced problems, accuracy can be misleading. If only 1% of transactions are fraudulent, a model that labels every transaction as legitimate has 99% accuracy but detects no fraud. The suitable metric depends on the costs of different errors.

Operational Success Criteria

These describe whether the result can operate in practice. Examples include response time, system availability, retraining frequency, explainability requirements, maximum review capacity and acceptable data latency.

A practical success statement might be: “The pilot will be considered successful if the model identifies at least 70% of eligible high-risk cases at a review volume below 500 cases per week, while meeting privacy and fairness requirements.” Such a statement is more useful than “build an accurate model.”

Cost-Sensitive Decisions

When false positives and false negatives have different consequences, threshold selection should reflect cost. If a false positive costs C_FP and a false negative costs C_FN, an illustrative expected error cost is:

This expression omits other costs and benefits and should not be treated as a complete business case. It is useful only when the costs are reasonably estimated, the error categories are defined consistently and the counts refer to the same evaluation population.

Deliverables and Project Planning

A project plan converts the lifecycle into activities, responsibilities, dependencies and decision points. Typical deliverables include:

A feasible plan should identify:

• tasks and estimated effort;
• required skills and owners;
• data access dependencies;
• review gates;
• technical infrastructure;
• privacy and security checks;
• risks and mitigation actions;
• criteria for stopping or redefining the project.

Scope should be controlled. A minimum viable analytical product may be preferable to an ambitious system that cannot be delivered or maintained. A staged plan might begin with a diagnostic report, proceed to a retrospective model evaluation, and then conduct a limited pilot before full deployment.

Stakeholders and Communication

Data science projects involve several stakeholder groups:

• Sponsors, who authorize resources and expect value;
• subject-matter experts, who explain processes and terminology;
• data owners, who control access and data definitions;
• analysts and engineers, who create the solution;
• front-line users, who act on the output;
• governance, legal and security teams, who assess risk;
• affected individuals or communities, whose interests may not be represented by the sponsor.

Different stakeholders require different forms of communication. Executives may need a concise value and risk summary; analysts need definitions and assumptions; users need workflow guidance; auditors need traceable evidence. A shared glossary prevents disagreement about terms such as “customer,” “active account,” “case resolved” or “late delivery.”

Illustrative Case: Predicting Missed Hospital Appointments

This fictional case illustrates a lifecycle approach.

Background and Problem

A community clinic experiences a high number of missed appointments. Staff want to use historical information to identify appointments that may require reminders. The objective is not simply to predict absence; it is to allocate limited reminder capacity fairly and effectively.

Business Understanding

The decision is whether and how to send an additional reminder before an appointment. Stakeholders include clinic managers, scheduling staff, clinicians, patients, data-protection officers and the information-technology team. Constraints include limited staff time, sensitive health information, communication preferences and the risk of unfairly targeting particular groups.

Analytical Question and Hypotheses

The analytical question is: “Using information available at the time an appointment is scheduled, which appointments have an elevated probability of being missed?”

A predictive hypothesis is that appointment history, lead time, prior cancellation behaviour and travel-related variables may improve risk estimates. This hypothesis does not claim that these factors cause absence.

Data Understanding

Potential variables include appointment date, booking date, previous attendance history, appointment type, preferred communication channel and distance from the clinic. Sensitive variables should be assessed carefully and used only when justified. The team must ensure that variables recorded after the appointment, such as a same-day cancellation code, are excluded from prediction.

The outcome must be defined precisely. For example, a missed appointment may mean that the patient neither attended nor cancelled within the clinic’s defined policy period. The team should also examine whether outcome recording differs across departments.

Data Preparation and Modelling

The data are divided according to time rather than randomly alone, because the model will be used to predict future appointments. A simple baseline could predict that every appointment has the overall historical absence rate, while a logistic regression or tree-based classifier could provide a more individualized estimate. The baseline is necessary to show whether modelling adds useful information.

Evaluation and Action

Suppose the clinic can make only 400 additional reminders per week. Evaluation should therefore examine performance among the highest-ranked 400 appointments, not only a general accuracy score. Precision at that capacity, recall, communication cost and attendance improvement are relevant.

A pilot should compare ordinary practice with the proposed reminder process. If the model identifies high-risk appointments but reminders do not reduce absence, the model may be predictive but operationally ineffective. The clinic may need a different intervention, such as transport support or easier cancellation.

Governance and Outcome

The pilot should use minimum necessary data, restrict access, document retention periods and provide an explanation of the reminder policy. Protected or vulnerable groups should be checked for unequal error rates and unequal access to helpful interventions. The outcome should be assessed not only by reduced missed appointments but also by patient experience, fairness, staff workload and unintended effects.

Learning Points

The case demonstrates that a good project defines the decision before choosing a model, excludes information unavailable at decision time, uses operational capacity in evaluation, distinguishes prediction from causation and tests the intervention rather than assuming that prediction creates value.

Ethical, Legal and Governance Considerations

Problem framing determines many ethical outcomes. Important principles include:

• Purpose limitation: collect and use data for a clear, legitimate purpose.
• Data minimization: use no more personal data than necessary.
• Transparency: communicate how outputs are produced and used where appropriate.
• Fairness: examine whether data, models or decisions create unjustified disadvantage.
• Accountability: assign responsibility for decisions and model performance.
• Security: protect data and analytical systems from unauthorized access.
• Human oversight: ensure that high-impact decisions are not delegated blindly to automated systems.

Fairness cannot be reduced to a single universal formula. Different fairness measures may conflict, particularly when groups have different base rates. Teams should identify the relevant harms, consult affected stakeholders and document the selected evaluation approach.

Privacy risk may arise even when direct identifiers are removed. Combinations of seemingly harmless attributes can enable re-identification. Access controls, retention limits, secure environments and appropriate consent or legal bases are therefore part of project design, not post-project administration.

Contemporary Practice: Reproducibility, MLOps and Generative AI

Modern data science increasingly uses version control, automated testing, experiment tracking, data-lineage tools and continuous monitoring. These practices support reproducibility and help teams identify when a result depends on a particular dataset or software environment.

Machine-learning operations, or MLOps, extends software-engineering discipline to model development and operation. It may include automated validation, model registries, deployment pipelines, drift monitoring and rollback procedures.

Generative AI can assist with code drafts, documentation, data dictionaries and brainstorming, but its outputs require human verification. It may produce incorrect code, fabricated references, insecure solutions or disclose confidential information if used carelessly. The lifecycle should therefore include review of generated content, protection of sensitive data and clear accountability for the final product.

Common Failure Modes and Best Practices

Common failure modes include:

1. Starting with a fashionable algorithm rather than a decision.
2. Defining success as accuracy without considering error costs.
3. Using leaked or post-outcome variables.
4. Ignoring the data-generating process and missingness mechanisms.
5. Treating correlation as proof of causation.
6. Building a model without identifying its user or workflow.
7. Failing to establish a baseline.
8. Deploying without monitoring, documentation or ownership.
9. Assuming that historical data are neutral and representative.
10. Expanding scope without revising time, resources and risk controls.

Best practice is to write a one-page project charter before modelling. It should state the problem, decision, population, outcome, time horizon, stakeholders, constraints, success criteria, data sources, risks, deliverables and next decision gate. The charter should remain editable as evidence changes, but revisions should be recorded so that the project’s reasoning remains traceable.

Chapter Summary

A data science lifecycle provides structure for transforming an organizational need into a useful analytical capability. CRISP-DM organizes work into business understanding, data understanding, data preparation, modelling, evaluation and deployment, while recognizing that real projects move iteratively between phases.

Effective problem framing begins with the decision and intended action, not with an algorithm. A strong analytical question identifies the unit of analysis, population, outcome and time horizon. Hypotheses guide investigation but must distinguish association, prediction and causation. Success criteria should include business value, analytical performance and operational feasibility. Metrics such as precision, recall and F1 score are useful only when interpreted in context and linked to error costs and capacity.

A professional project plan specifies deliverables, stakeholders, dependencies, risks and decision gates. Ethical and governance concerns—privacy, fairness, transparency, security and accountability—must be addressed from the beginning. Finally, deployment creates an ongoing responsibility for monitoring, maintenance and evaluation of real-world effects.

Key Terms

CRISP-DM: An iterative framework with six phases for planning and conducting data mining and data science projects.

Business understanding: Definition of the organizational context, objectives, constraints and expected value.

Data understanding: Examination of data sources, structure, quality, coverage and suitability.

Data preparation: Processes that transform raw data into a reproducible analysis-ready dataset.

Modelling: Application of analytical or machine-learning methods to represent patterns or make estimates.

Evaluation: Assessment of analytical results against technical, business, ethical and operational criteria.

Deployment: Making an analytical result available for practical use and establishing its support processes.

Problem framing: Translation of an organizational concern into a precise, feasible analytical project.

Analytical question: A question that specifies what evidence or estimate the analysis must produce.

Hypothesis: A testable proposition about a relationship, prediction or intervention.

Data leakage: Use of information unavailable at the time the real prediction or decision would be made.

Baseline: A simple reference method used to judge whether a proposed approach adds value.

Precision: The proportion of predicted positive cases that are actually positive.

Recall: The proportion of actual positive cases that are correctly detected.

F1 score: The harmonic mean of precision and recall.

Success criterion: A measurable condition used to judge whether a project has achieved its intended result.

Data drift: Change over time in the distribution or characteristics of input data.

Stakeholder: A person or group that affects, uses, governs or is affected by a project.

MLOps: Practices for developing, deploying, monitoring and maintaining machine-learning systems.

Data governance: Policies and responsibilities for data quality, access, security, privacy and accountable use.

Review Questions

Short Answer Questions

1. What are the six phases of CRISP-DM?
2. Why is a baseline useful in a modelling project?
3. Distinguish a predictive question from a prescriptive question.
4. What is data leakage, and why is it harmful?
5. Name three types of success criteria that should be defined for a data science project.

Long Answer Questions

1. Explain why data science lifecycles are iterative rather than strictly linear. Use examples from at least two CRISP-DM phases.
2. Discuss how an organizational objective can be translated into a decision, analytical question, dataset and measurable outcome.
3. Compare precision and recall, including situations in which each may be especially important.
4. Explain the difference between association, prediction and causation, and describe the risks of confusing them.
5. Describe the main contents of a feasible data science project plan and explain why stakeholder involvement is necessary.

Analytical / Application Questions

1. A bank wants to “use AI to improve loan decisions.” Reframe this statement into a specific decision, analytical question, possible hypothesis, success criteria and two important constraints.
2. A clinic can contact only 300 patients per week. Explain how this operational limit should influence model evaluation, threshold selection and pilot design.
3. A retailer reports that customers who receive more support calls are more likely to leave. Identify at least two possible explanations other than support calls causing churn, and propose an analysis that would investigate the issue responsibly.

Multiple Choice Questions with Answers

Question 1

Which CRISP-DM phase primarily defines the organizational objective, stakeholders and constraints?

A. Business understanding

B. Data preparation

C. Modelling

D. Deployment

Correct Answer: A. Business understanding

Question 2

Which question is predictive?

A. What was the cancellation rate last quarter?

B. Why did cancellations increase last quarter?

C. Which customers are likely to cancel next month?

D. Which retention action should be offered to each customer?

Correct Answer: C. Which customers are likely to cancel next month?

Question 3

A model predicts 40 positive cases, of which 30 are actually positive. What is its precision?

A. 0.25

B. 0.30

C. 0.75

D. 0.90

Correct Answer: C. 0.75

Question 4

Which example best illustrates data leakage?

A. Using a customer’s age at the time of prediction

B. Using a shipment’s final delivery status to predict whether it will be late

C. Using historical route information

D. Using the month in which an order was placed

Correct Answer: B. Using a shipment’s final delivery status to predict whether it will be late

Question 5

Why can accuracy be misleading for highly imbalanced classification data?

A. Accuracy cannot be calculated for classification

B. A model can achieve high accuracy by predicting the majority class while missing rare cases

C. Accuracy always penalizes false positives more than false negatives

D. Accuracy measures only deployment speed

Correct Answer: B. A model can achieve high accuracy by predicting the majority class while missing rare cases

Question 6

Which item is an operational success criterion?

A. The model uses a tree-based algorithm

B. The dataset contains 20 variables

C. The system returns a risk score within two seconds

D. The analyst completes exploratory analysis

Correct Answer: C. The system returns a risk score within two seconds

Question 7

What is the main purpose of a baseline?

A. To replace all stakeholder consultation

B. To provide a simple reference for judging added value

C. To guarantee fairness

D. To eliminate the need for testing data

Correct Answer: B. To provide a simple reference for judging added value

Question 8

Which statement correctly distinguishes prediction from causation?

A. A predictive relationship always proves that one variable causes another

B. Causal analysis never requires data

C. A model may predict an outcome accurately without identifying what would cause the outcome

D. Prediction and causation are identical terms

Correct Answer: C. A model may predict an outcome accurately without identifying what would cause the outcome

Question 9

Which activity belongs most directly to data preparation?

A. Defining the business sponsor

B. Encoding categories and handling missing values

C. Approving the final budget

D. Training end users after deployment

Correct Answer: B. Encoding categories and handling missing values

Question 10

Why should a deployed model be monitored?

A. Because data and real-world relationships may change over time

B. Because monitoring automatically proves causation

C. Because deployment removes the need for documentation

D. Because a model’s predictions never change after training

Correct Answer: A. Because data and real-world relationships may change over time