Data Products, Deployment, and MLOps

Introduction

A data science project creates value only when its outputs can be used reliably. A notebook that produces an accurate model is not yet a data product. A data product is a usable, maintainable service, application, report, or automated decision system whose behaviour is supported by data, software, documentation, monitoring, and governance.

Deployment is the process of making analytical or machine-learning functionality available to users or other systems. MLOps extends software engineering and DevOps practices to machine-learning systems. It addresses the fact that models depend not only on code, but also on training data, feature definitions, model parameters, infrastructure, and changing real-world conditions.

A production system must answer several questions: Can the result be reproduced? Can the system scale? How quickly does it respond? What happens when data are missing or invalid? How are new versions tested? How is model degradation detected? Who is accountable for its decisions?

Learning Objectives

By the end of this chapter, you should be able to:

• distinguish analytical notebooks, data products, APIs, and applications;
• package Python code using suitable project structures and dependency specifications;
• design a basic model-serving API and lightweight user interface;
• explain containers, cloud deployment, batch processing, and real-time inference;
• construct reproducible data and machine-learning pipelines;
• apply experiment tracking, version control, and model registries;
• design CI/CD workflows for data and ML systems;
• monitor service health, data quality, model performance, and drift;
• evaluate scalability, security, ethics, and lifecycle-management decisions.

1. From Notebook to Data Product

A notebook is primarily an exploratory environment. It supports rapid experimentation, visualisation, and communication, but it commonly contains hidden state, manually executed cells, hard-coded paths, undocumented assumptions, and untested transformations. These characteristics make direct production use risky.

A data product is an engineered product with defined users, inputs, outputs, quality expectations, and operational ownership. Examples include:

• a demand-forecasting API used by an inventory system;
• a fraud-risk score displayed to investigators;
• a customer-segmentation dashboard;
• an automated document-classification service;
• a scheduled data-quality report.

A mature data product has at least six layers:

The distinction between a model and a product is important. A model maps inputs to predictions, but a product supplies valid inputs, exposes predictions safely, records operational evidence, and supports users when the model is uncertain or unavailable.

2. Packaging Analytical Python Code

2.1 A maintainable project structure

A practical Python project separates application code from experiments and configuration. One possible structure is:

customer-risk/
├── pyproject.toml
├── README.md
├── src/
│   └── risk_service/
│       ├── __init__.py
│       ├── data.py
│       ├── features.py
│       ├── model.py
│       └── api.py
├── tests/
│   ├── test_features.py
│   └── test_api.py
├── notebooks/
├── configs/
├── Dockerfile
└── .github/workflows/ci.yml

The src layout reduces accidental imports from the project’s working directory. pyproject.toml records project metadata and dependencies. Tests verify behaviour, while notebooks remain useful for exploration rather than serving as the production execution path.

A production function should have a clear contract. For example, a feature function should specify required columns, accepted data types, treatment of missing values, output columns, and possible exceptions. Configuration should be externalised rather than embedded in code. Environment variables or a managed secret store should hold passwords, tokens, and connection strings; secrets must not be committed to source control.

2.2 Dependencies and reproducibility

A dependency specification should identify package versions sufficiently to recreate an environment. Exact pinning can improve reproducibility, but it also requires regular updates to receive security fixes. A sound practice is to maintain a tested lock file or constraints file and update it deliberately.

Reproducibility has several dimensions:

1. Code reproducibility: the same source revision is available.
2. Environment reproducibility: compatible operating-system and package versions are recorded.
3. Data reproducibility: the exact training data or immutable data snapshot is identifiable.
4. Process reproducibility: preprocessing, random seeds, parameters, and execution steps are recorded.
5. Result reproducibility: outputs and evaluation metrics can be traced to the preceding items.

A random seed makes some computations repeatable, but it does not guarantee identical results across hardware, library versions, parallel algorithms, or nondeterministic operations. Reproducibility should therefore be treated as an engineering property, not as a single parameter.

3. Serving Models through APIs

An application programming interface (API) allows software clients to communicate with a service. A typical prediction API accepts structured input and returns a prediction, probability, explanation, or error response.

A simple REST design might use:

POST /v1/predict
Request:  {"age": 42, "income": 52000, "tenure_months": 18}
Response: {"risk_probability": 0.27, "model_version": "2025.03"}

The API should validate the schema before inference. Validation includes required fields, types, ranges, categorical values, and missing-value policy. A malformed request should receive a clear client error, whereas an internal model or infrastructure failure should be represented as a server error without exposing sensitive implementation details.

Important API design considerations include:

• Versioning: /v1/predict permits future changes without silently breaking clients.
• Authentication and authorisation: identify callers and limit permitted actions.
• Idempotency: repeated requests should not create unintended duplicate side effects.
• Timeouts and rate limits: protect the service from overload.
• Observability: record request latency, status, model version, and safe identifiers.
• Uncertainty: return confidence or prediction intervals where meaningful, and define behaviour for low-confidence cases.
• Backward compatibility: preserve or formally deprecate existing contracts.

For online inference, latency is often represented by percentiles rather than only an average. The 95th percentile latency is the value below which 95% of observed requests finish. Percentiles reveal slow-tail behaviour that an average can conceal.

Batch inference is preferable when predictions can be generated periodically. It is often cheaper, easier to audit, and more tolerant of long-running computation. Online inference is appropriate when a decision must be made during a user interaction or transaction.

4. Lightweight Applications and User Interfaces

Streamlit, Gradio, and similar frameworks can turn Python functions into interactive analytical applications. They are useful for demonstrations, internal tools, and human-in-the-loop workflows. A well-designed interface should explain input definitions, display validation errors, show the model or data version, and distinguish prediction from recommendation.

A lightweight interface is not automatically production-ready. It may require authentication, concurrency controls, resource limits, accessibility testing, audit logging, and protection against malicious input. If a prediction affects employment, credit, healthcare, or public services, a visual interface must not conceal the model’s limitations or remove appropriate human review.

5. Containers and Cloud Services

A container packages application code, its runtime, and required libraries into an image. Containers improve portability and reduce “works on my machine” problems, although they do not eliminate differences in hardware, external services, data, or configuration.

A typical container workflow is:

1. write a Dockerfile;
2. install only required dependencies;
3. copy application code;
4. expose the service port;
5. define a non-root runtime user where possible;
6. build and scan the image;
7. push it to a registry;
8. deploy it to a compute service;
9. monitor and update it.

Images should be small, reproducible, and free of secrets. Multi-stage builds can separate compilation dependencies from the final runtime image. Containers should be treated as immutable: configuration is supplied at deployment time rather than by manually changing a running container.

Cloud platforms commonly provide object storage, managed databases, serverless functions, virtual machines, container platforms, orchestration systems, and managed ML services. The correct choice depends on workload, compliance requirements, latency, expected traffic, operational skills, and cost.

For a service receiving an average arrival rate λ requests per second, if each instance can sustainably process μ requests per second and a target utilisation ρ is selected below 1, a simple capacity estimate is:

Here, n is the number of instances, λ is the arrival rate, μ is the per-instance service rate, and ρ is the desired maximum utilisation. This is a planning approximation, not a complete queueing model. It assumes comparable instances, stable traffic, and a service rate measured under representative workload. Burst traffic, failures, warm-up time, and tail latency require additional capacity and load testing.

For example, if traffic averages 40 requests per second, each instance handles 15 requests per second, and desired utilisation is 0.8, then:

Therefore at least four instances are required under these assumptions. Production design may require more for redundancy and bursts.

6. Pipelines and Workflow Automation

A pipeline is an ordered, testable sequence of transformations. A machine-learning pipeline may contain:

ingest → validate → clean → transform → split → train → evaluate → register → deploy → monitor

A pipeline should make dependencies explicit. If evaluation depends on training, deployment must not occur before evaluation succeeds. Workflow systems such as Airflow, Prefect, Dagster, and cloud-native services can schedule tasks, retry transient failures, record metadata, and support dependency graphs.

A robust pipeline distinguishes:

• transient failures, such as temporary network errors, which may be retried;
• permanent failures, such as schema violations, which should stop the run;
• data-quality failures, which should trigger quarantine or human review;
• business-rule failures, where technically valid data violate operational constraints.

Data leakage is a major pipeline risk. Leakage occurs when information unavailable at prediction time enters training features or evaluation. Time-dependent data should usually be split chronologically, and transformations such as imputation or scaling should be fitted on training data and applied to validation and test data.

A pipeline’s metadata should include run identifier, code revision, input-data version, feature definition, parameters, environment, metrics, model artifact, and approval status. These records support auditability and diagnosis.

7. Experiment Tracking and Model Registries

Experiment tracking records what was tried and what happened. A run commonly includes:

• parameters, such as regularisation strength;
• metrics, such as precision, recall, latency, or calibration;
• source-code commit;
• training-data identifier;
• environment or image identifier;
• model artifact;
• plots and error analyses;
• notes and approval decisions.

A model registry manages lifecycle states such as development, validation, approved, production, retired, and archived. Registration is not the same as deployment. A registered model should be promoted only after tests and governance checks have passed.

Model selection should consider the operational objective, not accuracy alone. For binary classification, precision and recall are:

Here, TP is true positives, FP is false positives, and FN is false negatives. Precision answers “Of predicted positives, how many were correct?” Recall answers “Of actual positives, how many were found?” Their suitability depends on the cost of errors. A fraud system may tolerate investigation workload to improve recall, while an expensive intervention may require high precision.

The F1 score is the harmonic mean of precision and recall when both are important:

It is defined when precision and recall are not both zero. It should not replace class-specific analysis, threshold analysis, calibration, or business-cost evaluation.

8. CI/CD for Data and Machine Learning

Continuous integration (CI) automatically checks proposed code changes. Continuous delivery or deployment (CD) automates packaging and release to test or production environments.

A useful CI sequence is:

1. format and lint code;
2. run unit tests;
3. run data-contract and schema tests;
4. build the application or image;
5. scan dependencies and the image for vulnerabilities;
6. run integration tests;
7. publish artifacts with immutable identifiers.

ML systems require additional checks. A pipeline can test that feature columns exist, training and serving transformations agree, metric thresholds are met, protected-group performance is inspected, and the model is compatible with the serving runtime.

Deployment strategies include:

• recreate: stop the old version and start the new one;
• rolling: replace instances gradually;
• blue-green: maintain old and new environments and switch traffic;
• canary: direct a small share of traffic to the new version;
• shadow: send copied requests to a new model without using its decisions.

Canary deployment reduces release risk, but it requires representative traffic, monitoring, rollback criteria, and a way to distinguish model effects from changing traffic conditions.

9. Monitoring and Model Drift

Monitoring should cover four categories:

Data drift means that the distribution of inputs has changed. Concept drift means that the relationship between inputs and the target has changed. A model can experience either, both, or neither. Drift is a signal for investigation, not automatic proof that retraining is correct.

For a categorical variable, population stability can be summarised by the population stability index:

Here, pᵢ is the reference proportion in bin or category i, qᵢ is the current proportion, and ln is the natural logarithm. Both proportions must be positive; zero values require a documented smoothing rule. PSI depends strongly on binning and should be interpreted alongside statistical tests, effect size, business context, and model performance. It is not a universal decision threshold.

When labels arrive later, monitor performance against the actual target. Before labels arrive, monitor proxies such as feature validity, prediction distribution, abstention rate, and downstream outcomes. Monitoring must avoid collecting unnecessary personal information.

A production model should have a defined response plan: investigate, compare with a baseline, assess data and code changes, roll back if necessary, retrain under approval, or retire the model. Automatic retraining without controls can amplify data errors and feedback loops.

10. Lifecycle Management and Governance

A model lifecycle begins with a defined problem and ends with retirement. Typical stages are problem framing, data assessment, development, validation, approval, deployment, monitoring, retraining, and decommissioning.

Governance includes:

• purpose and intended-use documentation;
• data provenance and lawful basis for processing;
• access control and encryption;
• retention and deletion rules;
• model cards or system documentation;
• risk classification and human oversight;
• incident response and rollback procedures;
• records of material changes and approvals.

Privacy principles include data minimisation, purpose limitation, access restriction, secure storage, and appropriate deletion. Sensitive attributes may be needed for fairness evaluation, but their collection and use require legal and ethical justification. Removing a sensitive attribute does not guarantee fairness because correlated variables may act as proxies.

Fairness analysis should be context-specific. Possible measures include group-wise error rates, selection rates, calibration, and equal opportunity. These measures can conflict mathematically or operationally, particularly when base rates differ. Therefore, fairness cannot be reduced to selecting one metric; it requires stakeholder consultation, impact analysis, and documented trade-offs.

Explainability should match the decision context. A feature-importance chart may help developers, but a person affected by a decision may need an understandable reason, a correction process, and human review. Explanations must not be presented as proof of causality when the model is predictive rather than causal.

11. Applied Case: Illustrative Retail Demand Service

Background

A regional retailer wants daily forecasts for the next seven days for 2,000 products across 30 stores. Managers currently use spreadsheets, causing inconsistent assumptions and delayed replenishment.

Problem

The retailer needs a service that produces forecasts by 06:00 each morning, records the data and model version used, and allows managers to inspect exceptions. Forecast accuracy matters, but stock-out risk and operational reliability matter as well.

Analysis

Historical sales, promotions, holidays, prices, inventory, and store metadata are available. Sales are time-dependent, so random row-wise splitting could place future information in training data. The team instead creates chronological training, validation, and test periods. It also identifies stock-outs: observed sales may be low because inventory was unavailable, not because demand was low.

Approach and Solution

The team creates a versioned ingestion pipeline with schema checks for dates, product identifiers, non-negative prices, and duplicate records. Features are generated in one reusable module used by both training and inference. A baseline based on recent seasonal demand is compared with a gradient-boosting model.

The model is packaged in a container. A scheduled batch job generates forecasts, writes them to a controlled database, and publishes a dashboard. The dashboard displays forecast, confidence information where available, model version, and an exception flag. An API provides forecasts to the replenishment system.

CI runs unit tests, feature-contract tests, and container security scans. A registry stores candidate and approved models. Deployment is initially shadowed against the existing spreadsheet process. Monitoring checks missing data, forecast volume, latency of the API, distribution of predictions, forecast error after sales labels arrive, and performance by store and product category.

Outcome

The retailer obtains a repeatable morning process rather than an opaque spreadsheet. When a supplier feed omits a store, the pipeline stops and alerts the data owner instead of silently producing misleading forecasts. Managers retain the ability to override forecasts, and overrides are logged for later analysis.

Learning Points

The case demonstrates that model quality is only one component of a data product. Time-aware validation, data contracts, versioned artifacts, human oversight, monitoring, and incident procedures are equally important. It also illustrates why batch deployment can be preferable to real-time prediction when the business decision is made on a daily schedule.

12. Challenges, Trade-offs, and Contemporary Practice

MLOps introduces costs. Tests, deployment automation, monitoring, documentation, and governance require time and specialist skills. A small, low-risk project may need a simpler architecture than a high-volume regulated service. Overengineering can reduce delivery speed and create unnecessary operational burden.

Common failure modes include deploying notebook code without tests, training-serving skew, untracked data changes, leakage, silent schema changes, excessive logging of personal data, monitoring only infrastructure, and retraining without a rollback plan.

Contemporary practice increasingly includes feature stores, data contracts, model cards, software bills of materials, policy-as-code, privacy-enhancing techniques, and platform engineering. Large language model systems add evaluation of prompt changes, retrieval sources, hallucination, toxicity, data leakage, and token cost. These tools do not remove the need for clear objectives, validation, ownership, and human accountability.

The most effective MLOps programme is risk-based. It begins with the consequences of failure, identifies the minimum controls needed, automates repeatable checks, and continuously improves through operational evidence.

Chapter Summary

A production data product combines data, analytical logic, software, infrastructure, operations, and governance. Python code should be packaged into tested modules with explicit dependencies and configuration. APIs require validation, versioning, security, observability, and suitable choices between online and batch inference. Containers and cloud services improve portability and scalability, but capacity estimates must be validated through realistic testing.

Pipelines make data and model workflows repeatable. Experiment tracking and registries connect code, data, parameters, metrics, and artifacts. CI/CD supports controlled testing and release strategies such as canary and blue-green deployment. Monitoring must address system health, data quality, drift, model performance, fairness, and business outcomes. Finally, lifecycle management requires documentation, privacy, access control, human oversight, incident response, retraining criteria, and retirement procedures.

Key Terms

Data product: A usable, maintained product whose value depends substantially on data or analytical outputs.

Deployment: Making software or a model available in an operational environment.

MLOps: Practices for developing, deploying, monitoring, and governing machine-learning systems.

API: A defined interface through which software systems communicate.

Batch inference: Generating predictions for a collection of records on a schedule.

Online inference: Generating a prediction in response to an individual request.

Container: An isolated, portable runtime package containing application components and dependencies.

Pipeline: An ordered workflow of dependent data or ML tasks.

Data contract: An explicit agreement about the structure, meaning, quality, and ownership of data.

Experiment tracking: Recording parameters, data, code, metrics, and artifacts for an experiment.

Model registry: A managed catalogue of model versions and lifecycle states.

Continuous integration: Automated validation of proposed code changes.

Continuous delivery/deployment: Automated preparation or release of software changes.

Canary deployment: Releasing a new version to a small proportion of traffic first.

Data drift: Change in the distribution of input data.

Concept drift: Change in the relationship between inputs and the target outcome.

Observability: The ability to understand system behaviour through logs, metrics, and traces.

Training-serving skew: A mismatch between transformations used during training and those used in production.

Model card: Documentation describing a model’s purpose, performance, limitations, and risks.

Technical debt: Future cost created by expedient but poorly maintainable technical choices.

Review Questions

Short Answer Questions

1. Why is a notebook not automatically a production data product?
2. Distinguish online inference from batch inference.
3. What information should an experiment-tracking record contain?
4. Define data drift and concept drift.
5. Why should secrets not be stored in source-control repositories?

Long Answer Questions

1. Explain how a Python notebook can be transformed into a maintainable, testable data product.
2. Compare recreate, rolling, blue-green, canary, and shadow deployment strategies.
3. Describe a complete ML pipeline from ingestion through monitoring, including important failure controls.
4. Explain why model governance must address privacy, fairness, documentation, and human oversight.
5. Discuss the advantages and limitations of containers and cloud services for ML deployment.

Analytical / Application Questions

1. A service receives 60 requests per second. Each instance processes 20 requests per second, and the target utilisation is 0.75. Estimate the minimum number of instances using the capacity approximation in this chapter. Explain why a production architecture may require more.
2. A fraud model has high overall accuracy but substantially lower recall for one customer group. Design an investigation covering data quality, threshold effects, subgroup performance, fairness considerations, and possible operational responses.
3. A daily forecasting pipeline begins producing unusually low predictions after an upstream schema change. Propose a diagnostic and rollback plan that uses data contracts, versioning, monitoring, and human approval.

Multiple Choice Questions with Answers

Question 1

Which characteristic most clearly distinguishes a data product from an exploratory notebook?

A. It contains at least one chart.

B. It has defined users, interfaces, quality expectations, and operational ownership.

C. It is written in Python.

D. It uses a machine-learning algorithm.

Correct Answer: B. It has defined users, interfaces, quality expectations, and operational ownership.

Question 2

What is the main purpose of API input validation?

A. To increase the number of model parameters

B. To guarantee that every prediction is correct

C. To check that requests satisfy expected schemas and constraints before inference

D. To eliminate the need for monitoring

Correct Answer: C. To check that requests satisfy expected schemas and constraints before inference

Question 3

Which deployment strategy directs a small share of real traffic to a new version initially?

A. Canary deployment

B. Recreate deployment

C. Archival deployment

D. Static deployment

Correct Answer: A. Canary deployment

Question 4

Which item is most directly associated with training-serving skew?

A. Different preprocessing logic during training and production inference

B. A larger cloud storage budget

C. A shorter API URL

D. A higher monitor resolution

Correct Answer: A. Different preprocessing logic during training and production inference

Question 5

If λ = 40 requests per second, μ = 15 requests per second per instance, and ρ = 0.8, what is the minimum integer n satisfying n ≥ λ ÷ (ρμ)?

A. 2

B. 3

C. 4

D. 5

Correct Answer: C. 4

Question 6

Which measure is defined as TP ÷ (TP + FN)?

A. Precision

B. Recall

C. Specificity

D. Accuracy

Correct Answer: B. Recall

Question 7

What does a model registry primarily manage?

A. Physical office equipment

B. Model versions, artifacts, and lifecycle states

C. Only raw customer passwords

D. Network cables

Correct Answer: B. Model versions, artifacts, and lifecycle states

Question 8

Which observation is an example of concept drift?

A. A feature has a new file format but the same meaning

B. The relationship between a feature and the target changes over time

C. A container image is compressed

D. A model is stored in a registry

Correct Answer: B. The relationship between a feature and the target changes over time

Question 9

Why should a data-quality failure often stop or quarantine a pipeline run?

A. To ensure invalid or incomplete inputs do not silently produce unreliable outputs

B. To make all pipelines run more slowly

C. To prevent version control from working

D. To remove the need for data owners

Correct Answer: A. To ensure invalid or incomplete inputs do not silently produce unreliable outputs

Question 10

Which practice best supports reproducibility?

A. Keeping only the final prediction file

B. Changing dependencies manually on the production server

C. Recording code, data version, environment, parameters, and execution metadata

D. Deleting experiment notes after deployment

Correct Answer: C. Recording code, data version, environment, parameters, and execution metadata