Deployment, MLOps and Professional Practice

Introduction and context

Building a predictive model is only one stage of a data-science project. A model creates organisational value when it is reliably connected to a decision, application, operational process, or human user. Deployment is the act of making a model available for use; MLOps is the set of engineering, governance, and operational practices that keep data and machine-learning systems reliable throughout their life cycle.

A notebook experiment may use a static dataset, manually selected features, and an analyst’s local environment. A production system must instead handle changing inputs, concurrent users, failures, security threats, software dependencies, monitoring, audit requirements, and model updates. Consequently, production machine learning is not simply “putting a model online”. It is a socio-technical system involving people, processes, data, software, infrastructure, and organisational controls.

This chapter develops a practical framework for packaging and deploying analytical models through APIs or applications. It also examines testing, containers, orchestration, model registries, monitoring, data and concept drift, retraining, documentation, and collaborative professional workflows.

Learning objectives

By the end of this chapter, a student should be able to:

1. distinguish experimentation, deployment, and operationalisation;
2. design a basic batch, real-time, or application-based serving architecture;
3. package a model with its preprocessing logic and dependencies;
4. explain the roles of APIs, containers, orchestration, and model registries;
5. design tests and monitoring for data, software, model, and service quality;
6. identify data drift, concept drift, and performance degradation;
7. propose a controlled retraining and release process; and
8. evaluate professional, ethical, security, documentation, and governance requirements.

1. From model development to production service

A typical life cycle has the following stages:

• Problem definition: specify the decision, users, constraints, and success criteria.
• Data preparation: collect, validate, transform, and document data.
• Model development: train, compare, validate, and select candidate models.
• Packaging: save the model, preprocessing pipeline, configuration, and dependency specification.
• Deployment: make the model available through a batch job, API, application, or embedded system.
• Operation: observe reliability, latency, cost, data quality, fairness, and predictive performance.
• Improvement or retirement: retrain, roll back, replace, or decommission the model.

The distinction between an offline metric and a production objective is important. For example, a classifier may achieve high accuracy in a balanced test set but perform poorly in production because the real class prevalence is different, inputs are incomplete, or the cost of false negatives is much greater than the cost of false positives.

A useful production specification includes:

Deployment patterns

Batch inference produces predictions periodically for a stored population, such as daily credit-risk scores. It is efficient for large volumes and does not require low-latency serving, but predictions may be stale.

Online or real-time inference receives a request and returns a response, commonly through HTTP or gRPC. It supports immediate decisions but requires attention to latency, availability, concurrency, and security.

Streaming inference processes events continuously, often using a message broker. It is appropriate for fraud alerts or equipment telemetry, but event ordering, duplicate messages, and delayed data must be managed.

Embedded or edge inference places the model near the data source, such as in a mobile device or factory controller. It can reduce network dependence and improve privacy, but hardware and update constraints are significant.

2. Packaging a reproducible model

A deployable artifact should contain more than a serialized estimator. It should specify the exact preprocessing sequence, feature names, model parameters, runtime dependencies, configuration, and version information. If training scales a variable using a mean and standard deviation, the production service must use the training values rather than recomputing them from each request.

A robust package commonly includes:

• model and preprocessing pipeline;
• schema and validation rules;
• dependency file and runtime version;
• configuration separated from source code;
• prediction entry point;
• tests;
• licence and security metadata;
• model card and release notes; and
• a unique commit, data, and model identifier.

A pipeline can be represented conceptually as:

Here, x is a raw input record, T is the fitted preprocessing transformation, f_θ is the model with parameters θ, and ŷ is the prediction. The transformation T must be the same, or demonstrably compatible, during training and inference. This prevents training-serving skew.

A model artifact should not be loaded from an untrusted source without inspection. Serialisation formats may permit arbitrary code execution, and model files can be tampered with. Secure storage, access controls, integrity checks, and approved loading mechanisms are therefore engineering requirements rather than optional enhancements.

3. APIs and application integration

An application programming interface (API) defines how another system communicates with a model service. A prediction API normally specifies an endpoint, request schema, response schema, authentication method, error behaviour, and versioning policy.

For example, a request might contain customer attributes and a response might include a risk score, decision threshold, model version, and timestamp. A good API rejects malformed requests before inference. It distinguishes client errors, such as invalid fields, from server errors, such as unavailable dependencies. It should avoid exposing sensitive internal information through error messages.

An API contract should define:

1. required and optional fields;
2. data types, units, allowed ranges, and categorical values;
3. missing and unknown-value behaviour;
4. maximum request size and rate limits;
5. authentication and authorisation;
6. response and error formats;
7. timeout and retry behaviour; and
8. backward compatibility and deprecation policy.

Latency is often described using percentiles rather than an average. A service-level objective might require the 95th-percentile response time to be below a specified threshold. The 95th percentile is the value below which 95% of observed response times fall; it does not mean that every request meets the threshold.

A prediction endpoint should be idempotent where practical, meaning that repeating the same request does not unintentionally create multiple business actions. Prediction and decision actions should also be separated where a repeat request could cause financial or operational side effects.

4. Containers and orchestration

A container packages application code, libraries, and configuration in an isolated, reproducible runtime environment. It is not a complete virtual machine: containers share the host operating-system kernel, whereas virtual machines normally include a guest operating system. Containers improve portability, but they do not automatically provide security or reproducibility.

A production image should be small, built from a trusted and patched base, run as a non-root user, exclude secrets, and be scanned for vulnerabilities. Images should be immutable after release and identified by a digest or unique version rather than an ambiguous “latest” tag.

A container alone does not solve availability. Orchestration platforms schedule containers, restart failed instances, perform health checks, scale replicas, manage configuration, and support controlled rollouts. A deployment may contain a service, several replicas, a load balancer, secrets, resource limits, and a monitoring agent.

Important orchestration concepts include:

• liveness check: asks whether a process should be restarted;
• readiness check: asks whether a process is ready to receive traffic;
• resource request and limit: specify expected and maximum resource use;
• horizontal scaling: increases or decreases the number of instances;
• rolling release: gradually replaces an old version with a new one;
• blue-green release: maintains two environments and switches traffic between them;
• canary release: sends a small proportion of traffic to the new version first.

Scaling must consider model loading time, memory consumption, feature-store capacity, and downstream services. More replicas may increase throughput but cannot correct a slow database or an inefficient feature transformation.

5. Model registries and release management

A model registry is a controlled inventory of model artifacts and their metadata. It commonly records the model version, training data reference, source-code commit, evaluation metrics, owner, approval status, intended use, and deployment history.

A useful status sequence is development, validation, staging, production, and retired. Movement between stages should require evidence and approval appropriate to the model’s risk. A registry is not merely a file store: it creates traceability and supports rollback.

Reproducibility requires versioning several components together:

• code;
• training and validation data, or immutable references to them;
• feature definitions;
• hyperparameters;
• environment and dependencies;
• model artifact;
• evaluation results; and
• deployment configuration.

A release should be reproducible enough that an investigator can determine what generated a historical prediction. Exact bit-for-bit reproducibility may be difficult when hardware, random seeds, libraries, or distributed execution differ, so the required level should be specified and tested.

6. Testing machine-learning systems

Testing must cover both conventional software behaviour and statistical behaviour. A model can pass a unit test while producing harmful predictions because the data distribution changed. Conversely, a model can have good aggregate metrics while an API fails under load.

Main testing layers

A model test should use a fixed evaluation protocol and avoid leakage from the future or from the target variable. Thresholds must be agreed before viewing results where possible, otherwise the test becomes an informal selection exercise.

For binary classification, precision and recall are defined as:

Here, TP is true positives, FP false positives, and FN false negatives. These measures apply when positive and negative outcomes are clearly defined. Precision answers, “Among predicted positives, how many are positive?” Recall answers, “Among actual positives, how many were detected?” They should not be interpreted without considering class prevalence and decision costs.

The F1 score is the harmonic mean of precision and recall:

It is defined when precision and recall are meaningful and nonzero. It gives equal weight to the two measures and may be inappropriate when false positives and false negatives have very different costs.

7. Monitoring production models

Monitoring is the continuous collection and interpretation of evidence about system health. It should be designed before deployment, not added after an incident.

Infrastructure monitoring measures CPU, memory, network use, disk, and instance health. Service monitoring measures request rate, error rate, timeout rate, and latency. Data monitoring examines schema, missingness, ranges, category frequencies, and feature distributions. Model monitoring examines prediction distributions, confidence, calibration, and performance when labels become available. Business monitoring checks whether the model is improving the intended decision or outcome.

The four golden signals for services are commonly described as latency, traffic, errors, and saturation. A dashboard is useful only when it is connected to thresholds, alerts, ownership, and an incident response procedure.

Monitoring must respect privacy. Logging complete personal records may make diagnosis easier but can create unnecessary exposure. Prefer minimised, pseudonymised, access-controlled logs, with defined retention and deletion rules.

Drift and degradation

Data drift is a change in the distribution of input data. Concept drift is a change in the relationship between inputs and the target. Label drift is a change in the target or class prevalence. These may occur separately or together.

Suppose a categorical feature has distributions P in training and Q in production. The population stability index is sometimes used as a heuristic:

Here, P_i and Q_i are training and production proportions in bin or category i, and ln is the natural logarithm. The bins must be defined consistently, proportions must be non-negative and sum to one, and zero proportions require a documented smoothing rule. PSI is not a universal statistical test or proof of model failure; thresholds are context-dependent and should be validated empirically.

For numerical distributions, alternatives include quantile comparisons, population tests, and distance measures. Regardless of the method, a drift alert should trigger investigation rather than automatic retraining. A change may reflect a harmless seasonal pattern, a data pipeline error, or a genuine change in the population.

8. Retraining, rollback, and incident response

Retraining can be scheduled, triggered by evidence, or initiated manually. A safe process is:

1. identify a need through monitoring, new labelled data, or business change;
2. validate the new data and investigate its provenance;
3. train the candidate using a versioned pipeline;
4. evaluate predictive, fairness, robustness, and operational metrics;
5. compare with the current champion model;
6. approve and deploy to staging;
7. conduct shadow, canary, or limited release testing;
8. monitor the release; and
9. promote, pause, or roll back according to predefined criteria.

Automatic retraining is attractive but dangerous when labels are delayed, corrupted, adversarial, or unrepresentative. A retraining trigger should therefore be separated from an automatic production promotion decision.

Rollback requires a known-good artifact, compatible schemas, reversible configuration, and a tested procedure. During an incident, teams should preserve logs and timelines, communicate impact, protect users, and conduct a blameless post-incident review focused on system improvement.

9. Documentation, ethics, and governance

Documentation makes assumptions visible. A model card should describe intended use, out-of-scope use, training data, evaluation populations, performance, limitations, subgroup results, ethical risks, and maintenance ownership. A data sheet or dataset record should describe provenance, collection method, licensing, quality, missingness, representativeness, and permitted use.

Governance questions include:

• Is there a lawful and fair basis for collecting and using the data?
• Are individuals informed when automated decision systems affect them?
• Can a person challenge or appeal a consequential decision?
• Are sensitive attributes used, excluded, or required for fairness auditing?
• Is the system secure against unauthorised access and adversarial manipulation?
• Are retention, access, deletion, and cross-border transfer rules satisfied?
• Is the model proportionate to the decision’s importance?

Fairness cannot be reduced to one universal metric. Different measures can conflict, especially when groups have different base rates. Teams should define the relevant harm, population, comparison, and decision threshold with domain experts and affected stakeholders. Human oversight must be meaningful rather than a nominal approval step.

Professional practice also includes accessibility, clear communication of uncertainty, conflict-of-interest disclosure, version control, code review, reproducible analysis, and respect for licences and intellectual property.

10. Collaborative industry workflow

A professional team commonly uses version control with short-lived branches or trunk-based development, pull requests, peer review, automated tests, continuous integration, and controlled continuous delivery. Infrastructure and deployment configuration should be treated as code where feasible.

A practical division of responsibility may involve data scientists, data engineers, software engineers, platform engineers, security specialists, product owners, legal or compliance experts, and domain representatives. Responsibility must remain explicit: “the model decided” is not an adequate governance statement.

A useful repository structure separates source code, tests, configuration, documentation, infrastructure definitions, and deployment manifests. Secrets belong in a secret-management system, not in source control. Continuous integration should test code, schemas, images, and security dependencies. Continuous delivery should promote only approved artifacts and preserve an audit trail.

Applied case: illustrative loan-support model

Background

A fictional community lender develops a model to prioritise applications for manual review. The model does not make the final lending decision; it estimates the probability that an application will require additional verification.

Problem

The prototype has strong offline performance, but the lender needs a secure API integrated with its application portal. It must support auditability, avoid discriminatory treatment, and remain reliable when application patterns change.

Analysis

The team identifies three risks. First, the notebook preprocessing differs from the portal’s field encoding. Second, applications arrive in bursts during monthly campaigns. Third, labels become available only after verification is completed, so immediate performance measurement is impossible.

Approach and solution

The team packages preprocessing and the classifier as one versioned pipeline, defines an input schema, and exposes a prediction endpoint that returns a score, model version, and trace identifier. A container image is built from a patched base and scanned before publication. The service is deployed with readiness checks, resource limits, and two replicas.

A registry records the training-data snapshot, source commit, evaluation results, subgroup analysis, and approval decision. Tests check schema validation, missing values, transformation consistency, API errors, and load behaviour. Monitoring tracks latency, error rates, missingness, category frequencies, score distributions, and eventual verification outcomes.

Because labels are delayed, the team does not use an immediate accuracy alert. Instead, it uses data-quality alerts and a weekly outcome report. A drift alert opens an investigation ticket; it does not automatically replace the model. A canary release is used for a new version, and the previous version remains available for rollback.

Outcome

The service meets its response-time target during normal and peak traffic, and a preprocessing mismatch is detected before production release. The governance review also reveals that one input variable is a proxy for a protected characteristic, so it is removed and subgroup performance is re-evaluated.

Learning points

The case demonstrates that deployment quality depends on data contracts, testing, monitoring, governance, and ownership as much as on predictive accuracy. It also shows why delayed labels and fairness concerns require decisions beyond automated metric thresholds.

Challenges and contemporary developments

Production systems face changing infrastructure, dependency vulnerabilities, cloud costs, vendor lock-in, limited labels, and increasing model complexity. Large language models introduce additional concerns, including prompt and retrieval monitoring, hallucination evaluation, content safety, data leakage, and model-provider dependency. These concerns fit the same broad MLOps principles: define contracts, evaluate appropriate risks, monitor actual use, and retain human accountability.

Reproducible environments are increasingly supported by infrastructure-as-code, software bills of materials, signed artifacts, feature stores, experiment tracking, and policy-as-code. However, tools do not replace design judgment. An organisation can possess sophisticated platforms while lacking clear ownership, valid labels, or a defensible definition of success.

Best-practice checklist

Before production release, confirm that:

• the business purpose and out-of-scope uses are documented;
• training-serving transformations are aligned;
• inputs, outputs, errors, and versioning are specified;
• code, data references, dependencies, and artifacts are versioned;
• tests cover data, model, security, and operational behaviour;
• access, secrets, logging, privacy, and retention are controlled;
• dashboards, alerts, owners, and incident procedures exist;
• fairness, explainability, and human review are proportionate to risk;
• rollback has been tested; and
• retraining requires evidence, validation, and approval.

Chapter Summary

Deployment converts a data-science artifact into a dependable operational capability. Batch, online, streaming, and edge patterns provide different trade-offs in freshness, latency, complexity, and cost. Reliable packaging combines the model with preprocessing, dependencies, schemas, configuration, tests, and provenance. APIs establish explicit contracts, while containers improve environmental consistency and orchestration supports health checking, scaling, and controlled releases.

MLOps extends software engineering with data and model concerns. Testing must cover code, integration, schemas, predictive quality, fairness, security, load, and regression. Monitoring should include infrastructure, service, data, model, and business signals. Data drift, concept drift, and delayed labels require investigation rather than reflexive retraining. Registries, version control, documentation, approval workflows, canary releases, and rollback provide traceability and operational safety. Finally, professional practice requires privacy, security, fairness, accessibility, legal compliance, stakeholder involvement, and clear human accountability.

Key Terms

Deployment: Making a model available for operational use.

MLOps: Practices for developing, deploying, monitoring, and governing machine-learning systems.

Batch inference: Generating predictions periodically for stored records.

Online inference: Generating a prediction in response to an immediate request.

API: A defined interface through which software systems communicate.

Schema: A specification of fields, types, constraints, and meanings.

Training-serving skew: A mismatch between transformations used in training and production inference.

Container: An isolated package containing software and its runtime dependencies.

Orchestration: Automated management of deployed containers or services.

Readiness check: A test indicating whether a service can receive traffic.

Model registry: A controlled catalogue of model artifacts and metadata.

Canary release: A release in which a small portion of traffic first uses the new version.

Rollback: Reverting to a previously approved version.

Data drift: Change in the distribution of input data.

Concept drift: Change in the relationship between inputs and the target.

Monitoring: Continuous observation and interpretation of system behaviour.

Model card: Documentation describing a model’s purpose, performance, limitations, and risks.

Retraining: Fitting a model again using new or revised data.

Technical debt: Future cost created by expedient but inadequately managed technical decisions.

Review Questions

Short Answer Questions

1. Distinguish batch inference from online inference.
2. Why should preprocessing be packaged with the model?
3. What is the difference between a liveness check and a readiness check?
4. Define data drift and concept drift.
5. State two purposes of a model registry.

Long Answer Questions

1. Explain how an analytical model can be packaged and exposed through a secure prediction API.
2. Discuss the role of containers and orchestration in reliable deployment.
3. Describe a comprehensive testing strategy for a machine-learning service.
4. Explain how monitoring, retraining, and rollback should be connected in an operational life cycle.
5. Discuss ethical, privacy, security, and governance responsibilities in production model deployment.

Analytical / Application Questions

1. A fraud model’s input distributions change after a new payment channel is introduced, but labels are not yet available. Design an investigation and monitoring response without immediately retraining the model.
2. A model has high overall recall but substantially lower recall for a small demographic group. Analyse possible causes and propose technical and governance actions.
3. Design a deployment plan for a demand-forecasting model that serves both nightly inventory batches and an online dashboard. Justify the architecture, testing, monitoring, and rollback strategy.

Multiple Choice Questions with Answers

Question 1

Which deployment pattern is most appropriate when predictions are generated nightly for a large stored customer population?

A. Batch inference

B. Online inference

C. Embedded inference

D. Manual inference

Correct Answer: A. Batch inference

Question 2

What is the primary purpose of an input schema in a prediction API?

A. To guarantee perfect model accuracy

B. To define valid fields, types, and constraints

C. To replace authentication

D. To eliminate the need for monitoring

Correct Answer: B. To define valid fields, types, and constraints

Question 3

Which practice best reduces training-serving skew?

A. Recomputing scaling parameters for every request

B. Using separate undocumented preprocessing scripts

C. Packaging the fitted preprocessing pipeline with the model

D. Removing all input validation

Correct Answer: C. Packaging the fitted preprocessing pipeline with the model

Question 4

What does a readiness check determine?

A. Whether a service should receive traffic

B. Whether the training data is unbiased

C. Whether a model has maximum accuracy

D. Whether a container image is licensed

Correct Answer: A. Whether a service should receive traffic

Question 5

Which item is normally recorded in a model registry?

A. Only the model’s file size

B. The model version and evaluation metadata

C. Users’ unencrypted passwords

D. An unlimited collection of raw personal data

Correct Answer: B. The model version and evaluation metadata

Question 6

Concept drift refers to:

A. A change in the relationship between inputs and the target

B. A change in the container’s memory allocation only

C. A missing API endpoint

D. A change in source-code formatting

Correct Answer: A. A change in the relationship between inputs and the target

Question 7

Why should a drift alert not automatically trigger production retraining in every case?

A. Drift is impossible to measure

B. A detected change may be seasonal, harmless, or caused by a data error

C. Retraining never changes a model

D. Monitoring is unrelated to deployment

Correct Answer: B. A detected change may be seasonal, harmless, or caused by a data error

Question 8

If TP = 80 and FN = 20, what is recall?

A. 20%

B. 50%

C. 80%

D. 100%

Correct Answer: C. 80%

Question 9

Which release strategy initially sends only a small proportion of traffic to a new model?

A. Canary release

B. Full replacement without testing

C. Static archiving

D. Schema deletion

Correct Answer: A. Canary release

Question 10

Which statement best describes professional model governance?

A. The model is solely responsible for decisions

B. Governance is needed only after a serious incident

C. Governance includes accountability, documentation, risk assessment, and human oversight

D. Governance means maximising accuracy regardless of harm

Correct Answer: C. Governance includes accountability, documentation, risk assessment, and human oversight