Deployment, MLOps and Model Governance

Introduction

A machine-learning model becomes valuable only when it can operate reliably in the environment where decisions are made. Training a model in a notebook is therefore only one stage of the machine-learning lifecycle. A production system must receive data, validate it, generate predictions, return results within an acceptable time, record what happened, protect sensitive information, and continue performing as the real world changes.

Deployment is the process of making a trained model available for use. MLOps is the set of engineering, operational and organizational practices used to develop, release, monitor and maintain machine-learning systems. Model governance is the framework of policies, controls, responsibilities and evidence used to ensure that models are appropriate, explainable, secure, compliant and properly managed throughout their lifecycle.

These subjects are particularly important because machine-learning systems differ from ordinary software. Their behaviour depends not only on code but also on data, feature definitions, training procedures, model parameters and changing environmental conditions. A software release may remain stable when its inputs remain valid; a model may degrade even when its code has not changed because customer behaviour, economic conditions or data collection processes have changed.

Learning Objectives

By the end of this chapter, a learner should be able to:

1. Explain the machine-learning deployment lifecycle and distinguish batch, online and edge deployment.
2. Package a model and expose it through an application programming interface (API).
3. Describe the roles of containers, registries, orchestration platforms and cloud services.
4. Explain experiment tracking, data and model versioning, and reproducibility.
5. Design an automated MLOps pipeline with testing, approval and rollback controls.
6. Select suitable service-level, data-quality, drift and model-performance metrics.
7. Explain retraining strategies and the difference between monitoring, alerting and intervention.
8. Apply governance principles involving documentation, risk, fairness, privacy, security and auditability.

1. From Model Development to Production

A typical lifecycle contains the following stages:

1. Problem definition: specify the decision, users, constraints and success criteria.
2. Data preparation: acquire, validate, transform and label data.
3. Model development: train, compare and evaluate candidate models.
4. Packaging: combine model artefacts, preprocessing logic, dependencies and configuration.
5. Validation: test correctness, robustness, security, fairness and operational performance.
6. Release: approve and deploy the model to a controlled environment.
7. Operation: serve predictions and collect logs and monitoring data.
8. Maintenance: investigate incidents, retrain when justified, and retire obsolete models.

A central principle is training–serving consistency. If training data are transformed by one procedure but production data are transformed by another, the model may receive inputs with a different meaning or scale. For example, a model trained with a currency amount converted to euros must not receive unconverted local-currency amounts during serving. The safest practice is to package preprocessing and prediction as one versioned pipeline whenever possible.

A production model should be treated as a system rather than a file. Relevant artefacts include:

• source code and dependency specifications;
• training and validation data references;
• feature definitions and schemas;
• preprocessing and model parameters;
• evaluation results and decision thresholds;
• container or build specifications;
• security and privacy configurations;
• approval records and deployment history.

2. Deployment Patterns

Batch inference

In batch inference, predictions are generated periodically for a collection of records. A nightly fraud-risk score, weekly demand forecast or monthly customer segmentation process is a batch workload. It is suitable when immediate responses are unnecessary and when large-scale processing is more efficient than individual requests.

Batch systems must address late-arriving data, duplicate records, failed jobs and partial completion. Outputs should normally include a prediction timestamp, model version and input-data version so that results can be reproduced and interpreted later.

Online inference

An online service responds to an individual request, commonly through an HTTP or gRPC API. Online inference is appropriate for credit screening, recommendation, search ranking and real-time fraud detection. Important requirements include latency, availability, concurrency, authentication, rate limiting and graceful failure.

Latency is the elapsed time between receiving a request and returning a response. A system may specify a target such as a 95th-percentile latency rather than an average, because occasional slow responses can affect users substantially. Availability is often represented as:

Here, uptime and downtime must be measured over the same observation period. The expression is dimensionless and is often reported as a percentage. It does not by itself describe prediction quality.

Streaming and event-driven inference

Streaming inference processes events as they arrive, often through a message broker. It is useful when decisions must follow a continuous flow of transactions or sensor readings. The design must handle event ordering, duplicate events, replay, schema evolution and temporary service failure.

Edge inference

In edge deployment, the model runs near the data source, such as on a mobile device, industrial controller or vehicle. Benefits can include low latency, reduced network transmission and improved privacy. Constraints include limited memory, processor capacity, power and update capability. Quantization, pruning and model compression may be used, but they must be validated because they can alter accuracy.

Shadow, canary and blue–green releases

A shadow deployment sends production inputs to a new model without using its predictions for decisions. It evaluates behaviour safely. A canary deployment gives the new version a small fraction of live traffic and expands exposure if monitoring is satisfactory. A blue–green deployment maintains two environments: the active blue environment and the replacement green environment. Traffic can be switched between them, enabling rapid rollback.

A rollback should be a planned operation, not an improvised reaction. The organization should know which previous model, code, feature definitions and configuration constitute a complete working release.

3. Model Packaging and APIs

A minimal prediction service generally performs four activities:

1. authenticate and validate the request;
2. convert the request into the expected feature representation;
3. execute the model and apply any decision threshold;
4. return a structured response and record an appropriate audit event.

An API contract should define the endpoint, request schema, response schema, data types, missing-value policy, error codes, model version behaviour and authentication requirements. A prediction response might include a score, class label, model identifier and timestamp, while avoiding unnecessary sensitive information.

For a binary classifier, a model may return a probability p = P(Y = 1 | X = x), where Y is the target and X = x is the observed feature vector. A decision rule can be written as:

Here, ŷ is the predicted class and τ is the selected threshold. The threshold is not automatically 0.5. It should reflect the costs of false positives, false negatives, operational capacity and applicable policy. The model score and the business decision should therefore be conceptually separated.

Input validation should check type, range, permitted categories, missingness, maximum size and schema version. Validation protects both reliability and security. It should reject malformed requests rather than allowing a downstream library to fail unpredictably. Logs should avoid storing raw personal data unless there is a documented lawful and operational requirement.

4. Containers, Registries and Cloud Platforms

A container packages application code, libraries and runtime configuration into an isolated, portable unit. It does not usually contain a complete separate operating-system kernel; it shares the host kernel while isolating processes. Containers reduce dependency conflicts and improve deployment consistency, but they do not automatically make a system secure or reproducible.

A typical container workflow is:

1. define a build specification;
2. install pinned dependencies;
3. copy application code and model artefacts;
4. expose the service configuration;
5. build and scan the image;
6. tag the image with an immutable version or digest;
7. push it to a protected registry;
8. deploy it to a controlled runtime.

An image tag such as latest is convenient but ambiguous because its contents may change. Production releases should reference immutable digests or uniquely identified versions. Images should run with minimal privileges, contain only required packages, avoid embedded secrets and receive security updates.

Cloud platforms commonly provide managed storage, compute, registries, databases, message queues, monitoring and orchestration. They may offer managed model endpoints or serverless inference. Their advantages include elasticity and reduced infrastructure administration. Their risks include cost growth, vendor dependence, regional availability constraints, data-residency issues and misconfigured access controls.

Kubernetes and similar orchestrators manage containers across a cluster. They can restart failed instances, scale replicas and perform rolling updates. However, orchestration adds complexity. It does not replace application-level health checks, model validation, governance or cost management.

5. Experiment Tracking and Version Management

Machine-learning work is often difficult to reproduce because many variables change at once. Experiment tracking records the relationship between data, code, configuration and results. A useful experiment record includes:

• experiment and run identifiers;
• source-code commit;
• dataset or data snapshot identifier;
• feature and preprocessing version;
• algorithm and hyperparameters;
• random seeds where relevant;
• training environment and dependency versions;
• evaluation metrics and test-set definition;
• model artefact location;
• author, date and purpose.

A model registry provides a controlled catalogue of model versions. Typical lifecycle states are development, candidate, approved, production, deprecated and retired. Promotion should be based on evidence rather than merely on the newest timestamp or highest single metric.

Version control must cover more than source code. Data versioning, schema versioning, feature definitions, model weights, configuration and infrastructure definitions are all relevant. A reproducibility statement should answer: Which data, code, environment and parameters produced this model?

Randomness complicates reproducibility. A fixed random seed may make one computational run repeatable, but results can still vary because of parallel computation, hardware, library versions or nondeterministic operations. Reproducibility should therefore be treated as a property to test and document, not assumed from setting a seed.

6. Automated MLOps Pipelines

Continuous integration and continuous delivery practices can be adapted for machine learning. A pipeline may contain:

1. source and configuration validation;
2. unit tests for feature transformations;
3. data-schema and data-quality tests;
4. training or model-building jobs;
5. evaluation against fixed acceptance criteria;
6. security, dependency and licence scans;
7. model registration;
8. human or policy-based approval;
9. deployment to staging;
10. smoke, load and integration tests;
11. controlled production release;
12. post-release monitoring.

The pipeline should distinguish code promotion from model promotion. A code change may produce a new model, but a model may also change because the training data changed. Both changes require traceability.

Automated tests can include:

• schema tests: columns, data types and allowed values;
• range tests: physical or business limits;
• missingness tests: unexpected null proportions;
• leakage tests: features unavailable at prediction time;
• transformation tests: known input–output examples;
• performance tests: latency, throughput and memory;
• statistical tests: metric confidence intervals or comparison rules;
• fairness tests: group-level performance where appropriate;
• security tests: authentication, authorization and adversarial inputs.

A gate should prevent promotion when a required condition fails. For example, a candidate may require a recall above a policy threshold, no critical security vulnerabilities and documented approval. Gates must be designed carefully: a rigid numerical threshold can encourage metric gaming, while an entirely manual process can be slow and inconsistent.

7. Monitoring Production Models

Monitoring should cover four related dimensions.

Operational monitoring

Operational metrics include request rate, error rate, latency, CPU and memory use, queue delay, replica health and cost. A model with excellent predictive quality is still unsuccessful if the service is unavailable or too slow.

Data-quality monitoring

Data monitoring checks schema, missingness, ranges, category frequencies and relationships among variables. A broken upstream pipeline may produce plausible-looking but incorrect values, making simple service-health checks insufficient.

Data drift

Data drift means that the distribution of inputs has changed. For a feature X, a comparison may be expressed as:

This is a conceptual statement, not proof that performance has deteriorated. A practical measure for a numeric feature is the population stability index (PSI), often defined by partitioning values into bins:

Here, Aᵢ is the proportion in bin i for the current or production data, Eᵢ is the corresponding reference proportion, and both proportions must be positive. Bins should be defined consistently, and the result should be treated as a screening signal rather than a universal decision rule. Thresholds depend on context and should be calibrated empirically.

For categorical variables, analysts may compare category proportions using a statistical test or divergence measure. A drift signal can arise from legitimate seasonal change, altered customer composition or a faulty data pipeline; investigation is required before retraining.

Concept drift and performance monitoring

Concept drift concerns a change in the relationship between inputs and the target. In conceptual notation:

It is possible to have input drift without concept drift, or concept drift without obvious input drift. Performance monitoring requires delayed ground truth, such as confirmed fraud outcomes or later loan repayment status. Relevant measures may include accuracy, precision, recall, F1 score, calibration, mean absolute error or business loss, depending on the task.

For binary classification:

Here, TP is the number of true positives, FP false positives and FN false negatives. These metrics require clearly defined positive and negative classes and are meaningful only with an appropriate labelled evaluation sample.

Monitoring must distinguish an alert from an action. An alert may trigger investigation; an action may be rollback, feature correction, threshold adjustment or retraining. Automatic retraining without review can reproduce corrupted data or encode a temporary anomaly into the model.

8. Retraining and Lifecycle Decisions

Retraining can be scheduled, event-triggered or manually initiated. Scheduled retraining is simple but may waste resources or react poorly to sudden changes. Event-triggered retraining responds to drift, performance decline or new labelled data. Manual retraining is appropriate for high-risk models where expert review is required.

A retraining policy should specify:

• the data window and inclusion rules;
• label maturity requirements;
• validation and holdout strategy;
• comparison with the current champion model;
• fairness and subgroup checks;
• approval authority;
• deployment method and rollback plan;
• retention of old models and evidence.

A champion–challenger approach compares the production champion with one or more candidate challengers. The challenger should be evaluated on a representative, temporally appropriate dataset and under the same operational constraints. A higher offline score alone does not guarantee a better production model.

Models should be retired when their purpose ends, data sources disappear, risks become unacceptable, maintenance costs exceed value or a replacement is demonstrably safer. Retirement includes disabling endpoints, preserving required records, revoking credentials and communicating the change to stakeholders.

9. Model Governance and Responsible Use

Governance assigns accountability and establishes controls. A model owner is responsible for the model’s purpose, risks and lifecycle decisions. Data owners manage data access and quality. Engineering teams operate the service. Risk, compliance or review committees may provide independent challenge. These roles should be documented using a responsibility matrix.

A model card or model documentation record should describe:

• intended purpose and prohibited uses;
• training data provenance and limitations;
• population and environment considered;
• features and target definition;
• evaluation methods and subgroup results;
• known failure modes;
• interpretability information;
• monitoring plan and retraining triggers;
• owner, version, approval date and expiry or review date.

Risk-based governance

Not every model requires the same controls. A low-risk recommendation for internal document search may need less formal approval than a model influencing employment, credit, insurance, healthcare or access to essential services. Risk assessment should consider impact, affected populations, autonomy, reversibility, scale, uncertainty and legal obligations.

Fairness

Fairness assessment must be tied to the application and legal context. Group-level differences in selection rates, false-positive rates, false-negative rates or calibration may reveal disparate impacts, but no single metric resolves every ethical or legal question. Some fairness criteria are mathematically incompatible except under special conditions, particularly when base rates differ. Analysts must therefore document the selected criterion, its rationale, limitations and mitigation strategy.

Mitigation may occur before training through data improvement, during training through constrained objectives or reweighting, or after training through threshold and decision-process changes. Removing a protected attribute does not guarantee fairness because correlated variables may act as proxies.

Privacy and security

Sensitive data should be collected and retained only for legitimate purposes. Controls include least-privilege access, encryption, secure secrets management, retention limits, pseudonymization where suitable and audit logs. Models can also leak information through memorization or inference attacks. Security testing should consider malicious inputs, model extraction, poisoning, insecure dependencies and unauthorized access.

Explainability and contestability

Users affected by consequential decisions may need an understandable explanation, a route to correction and access to human review. A technically complex explanation is not automatically useful. Explanations should accurately describe the system, distinguish correlation from causation and communicate uncertainty. Governance should record when human override is required and how overrides are audited.

Auditability

An audit trail should connect a prediction or decision to the model version, input-data reference, relevant configuration, timestamp, responsible service and approval status, subject to privacy and retention requirements. Auditability is not the same as storing every raw feature indefinitely; records must be proportionate, secure and legally justified.

10. Applied Case: Illustrative Retail Credit-Delay Model

Background

A fictional retail bank develops a model to identify applications that may require additional manual review because of likely repayment difficulty. The model is not permitted to make a final approval decision automatically. It supports trained reviewers.

Problem

The initial prototype performs well on a historical test set, but the bank discovers that the production service sometimes receives missing income fields and that application patterns differ from the training period. The risk committee also requires evidence that performance is monitored across relevant customer groups.

Analysis

The team inventories the model artefacts and finds that the notebook used one income-imputation rule while the API used another. The model registry contains several files with unclear version labels. There is no formal rollback procedure, and production logs do not identify the model version. The team therefore treats the problem as a lifecycle and governance failure, not simply as a modelling problem.

Approach and solution

The bank creates a versioned feature pipeline containing schema validation, imputation and prediction. The service contract rejects invalid requests and returns a controlled error for missing mandatory fields. The model, preprocessing code and dependencies are built into a scanned container image referenced by an immutable digest.

The experiment-tracking system records the data snapshot, code commit, feature version, parameters and evaluation results. A model card states that the model is for reviewer prioritization only. The approval workflow requires risk review, subgroup performance analysis, security review and a documented rollback target.

The new release is first deployed in shadow mode. Monitoring covers latency, request errors, missingness, input distributions and delayed repayment outcomes. A canary release then serves a small proportion of applications. The system automatically stops expansion if operational thresholds fail, while performance degradation triggers investigation rather than automatic retraining.

Outcome

The bank identifies an upstream data-mapping defect before the model is used for decisions. After correction, the model is released with complete lineage and a tested rollback plan. Reviewers receive a clear explanation of the model’s role and are required to consider additional evidence rather than treating the score as a final decision.

Learning points

1. A model file alone is not a production system.
2. Training–serving consistency is a critical control.
3. Monitoring must include data, operations and delayed predictive outcomes.
4. Governance should define permitted use and human responsibility.
5. Canary deployment and rollback reduce the impact of release errors.

11. Challenges and Best Practices

Common challenges include unclear ownership, hidden data dependencies, unversioned features, insufficient labels, alert fatigue, excessive cloud cost, privacy risks and pressure to deploy before validation is complete. MLOps can also become over-engineered: a small, low-risk batch model may not require a large orchestration platform.

Practical best practices are:

• begin with a clearly defined decision and risk classification;
• keep models and preprocessing in versioned, testable artefacts;
• automate repeatable checks but retain appropriate human approval;
• use immutable releases and maintain a tested rollback path;
• monitor service health, data quality, drift and outcome performance;
• define alert thresholds with owners and response procedures;
• record assumptions, limitations and intended use;
• protect personal data and secrets throughout the pipeline;
• evaluate subgroup performance where relevant;
• review model value, risk and maintenance burden periodically;
• design for retirement from the beginning.

Contemporary Developments

Current practice increasingly treats machine-learning systems as part of broader software supply chains. Feature stores can standardize reusable features, though they introduce governance and consistency responsibilities. Model observability platforms combine operational and predictive monitoring. Infrastructure-as-code makes environments more repeatable. Confidential computing, privacy-enhancing methods and federated learning may reduce exposure of sensitive data, but they introduce technical and governance trade-offs.

Generative AI systems create additional concerns, including prompt and retrieval versioning, hallucination monitoring, evaluation of open-ended outputs, copyright and confidential-data leakage. The same lifecycle principles remain applicable: define intended use, test before release, monitor in context, document limitations and maintain accountable human oversight.

Chapter Summary

Deployment makes a trained model available for operational use, while MLOps provides repeatable practices for building, testing, releasing, monitoring and maintaining machine-learning systems. Batch, online, streaming and edge patterns have different latency, scale, reliability and privacy requirements. APIs require explicit contracts, validation, authentication and controlled error handling. Containers improve portability, but secure images, immutable versions and access controls remain necessary.

Experiment tracking and model registries connect code, data, features, parameters, environments and evaluation results. Automated pipelines can perform testing, security scanning, approval and staged release. Production monitoring must cover operational health, data quality, drift and delayed model performance. Drift is evidence for investigation, not an automatic justification for retraining.

Model governance establishes ownership, risk-based approval, documentation, fairness assessment, privacy, security, explainability, auditability and retirement controls. A reliable model is therefore not merely accurate: it is reproducible, appropriately used, observable, secure, maintainable and accountable.

Key Terms

Deployment: Making a trained model available for operational use.

MLOps: Engineering and organizational practices for managing machine-learning lifecycles.

Model governance: Policies and controls governing model risk, use, approval and accountability.

Batch inference: Generating predictions for a collection of records at scheduled or on-demand intervals.

Online inference: Returning a prediction in response to an individual request.

API: A defined interface through which software systems exchange requests and responses.

Container: A portable runtime package containing application code and dependencies.

Model registry: A controlled catalogue of model versions and lifecycle states.

Experiment tracking: Recording data, code, parameters, environments and results for a model run.

Data drift: Change in the distribution of model inputs over time.

Concept drift: Change in the relationship between inputs and the target variable.

Training–serving skew: Difference between transformations used during training and production serving.

Canary deployment: Gradual release of a new version to a small portion of live traffic.

Rollback: Returning a service to a previously approved working version.

Model card: Documentation describing a model’s purpose, performance, limitations and risks.

Champion model: The currently deployed model used as a baseline for comparison.

Challenger model: A candidate model evaluated against the champion.

Audit trail: Traceable evidence of actions, versions, decisions and system events.

Conceptual drift: Change in the statistical relationship represented by P(Y | X).

Service-level objective: A target for an operational property such as latency or availability.

Review Questions

5 Short Answer Questions

1. Distinguish batch inference from online inference and give one use case for each.
2. Why should preprocessing logic normally be versioned with the model?
3. What information should an experiment-tracking record contain?
4. Define data drift and explain why it does not necessarily prove performance degradation.
5. State two purposes of a model registry.

5 Long Answer Questions

1. Explain the stages of an MLOps pipeline from source validation to production monitoring.
2. Compare shadow, canary and blue–green deployment strategies, including their benefits and limitations.
3. Discuss the roles of containers, image registries and cloud orchestration in model serving.
4. Explain how model governance addresses fairness, privacy, security, explainability and auditability.
5. Describe a defensible retraining policy for a model whose labels become available several weeks after prediction.

3 Analytical / Application Questions

1. A production classifier shows stable latency and error rates, but the distribution of one important feature changes sharply after a business-process update. Design an investigation plan before deciding whether to retrain.
2. A candidate model has higher overall accuracy but substantially higher false-negative rates for one customer group. Recommend the evidence and governance steps required before deployment.
3. Design a release and rollback plan for an online fraud model. Specify the artefacts to version, tests to automate, monitoring metrics and conditions that should stop or reverse the release.

Multiple Choice Questions with Answers

Question 1

Which deployment pattern generates predictions periodically for a collection of records?

A. Batch inference

B. Online inference

C. Edge inference

D. Shadow inference

Correct Answer: A. Batch inference

Question 2

What is the main purpose of a model registry?

A. To replace all data-quality tests

B. To catalogue and control model versions and lifecycle states

C. To increase the mathematical complexity of a model

D. To eliminate the need for monitoring

Correct Answer: B. To catalogue and control model versions and lifecycle states

Question 3

Which practice best reduces training–serving inconsistency?

A. Using different preprocessing code in development and production

B. Removing all input validation

C. Packaging versioned preprocessing with the prediction pipeline

D. Deploying only the model weights

Correct Answer: C. Packaging versioned preprocessing with the prediction pipeline

Question 4

A canary deployment primarily means that:

A. The new model receives a small fraction of live traffic before wider release

B. The old model is permanently deleted before testing

C. The model is trained without labelled data

D. All production traffic is sent to two models and both outputs are used automatically

Correct Answer: A. The new model receives a small fraction of live traffic before wider release

Question 5

In the precision formula, what does FP represent?

A. False predictions that are negative

B. False positives

C. Fixed parameters

D. Feature proportions

Correct Answer: B. False positives

Question 6

Which statement about data drift is correct?

A. It always proves that model accuracy has declined

B. It is a change in the distribution of inputs and requires investigation

C. It can occur only when source code changes

D. It automatically requires immediate retraining

Correct Answer: B. It is a change in the distribution of inputs and requires investigation

Question 7

Which artefact most directly documents intended use, limitations and known risks of a model?

A. Model card

B. Container kernel

C. Load balancer

D. Message queue

Correct Answer: A. Model card

Question 8

Why are immutable image digests preferred for production releases?

A. They allow the contents referenced by a release to be identified reliably

B. They guarantee that the model is fair

C. They remove the need for access control

D. They make all predictions explainable

Correct Answer: A. They allow the contents referenced by a release to be identified reliably

Question 9

A model has TP = 80 and FP = 20. What is its precision?

A. 20%

B. 50%

C. 80%

D. 100%

Correct Answer: C. 80%

Question 10

Which action is most appropriate when a monitoring alert indicates possible concept drift?

A. Automatically delete all historical models

B. Investigate using suitable outcome labels and decide on an evidence-based response

C. Ignore the alert because operational latency is stable

D. Change the API endpoint without evaluating the model

Correct Answer: B. Investigate using suitable outcome labels and decide on an evidence-based response