Docker in CI/CD, Cloud, and Orchestration

Introduction

Docker changes software delivery by packaging an application, its runtime, and its dependencies into a portable image. In a development environment, this improves consistency. In a delivery environment, however, a container image is only one component of a larger system. A reliable production path must build the image reproducibly, test it, inspect it for vulnerabilities, publish it to a trusted registry, deploy it, and observe its behaviour after release.

This chapter examines Docker as part of that end-to-end delivery system. It explains a typical continuous integration and continuous delivery (CI/CD) pipeline, image security and supply-chain controls, deployment patterns in cloud environments, and the transition from Docker Compose to orchestration platforms such as Kubernetes. The emphasis is practical: a certificate-level DevOps practitioner should be able to design a sensible pipeline, identify its control points, and understand when a simple deployment model must evolve.

Learning Objectives

By the end of this chapter, you should be able to:

• describe the role of Docker in CI/CD;
• design pipeline stages for building, testing, scanning, signing, and publishing images;
• distinguish image tags, immutable digests, registries, and repositories;
• explain secure authentication and supply-chain practices;
• compare deployment options on cloud platforms;
• identify the differences between Docker Compose and Kubernetes;
• apply health checks, configuration management, rollback, and observability principles; and
• evaluate an appropriate migration path from Compose to orchestration.

Docker’s Role in the Delivery Lifecycle

A CI/CD pipeline is an automated sequence that takes a change from source control toward a deployable and operational state. Docker can provide a consistent unit of delivery throughout that sequence.

A representative flow is:

1. A developer commits code and opens a pull request.
2. CI checks out the source and builds the application image.
3. Automated unit, integration, and security tests execute.
4. The image is scanned and, if policy permits, signed or attested.
5. The approved image is pushed to a registry.
6. A deployment system updates a test, staging, or production environment.
7. Monitoring and health checks provide evidence about the release.
8. The release is promoted, paused, or rolled back according to policy.

Figure: A feedback-oriented Docker CI/CD pipeline.

The pipeline should promote the same tested artifact rather than rebuilding separately for each environment. Rebuilding can produce a different dependency resolution or base-image layer. Environment-specific values should normally be supplied through configuration, secrets, or deployment manifests, not by changing the application image.

Building Reproducible Images

A Dockerfile is a build recipe. Good Dockerfiles are explicit, small enough to maintain, and deterministic where practical. Common practices include:

• selecting a maintained and trusted base image;
• pinning important base-image versions, preferably with a digest where strict reproducibility is required;
• using multi-stage builds so compilers and development tools are excluded from the runtime image;
• copying dependency manifests before application source when this improves build-cache reuse;
• running the application as a non-root user;
• adding a .dockerignore file; and
• avoiding secrets in ARG, ENV, image layers, or source copied into the build context.

A simplified multi-stage example is:

FROM node:22-alpine AS build
WORKDIR /src
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:22-alpine
WORKDIR /app
ENV NODE_ENV=production
COPY package*.json ./
RUN npm ci --omit=dev && adduser -D appuser
COPY --from=build /src/dist ./dist
USER appuser
EXPOSE 8080
CMD ["node", "dist/server.js"]

The exact base image and commands depend on the application. EXPOSE documents an intended container port; it does not publish that port by itself. A runtime platform must configure networking explicitly.

Tags and Digests

An image reference such as registry.example.com/orders:1.4.0 contains a registry, repository, and tag. A tag is a human-readable label and can be moved to another image. A digest, such as sha256:..., identifies image content immutably. Production deployment specifications should preferably record a digest, or use a controlled mechanism that resolves a release tag to a digest and preserves that record.

A useful convention is to attach both a release tag and source metadata:

orders:1.4.0
orders:git-a81c2f7

Tags support discovery and operations; digests support reproducibility and auditability.

Pipeline Stages and Quality Gates

Source and change validation

The pipeline should run on pull requests and protected branches. It can check formatting, dependency manifests, Dockerfile quality, and whether prohibited files or credentials have been committed. Branch protection should require successful checks before merging.

Build

The CI runner builds the image in an isolated environment. BuildKit and related builders can provide caching, multi-platform builds, and improved secret handling. Cache use should not weaken isolation: untrusted pull-request code must not receive production credentials or unrestricted access to a sensitive cache.

Test

Testing should occur at several levels:

• Unit tests test functions or modules in isolation.
• Integration tests test interactions with databases, queues, or other services.
• Contract tests test agreed interfaces between components.
• End-to-end tests exercise important user workflows.
• Container tests verify the image starts, listens on the expected interface, uses correct permissions, and responds to a health endpoint.

For integration tests, disposable dependent services can be started with Compose or a test framework. Tests should avoid assuming that localhost inside one container refers to another container; services communicate using service names and container networks.

Security scanning

Scanning may include:

• operating-system and language-package vulnerability scanning;
• secret detection;
• static analysis of application code;
• Dockerfile and configuration checks; and
• licence-policy checks.

A vulnerability report is not automatically a reason to block every build. Teams should define severity thresholds, exploitability exceptions, remediation deadlines, and an approval process. Scanners can produce false positives and can miss vulnerabilities; scanning is a control, not proof of safety.

Publishing

Only images that pass required gates should be pushed to a release repository. CI should authenticate using short-lived workload identity or narrowly scoped credentials. Registry permissions should separate push, pull, and administrative roles. A registry should support access control, audit logs, retention policy, vulnerability information, and preferably image signing or provenance attestations.

Deployment and promotion

A mature pipeline separates artifact creation from environment promotion. For example, the pipeline can build once, deploy the digest to staging, run smoke tests, and then promote that exact digest to production. Deployment may be controlled by a pipeline, GitOps controller, or cloud service. Approval gates can be appropriate for high-risk production changes, but they should not replace automated evidence.

A Practical Pipeline Definition

The following pseudocode is platform-neutral:

checkout source
run linting and unit tests
build image with immutable source identifier
run integration tests against disposable services
scan image, dependencies, and configuration
if policy fails: stop
push image to trusted registry
record digest, test results, and provenance
 deploy digest to staging
run smoke and health tests
if approved or policy conditions pass: deploy same digest to production
monitor release and execute rollback policy if necessary

A pipeline should fail clearly and early. Logs must identify the failed stage without exposing credentials. The pipeline itself is production software: it requires review, access control, versioning, and testing.

Security and Software Supply-Chain Governance

Container security has several layers. The image may contain vulnerable packages; the application may have coding weaknesses; the runtime may be over-privileged; the registry may be compromised; or the deployment identity may have excessive authority.

Important controls include:

1. Least privilege: use non-root users, drop unnecessary Linux capabilities, and avoid privileged containers.
2. Minimal exposure: publish only necessary ports and use network policies or cloud security controls.
3. Secret separation: inject secrets at runtime through a secret manager; do not bake them into images.
4. Image provenance: record source commit, builder, dependencies, and build time.
5. Signing and verification: require trusted signatures or attestations before deployment where the platform supports it.
6. Patching: rebuild images when base images or dependencies receive security updates.
7. Runtime protection: use read-only filesystems, resource limits, isolation, and runtime monitoring where appropriate.
8. Access review: regularly review registry, CI, cloud, and cluster permissions.

The software supply chain includes code, dependencies, build tools, runners, registries, and deployment systems. A secure image cannot compensate for a compromised build runner or an unprotected deployment credential.

Cloud Deployment Options

Cloud platforms offer several levels of operational responsibility. Names differ among providers, but the underlying choices are similar.

Cloud deployment requires more than starting a container. The design must address networking, identity, persistent storage, configuration, secrets, logs, metrics, backups, scaling, availability zones, and cost.

Stateless web services are usually easiest to containerize. A container should be replaceable: persistent business data belongs in a managed database, object store, or explicitly designed persistent volume rather than in an ephemeral writable layer. Stateful workloads can run in orchestration systems, but they require careful storage, backup, replication, and recovery design.

Configuration and Secrets

Configuration such as an environment name or external service endpoint should be externalized. Secrets require stronger controls than ordinary configuration: encryption at rest, access auditing, rotation, and restricted distribution. Environment variables are convenient but can appear in process inspection, diagnostic output, or platform interfaces. A cloud secret manager or orchestrator secret mechanism is preferable for sensitive values.

Health, Scaling, and Availability

A deployment platform commonly distinguishes:

• a startup or initialization check, which determines whether an application has completed startup;
• a readiness check, which determines whether it should receive traffic; and
• a liveness check, which determines whether it is stuck and should be restarted.

A liveness check must not simply test a dependency whose temporary outage should not cause every instance to restart. Readiness can reasonably include dependency availability when traffic should be withheld. Poorly designed checks can create restart loops or remove all instances from service.

Scaling may be horizontal (more container instances) or vertical (more resources per instance). Horizontal scaling is effective only when the application and its dependencies can support concurrent instances. Queues, databases, rate limits, and connection pools may become bottlenecks.

From Docker Compose to Orchestration

Compose describes a multi-container application, commonly for development, testing, or a small controlled deployment. A Compose file can define services, networks, volumes, ports, environment variables, and dependencies. It is valuable because it gives developers a repeatable local environment.

An orchestrator manages workloads across one or more hosts. It schedules containers, maintains a desired number of replicas, routes traffic, performs rolling updates, handles service discovery, and integrates with storage, identity, and monitoring. Kubernetes is a widely used example.

The conceptual transition is from “start these containers together” to “continuously reconcile the desired state with the observed state.” If a service is declared to have three replicas, an orchestrator attempts to restore three when one fails. This is a control-loop model.

The mapping is not one-to-one. A Compose depends_on relationship expresses startup ordering or a condition; it does not provide reliable distributed readiness. In Kubernetes, services should tolerate ordering differences and retry connections. A Deployment manages replicated stateless Pods, while a StatefulSet is intended for workloads requiring stable identity or ordered storage relationships. A Job represents finite work.

Kubernetes Objects in Brief

A Pod is the smallest deployable unit and commonly contains one application container plus optional supporting containers. A Deployment manages replicated, replaceable Pods and supports rolling updates. A Service provides a stable virtual address for a set of Pods. An Ingress or Gateway exposes HTTP traffic through routing rules. A ConfigMap stores non-secret configuration, while a Secret stores sensitive configuration, subject to platform protection and external management practices.

A basic Deployment fragment might be conceptually represented as:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: orders
spec:
  replicas: 3
  selector:
    matchLabels:
      app: orders
  template:
    metadata:
      labels:
        app: orders
    spec:
      containers:
        - name: orders
          image: registry.example.com/orders@sha256:IMAGE_DIGEST
          ports:
            - containerPort: 8080
          readinessProbe:
            httpGet:
              path: /ready
              port: 8080

This fragment is illustrative rather than complete. Production manifests should also address resource requests and limits, security context, liveness and startup checks, update strategy, disruption budgets, configuration, and observability.

Deployment Strategies and Rollback

A rolling deployment gradually replaces old instances with new ones. It reduces interruption but requires compatible interfaces and sufficient capacity. A blue-green deployment maintains two environments and switches traffic between them, making rollback conceptually simple but potentially doubling resources. A canary deployment sends a small proportion of traffic to the new version and increases it when metrics remain healthy.

Rollback is effective only when the previous image, configuration, database compatibility, and deployment definition remain available. Database migrations deserve particular care: a migration that destroys compatibility may make application rollback unsafe. Expand-and-contract migration patterns reduce this risk by first adding compatible structures, then migrating use, and removing obsolete structures later.

Observability and Operational Feedback

A deployed container is not successful merely because it started. Operators need:

• Logs, preferably structured and correlated with request or trace identifiers;
• Metrics, such as request rate, error rate, latency, saturation, restart count, and resource consumption;
• Traces, where distributed requests cross services; and
• Alerts, tied to user impact or actionable conditions.

A useful operational question is not simply “Is the container running?” but “Can users complete the business operation within an acceptable time and error level?” Health checks, service-level objectives, and release dashboards should answer that question.

Illustrative Case: Northstar Orders

Background

Northstar, a small online retailer, runs an API, a web frontend, and a background worker. Developers use Docker Compose locally. Production initially runs manually started containers on a virtual machine.

Problem

Releases are inconsistent. The production host has an older image than the developer’s machine, secrets have occasionally appeared in shell history, and a failed release requires manually identifying and restarting the previous containers. The team also lacks reliable evidence that a release is healthy.

Analysis

The fundamental problem is not Docker itself; it is the absence of an artifact and promotion process. The team is rebuilding or selecting images informally, mixing configuration with deployment commands, and treating process survival as application health.

Approach

Northstar introduces the following pipeline:

1. Pull requests run unit tests, dependency checks, and Dockerfile linting.
2. A multi-stage build creates one image tagged with the commit identifier.
3. Integration tests start a temporary database and exercise the API.
4. The image is scanned; critical policy violations block publication.
5. The image is pushed to a private registry, and its digest is recorded.
6. Staging receives the digest and runs smoke tests against /ready and key API operations.
7. Production receives the same digest through a controlled deployment.
8. Secrets move to a managed secret store, and the container runs as a non-root user.
9. Logs and latency metrics are collected, with an automated rollback condition for severe release degradation.

As traffic grows, Northstar moves the stateless API and worker to a managed Kubernetes service. The database remains a managed database service. Compose remains the local development tool; Kubernetes becomes the production scheduling and reconciliation platform.

Outcome

The team can identify exactly which source revision is running, reproduce a tested artifact, and roll back to a known digest. Deployment becomes more repeatable, while operational complexity increases enough that Northstar must invest in cluster access control, resource policies, monitoring, and staff training.

Learning Points

The case demonstrates that orchestration is not a substitute for pipeline discipline. Build once and promote the same artifact; externalize configuration; design meaningful health checks; keep state in systems designed for persistence; and adopt Kubernetes because operational requirements justify it, not because it is fashionable.

Challenges, Limitations, and Governance

Containers improve packaging but do not eliminate dependency conflicts, insecure code, data-management problems, or operational responsibility. Container images can become large, scanners can produce false positives, and orchestration platforms can be difficult to configure securely. Kubernetes also introduces a substantial control plane, policy surface, networking model, and learning requirement.

Governance should define who may publish images, approve production deployment, access secrets, change cluster policies, and respond to vulnerabilities. Audit records should connect source changes to builds, image digests, approvals, and deployments. Regulatory requirements may require retention, data residency, access evidence, vulnerability management, and separation of duties. These controls should be proportionate to the application’s risk.

AI-assisted coding and image generation can accelerate development, but generated Dockerfiles and pipeline definitions must be reviewed. AI may suggest insecure defaults, obsolete base images, excessive privileges, or leaked secrets. Responsibility for the resulting deployment remains with the engineering team.

Best Practices Checklist

• Build images from reviewed Dockerfiles and maintained base images.
• Use multi-stage builds and .dockerignore where appropriate.
• Test the application and the container, not merely the Dockerfile syntax.
• Scan code, dependencies, images, and deployment configuration.
• Never place credentials in image layers or source control.
• Push only approved images to a controlled registry.
• Record immutable digests and promote the same artifact across environments.
• Run with least privilege and define resource and health policies.
• Externalize configuration and persistent data.
• Keep Compose for suitable local workflows and adopt orchestration for justified operational needs.
• Monitor user-facing health and maintain a tested rollback plan.

Chapter Summary

Docker provides a consistent artifact for modern delivery, but effective CI/CD requires more than building an image. A sound pipeline validates source changes, builds reproducibly, runs appropriate tests, scans for risk, publishes to a controlled registry, records provenance, and deploys the same immutable artifact through environments. Tags aid human use, whereas digests provide content identity.

Cloud deployment ranges from self-managed virtual machines to managed container services and Kubernetes. The chosen model determines operational responsibility, portability, cost, and complexity. Compose remains valuable for local development and small controlled systems, while orchestration platforms provide reconciliation, scheduling, service discovery, scaling, and rollout management. Migration should be driven by real requirements.

Security, configuration, health checks, observability, governance, and rollback are essential parts of container delivery. A container is operationally successful only when it is secure, supportable, observable, and useful to its users.

Key Terms

Container image: A packaged, immutable filesystem and metadata used to create containers.

CI/CD: Practices and automation for continuously integrating, testing, delivering, and deploying software.

Registry: A service that stores and distributes container images.

Repository: A named collection of related images in a registry.

Image tag: A mutable, human-readable label associated with an image.

Image digest: A content-derived identifier for a specific image.

Build context: Files made available to a Docker build.

Multi-stage build: A Docker build using multiple stages to separate compilation from runtime content.

Health check: An automated test of an application’s operational condition.

Readiness: The condition in which an instance should receive traffic.

Liveness: The condition in which an instance is functioning sufficiently to remain running.

Orchestration: Automated scheduling, supervision, scaling, and management of containers.

Pod: Kubernetes’ smallest deployable unit.

Deployment: A Kubernetes controller for managing replicated, replaceable Pods.

Service: A stable network abstraction for reaching selected workloads.

ConfigMap: Kubernetes storage for non-secret configuration.

Secret: Sensitive configuration managed by an orchestrator or external secret system.

Provenance: Evidence describing how and from what source an artifact was built.

Canary release: A release exposed gradually to a limited portion of traffic.

GitOps: An operating model in which version-controlled desired state drives deployment automation.

Review Questions

Short Answer Questions

1. Why should a production deployment usually use the same image digest that passed staging tests?
2. What is the difference between an image tag and an image digest?
3. Name three categories of automated tests useful in a Docker pipeline.
4. Why should application secrets not be baked into a Docker image?
5. What is the primary operational difference between Compose and an orchestrator?

Long Answer Questions

1. Describe a complete CI/CD pipeline for a Dockerized web application, including quality gates.
2. Explain how multi-stage builds, non-root execution, scanning, and signing contribute to container security.
3. Compare deployment on a virtual machine, a managed container service, and managed Kubernetes.
4. Explain the roles of Pods, Deployments, Services, and ConfigMaps in Kubernetes.
5. Discuss why database migrations can complicate rollback and describe an approach that reduces risk.

Analytical / Application Questions

1. A team builds a new image separately for development, staging, and production. Analyse the reproducibility risks and propose a corrected promotion design.
2. An application restarts repeatedly whenever its database is temporarily unavailable. Determine whether its health checks are likely misdesigned and recommend separate readiness and liveness behaviours.
3. A small team is considering Kubernetes for a low-traffic application currently deployed with Compose. Evaluate the technical and organizational factors that should inform the decision.

Multiple Choice Questions with Answers

Question 1

Which artifact should normally be promoted unchanged from staging to production?

A. The source branch only

B. The same tested image digest

C. A newly rebuilt image with the same tag

D. A manually copied container filesystem

Correct Answer: B. The same tested image digest

Question 2

What does an image digest primarily provide?

A. A mutable human-readable alias

B. Automatic horizontal scaling

C. Content-based identification of an image

D. A replacement for a registry

Correct Answer: C. Content-based identification of an image

Question 3

Which Dockerfile practice most directly keeps build tools out of the runtime image?

A. Using a multi-stage build

B. Publishing more ports

C. Adding an EXPOSE instruction

D. Using a floating production tag

Correct Answer: A. Using a multi-stage build

Question 4

Which item should generally be supplied at runtime rather than baked into an image?

A. A compiled application binary

B. A static HTML file

C. A secret database password

D. A base-image operating system

Correct Answer: C. A secret database password

Question 5

What is the main purpose of a readiness check?

A. To decide whether an instance should receive traffic

B. To compile application source code

C. To sign an image in a registry

D. To allocate persistent storage

Correct Answer: A. To decide whether an instance should receive traffic

Question 6

Which Kubernetes object provides a stable network abstraction for selected Pods?

A. ConfigMap

B. Service

C. Deployment

D. Job

Correct Answer: B. Service

Question 7

What is a major characteristic of orchestration compared with simple Compose startup?

A. It removes the need for application tests

B. It continuously works toward a declared desired state

C. It requires every workload to store data in its container

D. It prevents rolling updates

Correct Answer: B. It continuously works toward a declared desired state

Question 8

Which deployment strategy gradually exposes a release to a small proportion of users?

A. Canary

B. Big-bang

C. Cold backup

D. Single-stage

Correct Answer: A. Canary

Question 9

Why are persistent business data and databases commonly kept outside an ephemeral container filesystem?

A. Containers cannot use networks

B. Container filesystems are not designed to be the primary durable data store

C. Registries cannot store images with volumes

D. Kubernetes prohibits all storage

Correct Answer: B. Container filesystems are not designed to be the primary durable data store

Question 10

Which statement about Docker image tags is correct?

A. A tag is always immutable

B. A tag cannot identify a repository

C. A tag is a human-readable label that can be moved

D. A tag contains the complete vulnerability report

Correct Answer: C. A tag is a human-readable label that can be moved