Containers and Kubernetes Deployment

Introduction and Context

Modern applications are frequently packaged as containers and deployed on managed cloud platforms. A container packages an application with its executable code, libraries, configuration defaults, and a controlled runtime environment. Unlike a virtual machine, a container normally shares the host operating system kernel, making it lighter and faster to start. Kubernetes extends this model by providing a declarative platform for scheduling containers, maintaining desired replicas, exposing services, managing configuration, and performing controlled updates.

For DevOps practitioners, container deployment is not simply a matter of building an image and starting it. A production deployment must address image security, registry access, networking, health, secrets, scaling, observability, failure recovery, and change management. AWS provides several relevant choices, including Amazon Elastic Container Registry (ECR), Amazon Elastic Container Service (ECS), and Amazon Elastic Kubernetes Service (EKS). Google Cloud provides Artifact Registry and Google Kubernetes Engine (GKE). ECS uses AWS-native task and service abstractions, whereas EKS and GKE provide managed Kubernetes control planes.

Learning Objectives

By the end of this chapter, a learner should be able to:

• explain images, containers, registries, pods, deployments, services, and ingress;
• build smaller and safer container images using a Dockerfile;
• publish and retrieve images from ECR or Artifact Registry;
• deploy a workload to ECS, EKS, or GKE using appropriate configuration;
• configure services, ingress, scaling, secrets, health checks, and rolling updates;
• distinguish readiness, liveness, and startup health checks;
• identify common security, reliability, cost, and governance risks.

1. The Container Model

A container image is an immutable, versioned package composed of filesystem layers and metadata. A container is a running instance of an image with a writable runtime layer, a process, networking, and resource constraints. Images should be treated as build artifacts, not as mutable servers. A reliable release therefore refers to a specific image digest or an immutable tag rather than repeatedly reusing a tag such as latest.

A container image commonly contains:

1. a minimal base operating-system layer;
2. application dependencies;
3. application source or compiled binaries;
4. a default command and arguments;
5. metadata such as exposed ports and environment defaults.

The image is built from a Dockerfile. For example:

# syntax=docker/dockerfile:1
FROM python:3.12-slim AS runtime
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY src/ .
RUN useradd --create-home appuser
USER appuser
EXPOSE 8080
CMD ["python", "app.py"]

This example is illustrative. In a compiled application, a multi-stage build would compile the program in a builder stage and copy only the resulting binary into a smaller runtime stage. Multi-stage builds reduce attack surface and image size because compilers and development libraries need not be shipped to production.

The .dockerignore file should exclude version-control metadata, local caches, credentials, test output, and unnecessary development files. The build context should be as small as possible. A deterministic build should pin important dependency versions and should be reproducible from source-controlled files.

Image Security Practices

Important practices include:

• use maintained, minimal base images;
• run as a non-root user;
• avoid embedding passwords, tokens, or private keys in image layers;
• scan dependencies and images for known vulnerabilities;
• sign images and verify signatures where supported;
• pin versions or digests for critical base images;
• rebuild when a base image receives security updates;
• grant the container only the Linux capabilities and filesystem access it requires;
• make the root filesystem read-only when the application permits it.

A vulnerability scanner identifies known issues; it does not prove that an image is secure. Teams must also review application behaviour, exposed interfaces, dependency provenance, and runtime permissions.

2. Registries and Image Promotion

A container registry stores, indexes, scans, and distributes images. A repository normally contains multiple tags and immutable content digests. AWS Elastic Container Registry integrates with IAM and ECS/EKS. Google Artifact Registry supports container repositories and integrates with IAM and GKE.

A typical workflow is:

source code → build → test → scan → sign → push → deploy by digest → monitor

A registry repository should be configured with appropriate access policies, encryption, retention rules, vulnerability scanning, and audit logging. CI should authenticate using short-lived workload identity or an equivalent role-based mechanism rather than long-lived static credentials.

Tags are convenient for humans, but tags may be moved. A digest identifies the exact content. A deployment manifest that uses an image digest gives stronger release reproducibility than one that uses an unprotected mutable tag.

3. Kubernetes Architecture and Objects

Kubernetes consists conceptually of a control plane and worker nodes. The control plane stores the desired state and coordinates scheduling. Worker nodes run workloads through a container runtime. In managed Kubernetes, AWS or Google Cloud operates much of the control plane, while the customer remains responsible for workloads, permissions, networking choices, and operational configuration.

Kubernetes uses declarative configuration. The operator describes the desired state, and controllers continually compare actual state with that description and take corrective action. This reconciliation model supports self-healing, scaling, and repeatable deployment.

Important objects include:

A Pod is ephemeral. Its IP address may change when it is replaced. Clients should therefore use a Service rather than address a Pod directly. A Deployment selects Pods through labels and maintains the requested replica count.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: orders
spec:
  replicas: 3
  selector:
    matchLabels:
      app: orders
  template:
    metadata:
      labels:
        app: orders
    spec:
      containers:
        - name: orders
          image: REGION-docker.pkg.dev/PROJECT/repo/orders@sha256:IMAGE_DIGEST
          ports:
            - containerPort: 8080
          resources:
            requests:
              cpu: "250m"
              memory: "256Mi"
            limits:
              cpu: "500m"
              memory: "512Mi"
          readinessProbe:
            httpGet:
              path: /ready
              port: 8080
            periodSeconds: 10
          livenessProbe:
            httpGet:
              path: /health
              port: 8080
            periodSeconds: 20
---
apiVersion: v1
kind: Service
metadata:
  name: orders
spec:
  selector:
    app: orders
  ports:
    - port: 80
      targetPort: 8080
  type: ClusterIP

The requests value influences scheduling: Kubernetes seeks a node with sufficient allocatable resources. The limits value constrains usage. CPU is commonly expressed in millicores, where 1000m represents one logical CPU unit in the Kubernetes resource model. Memory is specified as a quantity such as Mi or Gi; teams should use consistent conventions and understand the distinction between binary and decimal units.

4. Services, Ingress, and Cloud Networking

A ClusterIP Service is reachable inside the cluster. A LoadBalancer Service asks the cloud provider to provision an external load balancer. An Ingress provides layer-7 routing such as host- and path-based HTTP routing, but it requires an Ingress controller or cloud integration.

A production ingress design should consider TLS certificates, DNS, redirects, request limits, authentication, web-application firewall rules, client IP handling, and timeouts. The exact implementation differs between EKS and GKE. For example, cloud-native controllers may create an AWS load balancer or a Google Cloud load balancer, while an ingress controller deployed within the cluster may use a different operational model.

Network policies can restrict which Pods may communicate. They are valuable for least-privilege networking, but their effectiveness depends on cluster networking support and correct policy selection. A policy that accidentally denies DNS or health traffic can make an otherwise healthy application fail.

5. Health Checks and Resilience

Kubernetes distinguishes three important probe types:

• Startup probe: allows slow-starting applications time to initialise. Liveness checking is normally delayed until startup succeeds.
• Readiness probe: indicates whether a Pod should receive traffic. A Pod may be running but not ready while loading data or waiting for a dependency.
• Liveness probe: indicates whether a process is stuck and should be restarted.

A readiness endpoint should test whether the application can safely serve requests, but it should not perform an excessively expensive dependency cascade. A liveness endpoint should usually be shallow; making liveness depend on every downstream service can cause restart storms when an external dependency is temporarily unavailable.

Rolling updates require an application to tolerate more than one version during the transition. Database changes should follow backward-compatible expansion-and-contraction patterns: first add compatible schema elements, deploy code that can use both versions, migrate data, and remove obsolete elements only after old versions are gone.

Useful Deployment settings include maxUnavailable, which limits unavailable replicas during an update, and maxSurge, which limits additional replicas above the desired count. Their meaning is relative to the requested replicas and must be interpreted together with scheduling capacity and PodDisruptionBudgets.

6. Secrets and Configuration

Configuration that is not confidential can be stored in a ConfigMap. Passwords, API keys, certificates, and tokens require a secret-management approach. Kubernetes Secrets are objects designed for sensitive values, but base64 encoding is not encryption. Cluster administrators must configure encryption at rest, strict RBAC, audit logging, and controlled access. In many production environments, AWS Secrets Manager, AWS Systems Manager Parameter Store, Google Secret Manager, or a CSI driver is preferred so that applications retrieve secrets without placing them permanently in manifests.

Secrets should be rotated, scoped to the minimum required audience, and prevented from appearing in logs, crash dumps, command histories, and CI output. A service account should receive only the permissions required by its workload. Cloud workload identity mechanisms are preferable to distributing cloud access keys inside containers.

7. Scheduling, Scaling, and Cost

The scheduler considers resource requests, node capacity, affinity rules, taints, tolerations, topology constraints, and other policies. Poorly chosen requests can produce either waste or instability: excessive requests leave capacity unused, while inadequate requests cause contention and eviction.

The HorizontalPodAutoscaler changes the number of replicas based on observed metrics. For CPU utilisation, a simplified conceptual relationship is:

Here, actual CPU usage and requested CPU must be measured over compatible intervals and expressed in the same CPU unit. The relationship is useful for understanding CPU-based autoscaling, but production HPA behaviour also includes control-loop timing, tolerance, stabilization windows, and metric availability. It is not a guarantee that replica count will immediately match demand.

A rough capacity estimate can be made when each replica has a measured sustainable throughput of r requests per second and the target demand is D requests per second:

The result must be rounded up to a whole number and then checked against latency objectives, failure tolerance, resource limits, dependency capacity, and headroom. This is an engineering estimate, not a universal scaling law.

Cluster autoscaling can add or remove nodes when Pods cannot be scheduled or capacity is underused. Cloud costs include worker nodes, control-plane-related charges where applicable, load balancers, storage, network transfer, registries, logs, and monitoring. Cost optimisation must not remove redundancy or security controls without assessing operational risk.

8. ECS, EKS, and GKE Deployment Choices

ECS uses task definitions to describe containers, CPU and memory, networking, IAM roles, logging, and health checks. An ECS service maintains a desired number of tasks and can perform rolling deployments. ECS is often operationally simpler when an organisation wants AWS-native scheduling without adopting the full Kubernetes API.

EKS and GKE provide managed Kubernetes control planes. They are appropriate when portability, Kubernetes ecosystem integration, multi-team platform patterns, or existing Kubernetes expertise is important. They also require disciplined management of namespaces, RBAC, upgrades, add-ons, policies, and cluster networking.

The selection should be based on workload and organisational needs rather than popularity. Questions include:

• Is Kubernetes compatibility a requirement?
• Who will operate upgrades and cluster add-ons?
• Are teams prepared to manage Kubernetes security and observability?
• Does the platform need specialised cloud integrations?
• What are the portability, cost, and staffing implications?

9. Applied Case: Illustrative Online Ordering Service

Background

A fictional retailer has an online ordering API and wants to move from manually configured virtual machines to a repeatable cloud deployment. The API is stateless, listens on port 8080, uses a managed relational database, and must remain available during routine releases.

Problem

The existing process copies application files to servers. Releases are inconsistent, secrets are stored in shell scripts, and a failed process is not automatically replaced. The retailer needs controlled deployment, traffic routing, health checks, and the ability to handle a seasonal demand increase.

Analysis

The team identifies four risks: non-reproducible builds, excessive credential exposure, lack of automatic recovery, and insufficient capacity during peaks. It chooses a container image stored in Artifact Registry and GKE because the organisation already operates Kubernetes for other services. The same design could use ECR and EKS with cloud-specific identity and load-balancing integrations.

Approach and Solution

CI builds a multi-stage image, runs unit tests and image scanning, signs the image, and pushes it to a protected repository. The deployment references the image digest. A Kubernetes Deployment requests three replicas, defines CPU and memory requests, and uses startup, readiness, and liveness probes. A ClusterIP Service provides internal discovery, and an HTTPS Ingress routes orders.example.com to the Service.

The application receives non-secret settings from a ConfigMap. Database credentials are retrieved through a cloud secret integration and are accessible only to the orders service account. RBAC prevents developers from reading production secrets. Network policies permit ingress from the gateway and egress only to approved services and DNS.

A rolling strategy starts replacement Pods before terminating all old Pods. The team verifies that the database schema is backward compatible. HPA uses CPU and a business-relevant latency or queue metric where available. Cluster autoscaling provides additional nodes, while dashboards track error rate, latency, saturation, restart count, and rollout status.

Outcome

The retailer obtains repeatable releases, automatic replacement of failed Pods, controlled traffic entry, and auditable access to secrets. The deployment is not considered complete until load testing, failure testing, rollback testing, and restoration procedures have been demonstrated.

Learning Points

The central improvement is a chain of controls rather than one tool: immutable artifacts, least-privilege identity, declarative deployment, meaningful health checks, capacity policies, and observable release operations. Kubernetes can automate recovery, but it cannot correct an unsafe image, an incompatible database change, or an incorrectly designed health endpoint.

10. Deployment Workflow and Operational Practice

A practical workflow is:

1. commit source and infrastructure definitions;
2. run static analysis, unit tests, and dependency checks;
3. build the image with a clean, controlled context;
4. scan and, where required, sign the image;
5. push it to ECR or Artifact Registry;
6. deploy to a non-production environment;
7. execute integration, security, and load tests;
8. promote the same immutable artifact;
9. observe the rollout and verify health and business metrics;
10. roll back quickly if defined success criteria are not met.

A rollback should be rehearsed. kubectl rollout status can show progress, and kubectl rollout undo can return a Deployment to a previous revision, but rollback is effective only if the previous image, configuration, database compatibility, and required secrets remain available. ECS provides analogous service deployment and rollback mechanisms through its deployment configuration and release tooling.

Observability should include structured logs, metrics, traces, deployment events, and audit records. Useful service-level indicators include availability, request latency, error rate, and saturation. Alerts should identify actionable conditions rather than simply report every restart.

Challenges, Limitations, and Governance

Containers do not provide perfect isolation; the kernel, runtime, node, and control plane remain part of the trust boundary. Kubernetes adds flexibility but also complexity. Misconfigured RBAC, public load balancers, unrestricted metadata access, vulnerable images, and excessive permissions are common risks.

Governance should define approved base images, registry retention, vulnerability remediation deadlines, image signing requirements, environment separation, access review, incident response, and evidence retention. Privacy regulations may affect log content, geographic placement, encryption, and retention. Secrets and customer data must not be exposed through diagnostic output.

AI-assisted tools can help generate Dockerfiles, manifests, tests, and policy suggestions, but generated configuration must be reviewed. AI systems may reproduce insecure defaults, invent unsupported fields, leak sensitive prompts, or omit operational assumptions. Human review, automated validation, policy checks, and controlled testing remain necessary.

Best-Practice Checklist

• Build once and promote the same immutable image.
• Prefer image digests for production deployment.
• Use non-root containers and minimal, maintained bases.
• Scan dependencies and images, then manage exceptions explicitly.
• Keep secrets outside source code and image layers.
• Define resource requests and limits based on measurements.
• Use readiness, liveness, and startup probes for different purposes.
• Expose Pods through Services, not hard-coded Pod addresses.
• Protect ingress with TLS, authentication, rate controls, and suitable network policy.
• Test rolling updates, rollback, node failure, and dependency failure.
• Apply least privilege to cloud identities, service accounts, and RBAC.
• Monitor technical and business outcomes, not merely Pod status.

Chapter Summary

Containers package applications into portable, versioned runtime artifacts, while registries provide controlled storage and distribution. Secure image construction requires minimal bases, non-root execution, dependency control, scanning, signing, and avoidance of embedded credentials. ECS offers AWS-native task and service deployment; EKS and GKE provide managed Kubernetes platforms with declarative controllers and a broader ecosystem.

Kubernetes Pods are ephemeral, so Services provide stable discovery and Ingress provides external HTTP routing. Deployments maintain replicas and support rolling changes. Readiness, liveness, and startup probes have distinct operational purposes. ConfigMaps hold ordinary configuration, while secrets require encryption, least privilege, rotation, and preferably integration with managed secret stores. Resource requests, limits, autoscaling, and cluster autoscaling connect reliability with cost.

Professional deployment combines immutable artifacts, identity controls, health checks, observability, backward-compatible change design, and tested rollback. The platform automates many tasks, but sound architecture and governance remain the responsibility of the delivery team.

Key Terms

Container image: Immutable package containing application software and runtime dependencies.

Container: Running instance of an image.

Dockerfile: Declarative file describing image construction steps.

Registry: Service that stores and distributes container images.

Image digest: Content-based identifier for an exact image.

Pod: Kubernetes unit containing one or more co-located containers.

Deployment: Kubernetes controller that manages replicated Pods and updates.

Service: Stable virtual endpoint for a set of Pods.

Ingress: HTTP/HTTPS routing mechanism into cluster Services.

ConfigMap: Kubernetes object for non-confidential configuration.

Secret: Kubernetes object or integrated mechanism for sensitive values.

Readiness probe: Check determining whether a Pod should receive traffic.

Liveness probe: Check determining whether a container should be restarted.

Startup probe: Check protecting slow-starting applications during initialisation.

Rolling update: Gradual replacement of an old workload version with a new one.

HorizontalPodAutoscaler: Controller that adjusts replica count using metrics.

RBAC: Role-based access control for users, groups, and service accounts.

ECR: AWS Elastic Container Registry.

EKS: AWS managed Kubernetes service.

GKE: Google managed Kubernetes service.

Review Questions

Short Answer Questions

1. What is the principal difference between a container image and a running container?
2. Why is an image digest generally safer for production deployment than a mutable tag?
3. What is the purpose of a Kubernetes Service?
4. How does a readiness probe differ from a liveness probe?
5. Why should secrets not be embedded in a Dockerfile?

Long Answer Questions

1. Explain a secure container-image build and promotion workflow from source commit to production deployment.
2. Compare ECS, EKS, and GKE in terms of abstractions, operational responsibility, and likely use cases.
3. Explain the roles of Pods, Deployments, Services, Ingress, ConfigMaps, and Secrets in a Kubernetes application.
4. Discuss how resource requests, limits, HPA, and cluster autoscaling affect reliability and cost.
5. Describe the design considerations for a safe rolling update, including health checks and database compatibility.

Analytical / Application Questions

1. A Deployment has three replicas, but only one receives traffic because two Pods fail readiness checks. Analyse the likely consequences and propose a diagnostic sequence.
2. Design a deployment plan for a stateless API that must survive a node failure and a routine release. Specify replicas, probes, Service type, ingress controls, identity, and rollback evidence.
3. A team wants to solve a traffic spike by doubling CPU limits but not changing replica count. Evaluate this decision and recommend a measurement-based scaling strategy.

Multiple Choice Questions with Answers

Question 1

Which Kubernetes object provides a stable endpoint for a changing set of Pods?

A. Deployment

B. Service

C. ConfigMap

D. Namespace

Correct Answer: B. Service

Question 2

What does a container image digest primarily identify?

A. The cluster where an image runs

B. The image's content

C. The number of replicas

D. The image's network port

Correct Answer: B. The image's content

Question 3

Which probe determines whether a Pod should receive application traffic?

A. Readiness probe

B. Liveness probe

C. Startup probe only

D. Resource limit

Correct Answer: A. Readiness probe

Question 4

Which practice best protects a production container from unnecessary privilege?

A. Run the process as root

B. Store credentials in the image

C. Run as a non-root user

D. Use the latest tag exclusively

Correct Answer: C. Run as a non-root user

Question 5

Which Kubernetes object is intended for non-confidential configuration?

A. ConfigMap

B. Ingress

C. Service

D. PodDisruptionBudget

Correct Answer: A. ConfigMap

Question 6

A workload requires three replicas, and each replica has a measured sustainable throughput of 100 requests per second. Ignoring headroom and other limits, what minimum replica count is suggested for 250 requests per second?

A. 1

B. 2

C. 3

D. 5

Correct Answer: B. 2

Question 7

What is the main purpose of a rolling update?

A. Replace all nodes simultaneously

B. Gradually replace an old workload version

C. Delete the registry repository

D. Disable health checks

Correct Answer: B. Gradually replace an old workload version

Question 8

Which service is AWS's managed Kubernetes offering?

A. ECR

B. ECS

C. EKS

D. S3

Correct Answer: C. EKS

Question 9

Why is base64 encoding alone insufficient for protecting a Kubernetes Secret?

A. It changes the service type

B. It is not encryption

C. It prevents Pod scheduling

D. It removes the Secret object

Correct Answer: B. It is not encryption

Question 10

What is a key reason for using a startup probe?

A. To expose a Pod publicly

B. To allow a slow-starting application time to initialise

C. To store database passwords

D. To select a registry repository

Correct Answer: B. To allow a slow-starting application time to initialise