8. Containers, Kubernetes and Serverless Platforms

Introduction and Context

Cloud platforms have changed the unit of deployment from the virtual machine to the application workload. A modern DevOps team may package an application as a container, deploy it to a managed Kubernetes service, expose it through an ingress layer, persist selected data in cloud storage, and use serverless functions for event-driven tasks. These technologies are related, but they solve different operational problems. Containers provide packaging and process isolation; Kubernetes provides orchestration and control; serverless platforms abstract much of the infrastructure and execute code or containers in response to demand.

The three major cloud providers offer broadly comparable managed services. Amazon Elastic Kubernetes Service (EKS), Google Kubernetes Engine (GKE), and Azure Kubernetes Service (AKS) provide managed Kubernetes control planes and integrate with their respective identity, networking, storage, monitoring, and load-balancing services. AWS Lambda, Google Cloud Functions and Cloud Run, and Azure Functions provide serverless execution models, although their triggers, deployment models, limits, and operational abstractions differ.

For DevOps professionals, the central question is not which platform is universally best. It is how to select and operate the appropriate abstraction for a workload while controlling reliability, security, cost, portability, and organizational complexity.

Learning Objectives

By the end of this chapter, the reader should be able to:

1. Explain container images, registries, runtimes, and isolation mechanisms.

2. Describe Kubernetes architecture and the responsibilities of its principal objects.

3. Compare EKS, GKE, and AKS as managed Kubernetes platforms.

4. Design approaches for scaling, service discovery, ingress, configuration, secrets, and persistence.

5. Distinguish Kubernetes workloads from function-based and container-based serverless workloads.

6. Evaluate portability, security, cost, governance, and operational trade-offs.

7. Apply these concepts to a multi-service cloud deployment and identify common failure modes.

From Immutable Images to Managed Workloads

A container image is an immutable, layered package containing an application, its dependencies, metadata, and usually a default startup command. A Dockerfile or an equivalent build definition describes how the image is produced. Images are stored in registries such as Amazon Elastic Container Registry (ECR), Google Artifact Registry, and Azure Container Registry (ACR). A container runtime, commonly based on containerd or another OCI-compatible implementation, retrieves an image and creates an isolated process from it.

Container isolation is primarily implemented through operating-system mechanisms such as namespaces and control groups. Namespaces provide separate views of processes, networking, mounts, and users, while control groups constrain CPU, memory, and other resources. Containers are therefore lighter than virtual machines, but they do not constitute an absolute security boundary equivalent to a separate physical system or a strongly isolated virtual machine. Kernel vulnerabilities, excessive privileges, exposed sockets, and insecure images can compromise the intended isolation.

A production image should be small, reproducible, signed or otherwise verified where feasible, and free of unnecessary tools and credentials. Multi-stage builds separate compilation dependencies from the runtime image. Images should run as a non-root user, define a health-oriented startup model, and receive configuration at runtime rather than embedding environment-specific values. A software bill of materials and vulnerability scanning improve supply-chain visibility, but a scan is not a substitute for patching, least privilege, and runtime controls.

The image is only one part of the deployment. A workload also requires a policy describing replicas, resources, networking, identity, storage, update behavior, and observability. Kubernetes supplies a declarative model for expressing that policy.

Kubernetes as a Reconciliation System

Kubernetes is best understood not merely as a container scheduler but as a distributed reconciliation system. An operator declares a desired state, and Kubernetes controllers continually compare that state with observed reality. They create, replace, or remove resources until the two converge. This approach supports self-healing, rolling updates, and repeatable operations, although it does not eliminate the need for capacity planning, application-level resilience, or human judgment.

A Kubernetes cluster has a control plane and worker nodes. The API server is the primary interface and validates and persists resource definitions. The state store, commonly etcd, holds cluster state. The scheduler selects suitable nodes for unscheduled Pods, while controllers implement behavior such as replica maintenance, job completion, and node lifecycle management. Worker nodes run Pods through a container runtime and provide networking and storage integration.

A Pod is the smallest schedulable unit in Kubernetes. It may contain one container or several tightly coupled containers sharing a network namespace and volumes. Most applications should use a Deployment, which manages stateless replicated Pods and supports rolling replacement. StatefulSet is designed for workloads requiring stable identities and ordered behavior, although it does not itself make an application stateful or guarantee database correctness. DaemonSet places a Pod on selected nodes, often for agents such as log collectors. Job and CronJob represent finite or scheduled work.

A Service gives a stable virtual endpoint to a changing set of Pods selected by labels. This separation between client-facing identity and replaceable Pods is fundamental to Kubernetes service discovery. ClusterIP is internal by default; NodePort exposes a port on nodes; LoadBalancer requests an external load balancer through the cloud integration. In larger systems, an Ingress or Gateway resource routes HTTP or HTTPS traffic by host, path, or other policy. The actual data-plane behavior depends on an ingress or gateway controller, such as an application load balancer integration or an NGINX-based controller.

ConfigMaps hold non-sensitive configuration, while Secrets hold sensitive values. Kubernetes Secrets are encoded rather than automatically encrypted in every environment; encryption at rest, access controls, external secret managers, and rotation procedures must be deliberately configured. Cloud-native designs commonly connect Kubernetes workloads to AWS Secrets Manager, Google Secret Manager, or Azure Key Vault through an operator or CSI integration.

Resource requests express the minimum resources needed for scheduling, and limits constrain consumption. These values influence bin-packing, quality of service, autoscaling, and reliability. Readiness probes determine whether a Pod should receive traffic; liveness probes detect a process that requires restart; startup probes protect slow-starting applications from premature liveness failure. A poorly designed probe can create an outage by repeatedly terminating a healthy but slow service.

Managed Kubernetes Across AWS, Google Cloud, and Azure

EKS, GKE, and AKS manage the Kubernetes control plane, reducing the burden of operating API servers and control-plane availability. The customer still manages workload configuration, node pools or compute profiles, application security, policies, and much of the reliability design.

EKS integrates with IAM, VPC networking, Elastic Load Balancing, ECR, CloudWatch, and AWS storage services. Pod-level access can be implemented through mechanisms such as IAM Roles for Service Accounts or newer pod identity approaches. EKS offers substantial flexibility, but teams must understand the interaction between Kubernetes identity and AWS identity, as well as the complexity of VPC networking and node management.

GKE has a strong integration with Google Cloud IAM, VPC networking, Artifact Registry, Cloud Monitoring, and Persistent Disk or Filestore. Its operational model includes substantial automation and, depending on the selected mode, Google may manage more of node provisioning and scaling. GKE is often attractive where teams value close integration with Google’s observability and data services.

AKS integrates with Microsoft Entra ID, Azure Monitor, Azure Container Registry, Azure Load Balancer or Application Gateway, and Azure managed disks and files. It is particularly relevant to organizations with Microsoft identity, governance, and enterprise networking investments. Configuration choices around identity, private clusters, network plugins, and policy enforcement remain important.

The common Kubernetes API improves application portability, but portability is never complete. Storage classes, ingress annotations, load-balancer behavior, IAM bindings, monitoring agents, DNS, networking models, and managed database integrations are provider-specific. A portable architecture therefore distinguishes Kubernetes-standard manifests from provider adapters and records where cloud-specific behavior is intentional.

Scaling, Placement, and Reliability

Kubernetes scaling occurs at several levels. Horizontal Pod Autoscaler (HPA) changes replica count using CPU, memory, or custom metrics. Vertical Pod Autoscaler can recommend or adjust resource requests in suitable operating modes. Cluster autoscalers add or remove nodes when Pods cannot be scheduled or nodes are underutilized. Managed services may also provide provider-specific node auto-provisioning.

Autoscaling is effective only when application metrics, resource requests, startup times, and downstream capacity are understood. An HPA based solely on CPU may fail for an I/O-bound service. Scaling a web tier without scaling its database may increase queueing and failure. Stabilization windows, maximum replica limits, cooldown behavior, and load tests should be treated as part of the design rather than left at defaults.

Placement controls improve resilience. Node selectors and affinities place workloads on appropriate hardware or zones; taints and tolerations reserve nodes for selected workloads; topology spread constraints distribute replicas across failure domains. PodDisruptionBudgets limit voluntary disruption, but they cannot guarantee availability during a total zone or regional failure. High availability also requires multiple replicas, stateless design where possible, graceful termination, timeouts, retries with backoff, and dependency-aware failure handling.

Networking, Discovery, and Persistence

Within a cluster, each Pod ordinarily receives an IP address, while Services provide stable discovery. Cluster DNS resolves a Service name to its virtual address. This model avoids hard-coding Pod addresses and allows replicas to be replaced transparently. NetworkPolicies restrict which Pods may communicate; they are valuable because a default open network can turn a compromised workload into a pathway to other services.

Ingress and gateway design should address TLS termination, certificate renewal, authentication, rate limiting, request size, timeouts, and access logging. Public endpoints should normally be protected by web application firewalls, DDoS controls, identity-aware gateways, or equivalent edge services. Internal services should not be exposed publicly merely because doing so is convenient.

Containers are ephemeral. Data written to a container’s writable layer may disappear when the container is replaced. PersistentVolumeClaims express a request for durable storage, while a StorageClass defines how storage is provisioned. Persistent volumes may use block disks, network file systems, or cloud-native storage services. Stateful applications require careful decisions about replication, backup, consistency, failover, and recovery objectives. For many databases, a managed database service is operationally safer than running the database inside Kubernetes, although regulatory, latency, or portability requirements may justify the latter.

Serverless Functions and Container-Based Serverless

Serverless does not mean that servers do not exist. It means that the provider manages much of the provisioning, capacity, and infrastructure operation, while the customer is charged or managed primarily according to invocation, execution, or allocated resources. Functions are typically short-lived units triggered by HTTP requests, queues, object events, schedules, or messaging systems.

AWS Lambda, Google Cloud Functions, and Azure Functions provide function-oriented programming models with provider runtimes, event integrations, identity controls, and monitoring. Cloud Run offers a container-based serverless model in Google Cloud, while AWS Fargate and Azure Container Apps provide related managed container abstractions. These models are useful when teams need container packaging without managing Kubernetes nodes or when workload demand is intermittent.

Function design must account for cold starts, concurrency, execution duration, memory limits, temporary storage, retry behavior, duplicate delivery, and downstream throttling. Event-driven systems are commonly at-least-once delivery systems, so handlers should be idempotent. A function that charges a customer, sends an email, or updates inventory must tolerate retries without producing an unintended duplicate effect. Durable workflows, queues, dead-letter destinations, and correlation identifiers improve reliability.

Serverless is not automatically cheaper or simpler. High, steady utilization may be less expensive on reserved or autoscaled containers. Complex distributed workflows can become difficult to debug when functions, queues, permissions, and retries are spread across services. Vendor-specific triggers can also create portability costs. The appropriate choice depends on traffic shape, latency requirements, team capabilities, governance, and the value of infrastructure abstraction.

A Decision Framework for Workload Placement

A practical placement decision can be based on five questions. First, is the workload long-running, stateful, event-driven, or batch-oriented? Second, how much control is needed over networking, scheduling, runtime, and operating system behavior? Third, is demand predictable or highly variable? Fourth, what are the latency, compliance, and data-residency requirements? Fifth, what operational capability does the organization possess?

Kubernetes is appropriate where an organization needs a common platform for many services, sophisticated scheduling, custom operators, hybrid deployment, or consistent operational controls. It imposes a substantial platform-engineering burden. Functions are effective for discrete event handlers, automation, integration logic, and irregular workloads, but require disciplined event and observability design. Managed container serverless is useful when the application can be containerized but the team does not need Kubernetes-level control.

This framework should be accompanied by total-cost analysis. Costs include compute, storage, network transfer, control-plane or platform fees, observability, security tooling, engineering time, incident response, and migration risk. A lower infrastructure invoice can be outweighed by higher operational complexity.

Delivery, Security, and Governance

A mature delivery pipeline builds and tests images, scans dependencies, signs artifacts where appropriate, pushes them to an approved registry, and deploys immutable version references. GitOps tools can reconcile cluster state from a version-controlled repository, creating an auditable path from change to deployment. Progressive delivery techniques such as canary releases and blue-green deployment reduce blast radius, but require meaningful metrics and automated rollback criteria.

Security begins before scheduling. Images should be sourced from trusted bases and rebuilt regularly. Admission policies can reject privileged containers, host-path mounts, unsigned images, missing resource requests, or disallowed registries. Workloads should use least-privilege service accounts, network segmentation, encrypted transport, and short-lived credentials. Cluster administrators and application developers should not automatically share unrestricted access.

Governance includes ownership, cost allocation, retention, audit logging, data classification, incident response, and recovery testing. Kubernetes adds an extensive API surface, so audit logs and policy-as-code are especially important. Serverless systems require equivalent attention to function permissions, event-source authorization, secrets, log retention, and cross-service data access.

Illustrative Case Study: Portable Order Processing

Background

A fictional retailer, Northstar Commerce, operates an online store in three regions. Its web application is composed of an API, catalog service, order service, payment integration, and asynchronous notification worker. The company wants to reduce deployment time and support both AWS and Google Cloud for resilience and negotiation flexibility.

Situation and Problem

The original application runs on manually configured virtual machines. Releases are slow, scaling is inconsistent, and a failed machine requires manual intervention. The team initially proposes moving every component to Kubernetes, including the database and all scheduled jobs.

Analysis

The architecture group identifies different workload characteristics. The API and catalog are stateless and experience variable traffic. Order processing requires durable records and controlled retries. Notifications are event-driven and can tolerate small delays. The relational database needs strong consistency and tested backup recovery. Running it inside Kubernetes would increase the operational surface without delivering a clear business benefit.

Approach and Solution

The group builds reproducible container images and stores them in ECR and Artifact Registry. Kubernetes-standard Deployments and Services describe the API and catalog, while provider-specific overlays define load balancers, identity bindings, storage classes, and monitoring. HPA scales the API using request-rate and latency metrics, and topology constraints distribute replicas across zones. An ingress layer terminates TLS and applies rate limits.

The database moves to a managed relational service with cross-region backup procedures. The order service writes an order record and publishes an event through a durable messaging system. A serverless function processes notification events and records an idempotency key before sending a message. Failed events move to a dead-letter destination for investigation. Secrets are retrieved from the cloud secret manager rather than stored in images or ordinary configuration files.

The delivery pipeline performs unit, integration, image, and policy checks. It deploys first to a staging cluster, then uses a canary release with automated rollback based on error rate and latency. Dashboards correlate ingress requests, Kubernetes workload metrics, message age, database performance, and function failures.

Outcome

Deployment becomes repeatable, scaling is separated by workload behavior, and the most stateful component is operated through a managed database service. The application remains portable at the application-manifest level, while provider-specific integrations are explicitly documented rather than hidden. The retailer accepts that complete cloud independence is unrealistic but reduces avoidable coupling.

Learning Points

The case demonstrates that portability is an architectural choice, not a property automatically supplied by Kubernetes. It also shows why a mixed model is often superior to a single platform: Kubernetes serves continuously running services, messaging decouples order processing, serverless handles irregular notification work, and a managed database reduces stateful operations.

Challenges and Common Mistakes

Several recurring mistakes undermine container and Kubernetes initiatives. Treating containers as virtual machines leads to oversized images, embedded SSH servers, and mutable in-container configuration. Treating Kubernetes as a complete platform without ownership, policy, and observability produces uncontrolled complexity. Omitting resource requests causes poor scheduling and unreliable autoscaling. Exposing the API server, dashboards, or internal services publicly creates avoidable security risk.

Other problems include using liveness probes as readiness checks, retry storms, unbounded queues, insufficient Pod disruption budgets, ignoring image provenance, and assuming that a successful deployment proves application health. Teams also frequently run databases in Kubernetes without capacity, backup, and restore expertise. In serverless systems, duplicate event processing, hidden retry behavior, excessive function granularity, and poor trace correlation create operational confusion.

Portability itself has a cost. Avoiding every provider-specific service may sacrifice reliability or productivity, whereas embracing every proprietary feature may make migration expensive. The sound approach is to classify dependencies, isolate them behind interfaces where useful, and make deliberate rather than accidental choices.

Current Developments and Professional Practice

Managed Kubernetes is increasingly automated through node auto-provisioning, managed add-ons, policy engines, and integrated security services. Gateway APIs provide a more expressive and role-oriented evolution of traditional ingress models. eBPF-based networking and observability can improve visibility without application changes, although they introduce specialized operational requirements.

Serverless platforms increasingly support container images, GPUs, longer-running workloads, event workflows, and scale-to-zero behavior. Platform teams are adopting internal developer platforms that provide approved templates, golden paths, identity integration, policy checks, and standardized telemetry. Software supply-chain security is also becoming more systematic through provenance attestations, SBOMs, signing, and verification during deployment.

AI workloads add further complexity. Inference services may require GPUs, model artifacts, specialized autoscaling signals, and strict latency controls. Batch model processing may fit Kubernetes jobs or managed batch services, while event-triggered enrichment may fit functions. The same placement framework applies: understand workload behavior, control requirements, data sensitivity, and total operational cost.

Best-Practice Principles

A professional operating model generally follows these principles:

• Build immutable, minimal, reproducible images and keep credentials outside them.
• Treat Kubernetes configuration as versioned, reviewed, testable code.
• Define requests, limits, probes, disruption policy, identity, and network policy explicitly.
• Separate standard Kubernetes resources from provider-specific integrations.
• Prefer managed stateful services when they meet requirements and reduce operational risk.
• Design event handlers to be idempotent and make retries, dead letters, and timeouts visible.
• Use least privilege, artifact verification, admission policy, and centralized audit records.
• Measure reliability, deployment frequency, recovery time, cost, and resource efficiency together.
• Test failure, rollback, backup restoration, scaling, and regional recovery rather than assuming them.
• Select Kubernetes, managed containers, or functions according to workload characteristics, not fashion.

Chapter Summary

Containers package applications as reproducible, portable images, but their security and reliability depend on image quality, runtime controls, and operational discipline. Kubernetes provides declarative orchestration through Pods, controllers, Services, ingress or gateway resources, configuration objects, policies, and persistent-volume abstractions. EKS, GKE, and AKS reduce control-plane administration while retaining important responsibilities for identity, networking, workload security, scaling, and governance.

Autoscaling, service discovery, ingress, persistence, and placement must be designed together. Serverless functions and container-based serverless platforms reduce infrastructure management and suit event-driven, intermittent, or operationally focused workloads, but introduce concerns around retries, cold starts, observability, limits, and vendor coupling. Effective platform decisions weigh reliability, control, cost, portability, security, and team capability. The most robust architectures often combine these models rather than forcing every workload into one platform.

Key Terms

Container image: An immutable, layered package containing an application, dependencies, metadata, and startup instructions.

Container registry: A service that stores, scans, and distributes container images.

Pod: Kubernetes’ smallest schedulable unit, containing one or more co-located containers.

Deployment: A Kubernetes controller that manages replicated, usually stateless, Pods and rolling updates.

StatefulSet: A controller for workloads requiring stable identities, ordered behavior, or persistent-volume associations.

Service: A stable virtual endpoint that selects and exposes a group of Pods.

Ingress: A resource and routing model for directing external HTTP or HTTPS traffic to Services.

Gateway API: A Kubernetes networking model providing more expressive and role-oriented traffic management.

ConfigMap: A Kubernetes object for non-sensitive configuration data.

Secret: An object intended for sensitive configuration, requiring appropriate encryption and access controls.

Readiness probe: A health test determining whether a Pod should receive traffic.

Liveness probe: A health test indicating whether a container should be restarted.

Horizontal Pod Autoscaler: A controller that adjusts Pod replica count using resource or custom metrics.

PersistentVolumeClaim: A request for durable storage made by a Kubernetes workload.

NetworkPolicy: A rule set controlling permitted network communication between workloads or endpoints.

Serverless computing: A model in which the provider manages much of infrastructure capacity and provisioning for application execution.

Cold start: Initialization delay incurred when a serverless execution environment must be created or loaded.

Idempotency: The property that repeating an operation produces the same intended result as performing it once.

GitOps: An operating approach in which version-controlled declarations drive automated environment reconciliation.

Software bill of materials: An inventory of components and dependencies contained in a software artifact.

Review Questions

Short Answer Questions

1. Why is a Kubernetes Service needed when Pods already have IP addresses?

2. Distinguish a readiness probe from a liveness probe.

3. What is the purpose of a PersistentVolumeClaim?

4. Why should serverless event handlers be idempotent?

5. Name two important portability limitations across EKS, GKE, and AKS.

Long Answer Questions

1. Explain Kubernetes as a reconciliation system and relate the API server, scheduler, controllers, Pods, and Deployments to this model.

2. Compare Kubernetes, managed container serverless platforms, and functions for long-running, event-driven, and batch workloads.

3. Discuss how resource requests, limits, probes, HPA, cluster autoscaling, and topology constraints contribute to reliability.

4. Describe a secure delivery pipeline for containerized applications, including image construction, registry controls, policy, identity, and deployment.

5. Analyze the operational and architectural trade-offs of running a database inside Kubernetes instead of using a managed database service.

Analytical / Application Questions

1. A service has low CPU utilization but increasing request latency and queue depth. Explain why CPU-only HPA may fail and propose a better scaling approach.

2. Design a deployment strategy for a notification system that may receive duplicate events and experience downstream outages. Include retries, idempotency, dead-letter handling, and observability.

3. An organization demands multi-cloud portability but wants to use provider-specific load balancers, secret managers, and storage. Propose an architecture and governance approach that balances portability with provider integration.

22. Multiple Choice Questions with Answers

Question 1

Which Kubernetes object provides a stable endpoint for a changing set of Pods?

A. Service

B. ConfigMap

C. Job

D. DaemonSet

Correct Answer: A. Service

Question 2

A workload is I/O-bound, but its Horizontal Pod Autoscaler uses only CPU utilization. What is the most likely problem?

A. Pods will necessarily become privileged

B. Scaling may not occur when the real workload pressure increases

C. The cluster API server will stop accepting requests

D. Persistent volumes will be deleted automatically

Correct Answer: B. Scaling may not occur when the real workload pressure increases

Question 3

Which statement best describes a Kubernetes readiness probe?

A. It determines whether a container image is trusted

B. It determines whether a Pod should receive traffic

C. It permanently restarts every Pod on a schedule

D. It provisions a cloud load balancer

Correct Answer: B. It determines whether a Pod should receive traffic

Question 4

Why should a serverless event handler use an idempotency key?

A. Event systems may deliver the same event more than once

B. It reduces the size of the container image

C. It guarantees zero cold starts

D. It replaces the need for authentication

Correct Answer: A. Event systems may deliver the same event more than once

Question 5

Which statement most accurately describes portability among EKS, GKE, and AKS?

A. All provider integrations behave identically because Kubernetes is standardized

B. Portability is impossible because Kubernetes APIs differ completely

C. Core resources are portable, but networking, storage, identity, and monitoring integrations may be provider-specific

D. Only serverless functions can be portable

Correct Answer: C. Core resources are portable, but networking, storage, identity, and monitoring integrations may be provider-specific

Question 6

A Kubernetes application loses files whenever its Pod is replaced. What is the most appropriate design response for durable data?

A. Store the data in the container writable layer

B. Use a PersistentVolumeClaim or an appropriate managed storage service

C. Increase the Pod’s CPU limit

D. Replace the Deployment with a ConfigMap

Correct Answer: B. Use a PersistentVolumeClaim or an appropriate managed storage service

Question 7

What is the primary operational value of a Deployment?

A. It manages replicated Pods and supports controlled updates

B. It stores encrypted secrets automatically

C. It replaces the Kubernetes control plane

D. It routes all external traffic without a controller

Correct Answer: A. It manages replicated Pods and supports controlled updates

Question 8

Which practice most directly reduces the risk of an untrusted container artifact reaching production?

A. Using larger base images

B. Disabling admission checks

C. Artifact scanning, signing or provenance verification, and deployment policy

D. Embedding cloud credentials in the image

Correct Answer: C. Artifact scanning, signing or provenance verification, and deployment policy

Question 9

Northstar’s notification workload is irregular and event-triggered, while its API is continuously running. Which placement is most defensible?

A. Run both exclusively as database processes

B. Use a serverless function for notifications and a continuously running managed workload for the API

C. Run both only as CronJobs

D. Store both workloads in ConfigMaps

Correct Answer: B. Use a serverless function for notifications and a continuously running managed workload for the API

Question 10

What is a major risk of using a liveness probe as if it were a readiness probe?

A. Healthy but temporarily unready containers may be repeatedly restarted

B. The Service will gain a second IP address

C. The image registry will lose its audit log

D. The cluster will automatically become multi-region

Correct Answer: A. Healthy but temporarily unready containers may be repeatedly restarted