7. Containers, Kubernetes and Cloud-Native Architecture

Introduction and Context

Cloud deployment has moved beyond the simple practice of placing an application on a virtual machine. Modern systems are expected to be portable, rapidly deployable, resilient, observable, and capable of responding to fluctuating demand. Containers and Kubernetes address these requirements by packaging applications consistently and coordinating their execution across distributed infrastructure.

A container is a process-isolation mechanism that packages an application with its runtime dependencies while sharing the host operating system kernel. Kubernetes is an orchestration platform that schedules and manages containerized workloads. Managed Kubernetes services—Amazon Elastic Kubernetes Service (EKS), Google Kubernetes Engine (GKE), and Azure Kubernetes Service (AKS)—reduce the operational burden of managing the Kubernetes control plane while retaining substantial architectural flexibility.

Cloud-native architecture extends beyond the use of containers. It includes independently deployable services, declarative infrastructure, automation, elastic scaling, resilience engineering, observability, and a culture of continuous delivery. The central architectural question is not merely how to run containers, but how to design, secure, operate, and govern distributed applications effectively.

Learning Objectives

After studying this chapter, learners should be able to:

1. Explain container isolation, images, registries, and container execution.

2. Build secure, efficient, and reproducible container images.

3. Describe Kubernetes objects, control-plane functions, scheduling, and reconciliation.

4. Compare EKS, GKE, and AKS from an operational and architectural perspective.

5. Apply Kubernetes techniques for service discovery, ingress, configuration, secrets, and scaling.

6. Explain the role of service meshes in traffic management, security, and observability.

7. Evaluate cloud-native designs in relation to security, cost, reliability, governance, and organizational capability.

From Virtual Machines to Cloud-Native Workloads

Virtual machines emulate complete computers and normally include a guest operating system. They provide strong isolation but consume more storage and memory than containers. Containers isolate processes using operating-system features such as namespaces and control groups. They therefore start quickly and permit higher workload density, although their isolation boundary is generally different from that of a virtual machine.

A container image is an immutable, layered package containing an application, its dependencies, metadata, and a startup command. A running container is an instance of an image with a writable execution layer. Images are stored in registries such as Amazon Elastic Container Registry (ECR), Google Artifact Registry, and Azure Container Registry (ACR).

Containerization improves consistency between development, testing, and production. It does not, however, automatically make an application cloud-native. A monolithic application can be placed in a container without becoming modular, resilient, or independently scalable. Cloud-native design is an architectural and operational approach rather than a packaging format.

Important terminology includes the following:

• Image: An immutable artifact used to create containers.
• Container runtime: Software that downloads images and executes containers, commonly containerd or CRI-O in Kubernetes environments.
• Registry: A service that stores and distributes images.
• Pod: The smallest deployable unit in Kubernetes, containing one or more closely coupled containers.
• Cluster: A collection of machines and Kubernetes control-plane services.
• Node: A worker machine on which pods run.
• Namespace: A logical boundary for grouping and governing Kubernetes resources.
• Declarative configuration: A specification of the desired state rather than a sequence of imperative commands.
• Reconciliation: The continuous process by which controllers move actual state toward desired state.

Building Secure and Efficient Images

A Dockerfile or equivalent build definition describes how an image is constructed. A typical build copies application files, installs dependencies, defines environment settings, exposes a port, and specifies a command. The build should be deterministic: dependencies should be explicitly versioned, and the same source and build inputs should produce materially equivalent artifacts.

A production image should contain only what is required to run the application. Multi-stage builds are particularly useful. A builder stage can contain compilers and development tools, while a final runtime stage contains only the compiled artifact and its runtime dependencies. This reduces image size and attack surface.

Security begins during the build rather than after deployment. Effective practices include:

• Use trusted and preferably minimal base images.
• Pin base-image versions and update them through a controlled process.
• Run the application as a non-root user.
• Do not place passwords, tokens, private keys, or cloud credentials in images.
• Scan images for known vulnerabilities and investigate severity, exploitability, and reachability.
• Generate a software bill of materials (SBOM) where organizational policy requires supply-chain visibility.
• Sign images and verify signatures before deployment when a suitable trust system is available.
• Use read-only filesystems and drop unnecessary Linux capabilities where compatible with the application.
• Define resource requests and limits at deployment time.

A vulnerability scanner does not by itself establish that an image is safe. A reported package may be unreachable, while an apparently moderate issue may become serious in a publicly exposed service. Image security therefore requires technical scanning, patch management, provenance controls, runtime restrictions, and incident response.

The image lifecycle commonly follows this sequence: source commit, automated build, unit and integration tests, image scanning, signing or attestation, publication to a registry, deployment to a non-production environment, verification, and controlled promotion. The same immutable image should preferably be promoted between environments rather than rebuilt separately for each environment.

Kubernetes as a Reconciliation System

Kubernetes represents desired state through an API. For example, a Deployment may specify that three replicas of a particular image should be running. Controllers observe the cluster and create or replace ReplicaSets and pods until the observed state approximates the declared state. This reconciliation model supports self-healing, rolling updates, and declarative operations.

The Kubernetes control plane includes the API server, a persistent state store such as etcd, the scheduler, and controllers. The API server is the principal interface for clients and internal components. The scheduler selects suitable nodes for unscheduled pods by considering resource requests, constraints, affinity, taints, and tolerations. Controllers implement behavior for Deployments, Jobs, Services, Nodes, and other resources. Worker nodes run the kubelet, a container runtime, and networking components.

A Pod should not be treated as a lightweight virtual machine. Containers in the same pod share a network namespace and can communicate through localhost. They may also share storage volumes. A common pod contains one main application container and, in carefully justified cases, a sidecar responsible for functions such as proxying or log processing.

A Deployment manages stateless replicas and supports rolling updates and rollbacks. A StatefulSet is intended for workloads requiring stable identities or ordered behavior, although databases should not automatically be placed inside Kubernetes merely because StatefulSets exist. A DaemonSet schedules a pod on selected nodes, often for node-level agents. A Job runs work to completion, while a CronJob creates Jobs on a schedule.

Kubernetes networking separates pod identity from individual pod instances. Pods are replaceable and their IP addresses can change. A Service provides a stable virtual endpoint and selects pods using labels. Common service types include ClusterIP for internal access, NodePort for exposing a port on nodes, and LoadBalancer for integration with a cloud load balancer. In production, external HTTP or HTTPS traffic is normally handled through an Ingress or the newer Gateway API, together with an ingress controller or gateway implementation.

Health checks are essential to reliable orchestration. A liveness probe determines whether a container should be restarted. A readiness probe determines whether it should receive traffic. A startup probe protects slow-starting applications from premature liveness failures. Poorly designed probes can create restart loops or route traffic to an application that is technically running but not ready to serve requests.

Configuration should be separated from the image. ConfigMaps hold non-sensitive configuration, while Secrets are intended for sensitive values. A Kubernetes Secret is not automatically equivalent to a fully protected enterprise secrets-management system; access control, encryption at rest, auditing, rotation, and integration with services such as AWS Secrets Manager, Google Secret Manager, or Azure Key Vault must be considered.

Managed Kubernetes: EKS, GKE and AKS

Managed Kubernetes services operate the control plane on behalf of the customer, but they do not eliminate responsibility for the workloads. The customer normally remains responsible for cluster configuration, node pools or compute profiles, identities, network policies, images, applications, data, and much of the security posture.

Amazon EKS integrates with AWS identity and access management, VPC networking, load balancers, storage services, and compute options such as managed node groups, self-managed nodes, and serverless pod execution through Fargate. It is appropriate when an organization already relies heavily on AWS infrastructure and governance.

Google Kubernetes Engine integrates closely with Google Cloud networking, IAM, Artifact Registry, logging, monitoring, and automated cluster-management features. Its operational model is often attractive to organizations seeking strong integration with Google’s data, analytics, and platform services.

Azure Kubernetes Service integrates with Microsoft Entra ID, Azure networking, Azure Monitor, managed identities, and Azure Container Registry. It is commonly selected by organizations with substantial Microsoft identity, governance, or enterprise application investments.

The choice should not be made solely on brand familiarity. Relevant criteria include regional availability, identity integration, networking behavior, storage options, upgrade policies, security controls, compliance requirements, team expertise, support model, workload portability, and total cost. Kubernetes manifests can be portable while load balancer behavior, identity models, storage classes, ingress controllers, and observability systems remain provider-specific.

A managed control plane also introduces governance questions. Organizations should define who may create clusters, which versions are supported, how upgrades are tested, how privileged access is approved, and how costs are allocated. A platform team may provide a standardized internal Kubernetes platform so that application teams consume secure templates rather than designing every cluster independently.

Scaling, Reliability and Traffic Management

Kubernetes scaling operates at several levels. A Horizontal Pod Autoscaler (HPA) adjusts replica counts according to metrics such as CPU utilization, memory, or application-specific indicators. A Vertical Pod Autoscaler can recommend or adjust resource requests in supported operating modes. Cluster autoscaling adds or removes worker capacity when pods cannot be scheduled or nodes are underutilized. In some environments, event-driven tools scale workloads based on queue depth or external events.

Scaling is not equivalent to resilience. An application may have many replicas and still fail if all replicas are placed in one availability zone, share an overloaded database, or depend on a single external service. Reliability requires appropriate replica distribution, graceful shutdown, timeouts, retries with backoff, circuit breaking, idempotent operations, capacity planning, and tested recovery procedures.

Resource requests influence scheduling; limits constrain consumption. Requests set the amount of capacity Kubernetes attempts to reserve, while limits define an upper boundary. Incorrectly low requests cause contention and scheduling problems; excessively high requests waste capacity and distort autoscaling. Performance testing should inform these values.

An Ingress routes external requests based on hostnames and paths. TLS termination, authentication, rate limiting, web application firewalls, and certificate management may be implemented at the gateway or through integrated cloud services. DNS, certificates, load balancers, and application routing must be designed together, particularly when multiple environments or regions are involved.

Service Meshes and Distributed Application Behavior

A service mesh provides a dedicated communication layer for service-to-service traffic. It commonly uses sidecar or node-level proxies and a control plane that distributes policy and configuration. Mesh capabilities may include mutual TLS, identity-based authorization, traffic shifting, retries, timeouts, circuit breaking, telemetry, and canary releases.

Mutual TLS authenticates workloads to one another and encrypts traffic in transit. This can support zero-trust principles, but it does not replace application authorization. A service may be cryptographically authenticated and still be forbidden from performing a particular operation. Policies must therefore express business and security boundaries as well as transport protection.

A mesh adds operational complexity, resource consumption, and another control plane. It can also make failures harder to diagnose when application behavior, proxy behavior, certificates, and policy interact. Teams should adopt a mesh when its capabilities address clear requirements, not as a universal indicator of maturity. Native cloud load-balancing, application libraries, or gateway features may be sufficient for smaller systems.

Cloud-Native Architecture in Practice

Cloud-native systems are commonly composed of independently deployable services, asynchronous messaging, managed data services, infrastructure as code, continuous integration and delivery, centralized observability, and automated policy enforcement. These patterns are useful when they match organizational and domain needs. Splitting a small, cohesive application into many microservices can increase network failure modes, testing complexity, operational cost, and data-consistency challenges.

A useful design approach begins with business capabilities and operational boundaries. Each service should have a clear responsibility, an explicit API, independently managed deployment, and an ownership model. Distributed transactions should be minimized. Events and queues can decouple producers and consumers, but they introduce concerns such as ordering, duplication, dead-letter handling, and eventual consistency.

Observability combines logs, metrics, and traces. Logs should be structured and correlated with request or trace identifiers. Metrics should measure both infrastructure and user-visible behavior, including latency, error rate, saturation, and throughput. Distributed tracing helps identify where a request spends time across services. Observability data should be governed because it can contain personal, confidential, or security-sensitive information.

Infrastructure and policies should be managed as code where practical. GitOps extends this approach by treating a version-controlled repository as the authoritative source for desired deployment state. A controller then applies approved changes to the cluster. GitOps improves auditability and rollback, but repository access, secret handling, emergency changes, and drift management still require explicit procedures.

Illustrative Case Study: A Retail Checkout Platform

Background

A fictional online retailer operates a monolithic checkout application on virtual machines. Releases occur monthly, traffic peaks during promotional events, and failures in payment or inventory integrations are difficult to isolate. The retailer chooses GKE for a gradual modernization initiative.

Problem

The application experiences long deployment windows and cannot scale only the order-processing component. During peak periods, increased traffic causes resource contention. Sensitive payment-related configuration is stored in deployment scripts, and operational teams have limited visibility into request latency.

Analysis

A complete rewrite would create substantial business risk. The team therefore identifies order validation, payment coordination, and notification as candidate boundaries while retaining the existing checkout interface. It also determines that payment calls require strict timeouts, idempotency, audit records, and controlled retry behavior. The database remains a managed service rather than being moved into the initial Kubernetes cluster.

Strategy and Solution

The team creates a multi-stage image and runs the application as a non-root user. Images are scanned and stored in Artifact Registry. Kubernetes Deployments manage stateless components, Services provide stable internal endpoints, and an external gateway manages TLS and routing. ConfigMaps contain non-sensitive settings; secrets are retrieved through a managed secrets service using workload identity rather than embedded credentials.

Readiness and startup probes prevent premature traffic. Resource requests are established through load testing, and an HPA scales order-processing replicas based on CPU and queue depth. Cluster nodes are distributed across availability zones. Network policies restrict which services may communicate. Structured logs, metrics, and traces are exported to the cloud observability platform.

A service mesh is not introduced initially. The team first implements application-level timeouts, idempotency keys, and gateway controls. Later, a limited mesh deployment is evaluated for mutual TLS and progressive traffic shifting between checkout versions.

Outcome

Deployments become repeatable and can be rolled back by changing the declared image version. Order processing scales independently, while the managed database remains protected from uncontrolled connection growth through pooling and limits. The organization also establishes image-retention rules, cluster upgrade procedures, on-call ownership, and access reviews.

Learning Points

The case demonstrates that successful modernization combines packaging, architecture, security, operations, and organizational ownership. Kubernetes solves scheduling and reconciliation problems, but it does not automatically solve poor service boundaries, database bottlenecks, insecure credentials, or inadequate testing.

Risks, Limitations and Common Mistakes

Kubernetes can become an expensive abstraction when a simpler managed container service is adequate. It requires expertise in networking, identity, storage, upgrades, observability, and incident response. Excessive customization can create dependence on undocumented platform behavior.

Common mistakes include using the latest untested image tag, running containers as root, storing secrets in source control, omitting resource requests, exposing internal services publicly, relying only on liveness probes, granting broad cluster-admin permissions, and treating a namespace as a complete security boundary. Other errors include placing stateful workloads in the cluster without backup and recovery testing, adding retries without limits, and adopting a service mesh before understanding baseline traffic behavior.

Security governance should include least privilege, image provenance, vulnerability response, audit logging, network segmentation, encryption, backup, retention, and separation of duties. Regulatory obligations may affect data location, access records, payment processing, personal data handling, and incident reporting. Container security is a shared responsibility: the cloud provider secures relevant managed infrastructure, while the organization secures configuration, identities, images, workloads, and data.

Current Developments

Kubernetes usage is increasingly complemented by the Gateway API, workload identity mechanisms, policy-as-code, confidential-computing options, and managed serverless container execution. Platform engineering teams are creating standardized “golden paths” that combine approved base images, deployment templates, observability, security policies, and cost controls.

The wider ecosystem is also moving toward software-supply-chain transparency through signed artifacts, provenance attestations, SBOMs, and admission policies. Artificial intelligence can assist with anomaly detection, capacity forecasting, log analysis, and operational recommendations, but its outputs require human validation. Automated remediation is appropriate only when the consequences are understood and rollback is possible.

Best-Practice Framework

A practical cloud-native operating model should:

1. Define ownership for clusters, namespaces, workloads, data, and incidents.

2. Standardize secure base images and continuously scan dependencies.

3. Prefer declarative configuration and version-controlled promotion.

4. Apply least privilege to humans, workloads, and automation.

5. Separate configuration and secrets from images.

6. Set resource requests, limits, probes, disruption policies, and scaling rules.

7. Design for zone failure, graceful termination, and dependency failure.

8. Instrument services before introducing complex operational tooling.

9. Test upgrades, backups, rollbacks, and disaster recovery.

10. Measure cost, reliability, delivery performance, and security outcomes.

Chapter Summary

Containers package applications consistently, but cloud-native architecture requires more than containerization. Secure image construction involves minimal and trusted bases, non-root execution, vulnerability management, provenance, and controlled promotion. Kubernetes provides declarative APIs, scheduling, controllers, service discovery, health management, and reconciliation. EKS, GKE, and AKS offer managed control planes with different integrations and operational choices; responsibility for workloads and governance remains with the customer.

Services, ingress, configuration objects, secrets, autoscaling, resource management, and observability form the operational foundation of Kubernetes deployments. Service meshes can add identity, encrypted service communication, traffic control, and telemetry, but they also introduce complexity. Effective cloud-native engineering balances portability, reliability, security, cost, organizational capability, and business requirements.

Key Terms

Container: An isolated process running from an image while sharing the host kernel.

Container image: An immutable, layered artifact containing application code and runtime dependencies.

Registry: A repository that stores and distributes container images.

Kubernetes: A platform for declaratively orchestrating containerized workloads.

Pod: Kubernetes’ smallest deployable unit, containing one or more containers.

Deployment: A Kubernetes resource that manages replicated, usually stateless, application pods.

Service: A stable network endpoint that selects and exposes pods.

Ingress: A mechanism for routing external HTTP or HTTPS traffic to services.

ConfigMap: A Kubernetes object for non-sensitive configuration data.

Secret: An object intended to hold sensitive configuration, subject to additional protection.

Control plane: Kubernetes components that expose the API and manage cluster state.

Reconciliation: The process of converging actual system state toward desired state.

Horizontal Pod Autoscaler: A controller that adjusts pod replicas based on metrics.

Service mesh: A communication layer providing traffic, security, and observability capabilities between services.

Mutual TLS: Two-way certificate-based authentication and encryption between communicating parties.

GitOps: An operating approach in which version-controlled desired state drives deployment.

SBOM: A software bill of materials listing components and dependencies in an artifact.

Workload identity: A mechanism allowing workloads to access cloud services without embedded long-lived credentials.

Review Questions

Short Answer Questions

1. Distinguish a container image from a running container.

2. What is the purpose of a Kubernetes Service?

3. Differentiate readiness, liveness, and startup probes.

4. Why are multi-stage image builds useful?

5. What responsibility remains with a customer when using a managed Kubernetes service?

Long Answer Questions

1. Explain Kubernetes’ reconciliation model and show how Deployments, ReplicaSets, and pods cooperate.

2. Compare EKS, GKE, and AKS using identity, networking, storage, operations, and governance criteria.

3. Describe a secure container-image lifecycle from source commit to production deployment.

4. Explain Kubernetes configuration, secrets, ingress, service discovery, and autoscaling as parts of one deployment design.

5. Discuss the benefits and costs of introducing a service mesh into a microservices platform.

Analytical / Application Questions

1. A workload has three replicas, but all are scheduled in one availability zone. Analyze the reliability risk and propose Kubernetes and infrastructure changes.

2. An organization’s pods restart frequently after a new release, although the application eventually becomes healthy. Diagnose possible probe and resource-configuration errors and recommend a testing approach.

3. A company is considering Kubernetes for a small internal application with predictable traffic. Develop a decision framework comparing managed Kubernetes with a simpler managed container service.

22. Multiple Choice Questions with Answers

Question 1

Which practice most directly reduces the attack surface of a production container image?

A. Including compilers and debugging tools in the final image

B. Using a multi-stage build with a minimal runtime stage

C. Storing credentials in environment variables inside the image

D. Running the application as the root user

Correct Answer: B. Using a multi-stage build with a minimal runtime stage

Question 2

What is the primary purpose of a Kubernetes Service?

A. To build container images

B. To provide a stable endpoint for selected pods

C. To store cluster state permanently

D. To compile application source code

Correct Answer: B. To provide a stable endpoint for selected pods

Question 3

A readiness probe fails while a container process remains alive. What should Kubernetes normally do?

A. Restart the container immediately

B. Delete the entire cluster

C. Stop directing normal traffic to the pod

D. Increase the container image size

Correct Answer: C. Stop directing normal traffic to the pod

Question 4

Which Kubernetes component selects suitable nodes for unscheduled pods?

A. Scheduler

B. Registry

C. Ingress controller

D. Kubelet only

Correct Answer: A. Scheduler

Question 5

Which statement best describes a managed Kubernetes service?

A. The provider assumes responsibility for application secrets and code

B. The provider operates relevant control-plane infrastructure, while customers still govern workloads and configuration

C. The service eliminates the need for identity management

D. The service guarantees that all Kubernetes resources are portable across clouds

Correct Answer: B. The provider operates relevant control-plane infrastructure, while customers still govern workloads and configuration

Question 6

A team wants replicas to increase when queue depth rises. Which approach is most appropriate?

A. Event- or metric-driven autoscaling using an application-relevant metric

B. Removing all resource requests

C. Replacing the Service with a ConfigMap

D. Disabling health probes

Correct Answer: A. Event- or metric-driven autoscaling using an application-relevant metric

Question 7

Why should credentials generally not be embedded in a container image?

A. Images cannot contain text files

B. Images are immutable and therefore credentials cannot be used

C. Credentials may persist in image layers and be exposed to anyone with image access

D. Kubernetes automatically deletes all credentials from images

Correct Answer: C. Credentials may persist in image layers and be exposed to anyone with image access

Question 8

What is a principal benefit of a service mesh?

A. It replaces the need for application design

B. It can provide workload identity, mutual TLS, traffic policy, and service telemetry

C. It guarantees that databases never fail

D. It eliminates all network latency

Correct Answer: B. It can provide workload identity, mutual TLS, traffic policy, and service telemetry

Question 9

Why are resource requests important in Kubernetes?

A. They help the scheduler determine suitable placement and influence capacity planning

B. They encrypt all pod traffic

C. They define the application’s business permissions

D. They replace container images

Correct Answer: A. They help the scheduler determine suitable placement and influence capacity planning

Question 10

In the illustrative retail case, why was a service mesh not introduced immediately?

A. Kubernetes cannot run service meshes

B. The team first addressed simpler requirements with application and gateway controls before adding mesh complexity

C. The application had no network communication

D. Managed Kubernetes services prohibit mutual TLS

Correct Answer: B. The team first addressed simpler requirements with application and gateway controls before adding mesh complexity