Cloud-Native Infrastructure and Containers

Introduction

Microservices are not made cloud-native merely by placing services on a virtual machine or publishing them through a cloud provider. Cloud-native architecture combines independently deployable software with automated infrastructure, declarative operations, elastic resource management, resilience, and observability. Containers provide a consistent packaging unit; orchestration coordinates containers at scale; infrastructure as code makes environments reproducible; and related technologies such as service meshes and serverless platforms address communication and operational efficiency.

This chapter examines the infrastructure layer required to operate microservices reliably. The central design problem is to balance developer autonomy, platform standardisation, security, cost, and operational complexity. A small application may need only a managed container service, whereas a large enterprise may require Kubernetes, a service mesh, policy enforcement, multiple regions, and hybrid connectivity.

Learning Objectives

By the end of this chapter, students should be able to:

1. Explain container images, runtimes, registries, and isolation mechanisms.
2. Describe Kubernetes architecture and the lifecycle of a workload.
3. Distinguish configuration, secrets, persistent data, and ephemeral state.
4. Apply infrastructure-as-code and immutable-infrastructure principles.
5. Explain the purpose and limitations of service meshes and serverless platforms.
6. Evaluate multi-cloud and hybrid deployment choices.
7. Design a secure, observable, and operationally realistic deployment for microservices.

1. From Virtual Machines to Containers

A virtual machine (VM) emulates a complete computer, including a guest operating system kernel. A hypervisor allocates virtual CPU, memory, storage, and network devices to each VM. Containers generally share the host operating system kernel while isolating processes through operating-system mechanisms. On Linux, namespaces isolate process, network, mount, and user views, while control groups (cgroups) constrain and measure resource consumption.

A container is therefore not a miniature VM. It is a process, or group of processes, with a packaged filesystem and declared runtime requirements. Containers usually start faster and consume fewer resources than VMs, but their isolation boundary depends on the kernel, runtime, configuration, and platform security. A container image commonly contains application binaries, libraries, metadata, and a default command. It does not normally contain a separate kernel.

Images, registries, and runtimes

A container image is built from immutable layers. A Dockerfile or equivalent build description specifies a base image, files to copy, dependencies, environment defaults, and the startup command. A registry stores and distributes images. A runtime such as containerd or CRI-O creates and executes containers, while a higher-level tool or orchestrator manages their lifecycle.

A production image should be small, deterministic, and free from credentials. Multi-stage builds can compile an application in one stage and copy only the required artifact into a minimal runtime stage. Images should be identified by a cryptographic digest when reproducibility matters; mutable tags such as latest are convenient for experimentation but weaken deployment traceability.

The image supply chain should include dependency scanning, malware detection, provenance records, signature verification, and admission policies. A signed image is not automatically safe: its source, build process, dependencies, runtime permissions, and vulnerability response must also be assessed.

Container design principles

A well-designed container generally follows these principles:

• one principal service process per container, with supporting processes separated when independent scaling is useful;
• configuration supplied at runtime rather than embedded in the image;
• logs written to standard output and error for collection by the platform;
• graceful handling of termination signals;
• explicit health endpoints or commands;
• non-root execution where possible;
• read-only filesystems and dropped Linux capabilities when compatible;
• resource requests and limits defined for scheduling and protection.

Containers do not eliminate application design responsibilities. A service must still manage timeouts, retries, idempotency, schema compatibility, and graceful shutdown.

2. Orchestration and Kubernetes

Running a few containers manually is feasible, but production microservices require scheduling, service discovery, rolling updates, failure recovery, scaling, and policy enforcement. Orchestration automates these functions across a cluster of machines.

Kubernetes is a widely used declarative orchestration platform. A user describes a desired state, such as “run three replicas of image X,” and Kubernetes controllers continuously compare actual state with desired state. This reconciliation model is a practical implementation of control-loop thinking: failures are expected, observed state is compared with intended state, and corrective action is applied.

Kubernetes architecture

A cluster contains a control plane and worker nodes. The control plane commonly includes:

• the API server, which exposes the cluster API;
• a persistent state store, commonly etcd;
• the scheduler, which selects suitable nodes for unscheduled Pods;
• controller managers, which reconcile resources such as Deployments and Nodes.

Worker nodes run a container runtime, a node agent called kubelet, and networking components. The smallest schedulable unit is a Pod, which contains one or more tightly coupled containers sharing networking and selected storage. Most application containers run in one-container Pods, while sidecar patterns place a supporting container beside the main process.

A Deployment manages the desired number and version of stateless Pod replicas. A ReplicaSet, usually created by a Deployment, maintains replica count. A Service provides a stable virtual endpoint and selects Pods using labels. An Ingress or Gateway exposes HTTP or HTTPS traffic from outside the cluster, although exact capabilities depend on the installed controller.

Other important resources include:

• ConfigMap: non-sensitive configuration data;
• Secret: data intended to be sensitive, though protection depends on encryption, access control, and platform configuration;
• Job: finite work that should complete;
• CronJob: scheduled finite work;
• StatefulSet: stable identity and storage patterns for stateful workloads;
• DaemonSet: one Pod, or a controlled Pod set, on selected nodes;
• Namespace: a logical boundary for organisation, access control, and quotas.

Scheduling, health, and scaling

The scheduler considers resource requests, node capacity, affinity rules, taints, tolerations, and other constraints. A resource request represents the amount used for scheduling; a limit represents an enforced ceiling or scheduling-related constraint, depending on resource and implementation. Incorrect values create problems: requests that are too high waste capacity, while requests that are too low cause contention and unreliable performance.

Kubernetes distinguishes several health concepts. A startup probe allows slow-starting applications time to initialise. A readiness probe indicates whether a Pod should receive traffic. A liveness probe indicates whether a container should be restarted. A liveness probe should not merely test whether a process exists; an overly aggressive probe can create a restart loop during temporary overload or dependency failure.

A Horizontal Pod Autoscaler (HPA) adjusts replica count using metrics such as CPU utilisation or application-specific indicators. For a simple average-utilisation policy, the target replica estimate can be represented as:

Here, R_current is the current replica count, U_current is measured average utilisation, U_target is the desired utilisation, and ceil rounds upward to the next whole replica. The formula assumes that utilisation is comparable across replicas and that scaling responds approximately proportionally. It is not a complete capacity model: startup delay, queue length, concurrency, downstream limits, and stabilisation windows must also be considered.

For example, if four replicas average 80% CPU utilisation and the target is 50%, the estimate is ceil(4 × 0.80 ÷ 0.50) = ceil(6.4) = 7. This does not guarantee that seven replicas will meet demand because CPU may not be the limiting resource.

Deployment strategies

A rolling update gradually replaces old Pods with new Pods. It reduces disruption but requires backward-compatible application and database changes. A blue-green deployment maintains two environments and shifts traffic from the old version to the new one, allowing rapid rollback at the cost of additional capacity. A canary deployment sends a controlled fraction of traffic to a new version and evaluates technical and business indicators before increasing exposure.

Rollback is only safe when the previous version remains compatible with current data and external contracts. Database migrations therefore commonly use an expand-and-contract sequence: add compatible structures, deploy code that can use both forms, migrate data, remove obsolete code, and finally remove obsolete structures.

3. Configuration, Secrets, and State

Configuration includes values that vary between environments, such as service endpoints, feature flags, timeouts, and log levels. It should be external to the image so the same artifact can be promoted from testing to production. Configuration must be validated at startup, with safe defaults and explicit failure for missing critical values.

Secrets include passwords, private keys, tokens, certificates, and encryption keys. A Kubernetes Secret is an API object, not a complete secret-management strategy. Teams should restrict access using least-privilege roles, encrypt secret data at rest, audit access, rotate credentials, and consider an external secret manager or hardware-backed key service. Secrets must not appear in source control, image layers, logs, crash reports, or unprotected build output.

The principle of least privilege applies at several levels: cloud identity, Kubernetes service accounts, namespaces, network policies, filesystem permissions, and database roles. Workloads should use short-lived identities where possible rather than shared static credentials.

Containers are often ephemeral. Local container filesystems should be treated as disposable because a rescheduled Pod may run on another node. Persistent data requires a storage abstraction such as a PersistentVolumeClaim backed by block, file, or database storage. Stateful systems need careful analysis of replication, consistency, backup, recovery time, recovery point, and failure domains. Kubernetes can run databases, but orchestration does not automatically make database operations safe.

4. Infrastructure as Code and Immutable Infrastructure

Infrastructure as code (IaC) represents networks, identities, clusters, databases, policies, and supporting resources in version-controlled declarative files. Tools such as Terraform, OpenTofu, CloudFormation, Pulumi, and provider-specific systems differ in language and state management, but the engineering principles are similar.

A reliable IaC workflow generally includes:

1. code review and validation;
2. static analysis and policy checks;
3. a generated plan showing proposed changes;
4. controlled approval;
5. automated application through a pipeline;
6. post-deployment verification and drift detection.

IaC state may contain sensitive values and must be protected through access controls, encryption, locking, backup, and separation of environments. Modules should standardise secure patterns without hiding important architectural decisions. Excessive abstraction can make generated infrastructure difficult to understand.

Immutable infrastructure means replacing an instance or image rather than manually modifying a running production instance. The approach improves repeatability, auditability, and rollback. It does not mean that all data is immutable; persistent state remains in managed storage or databases. Nor does it forbid emergency intervention. Rather, emergency changes should be recorded and reconciled into the declared source of truth.

A useful distinction is:

5. Networking and Service Discovery

Microservices communicate through networks that introduce latency, packet loss, congestion, address changes, and partial failure. Kubernetes Services provide stable virtual addressing while labels select current Pods. Cluster DNS commonly maps service names to these endpoints. Applications should use connection timeouts, bounded retries, exponential backoff with jitter, and circuit-breaking or load-shedding strategies where appropriate.

Retries require special care. Retrying a non-idempotent operation can create duplicate effects. A request may also be accepted by a downstream service even when the response is lost. Idempotency keys, transactional design, and explicit failure semantics are therefore more important than simply increasing retry counts.

Network policies can restrict which Pods may communicate. They should be tested because a policy that is too permissive increases attack surface, while one that is too restrictive breaks legitimate dependencies. Ingress controls north-south traffic; service-to-service traffic is often described as east-west traffic.

6. Service Meshes

A service mesh adds a dedicated communication layer, commonly through a proxy associated with each workload. The proxy can provide mutual TLS, service identity, traffic routing, retries, timeouts, telemetry, and policy enforcement. A control plane distributes configuration and identity material; a data plane carries application traffic.

A mesh can standardise cross-cutting communication functions that would otherwise be implemented inconsistently in every programming language. It can support traffic splitting for canaries and provide a stronger identity model than IP addresses alone. However, it adds resource consumption, configuration complexity, latency, and another failure and upgrade domain. A mesh does not repair poor service boundaries, incorrect retry policies, insecure application authorisation, or unavailable dependencies.

Teams should adopt a mesh when its concrete requirements—such as uniform mutual TLS, advanced traffic control, or cross-language telemetry—justify its operational cost. For a small system, platform-native ingress, application libraries, and managed networking may be simpler.

7. Serverless Options

Serverless computing abstracts much of the server and cluster management. Function-as-a-Service (FaaS) runs event-triggered functions, while serverless container platforms run containers without requiring teams to manage nodes directly. Typical characteristics include automatic scaling, usage-based pricing, event integration, and limits on execution duration, networking, filesystem persistence, or runtime control.

Serverless is effective for sporadic workloads, event processing, scheduled jobs, APIs with variable demand, and rapid experimentation. It can reduce operational work, but cold-start latency, concurrency limits, vendor-specific APIs, observability differences, and unpredictable cost at sustained high load must be evaluated. Serverless does not remove architecture: queues, retries, idempotency, authentication, data ownership, and failure handling remain necessary.

A practical decision is not “containers versus serverless” in absolute terms. A platform may use managed serverless functions for image processing, managed containers for long-running APIs, and Kubernetes for specialised scheduling or portability requirements.

8. Multi-Cloud and Hybrid Deployment

Multi-cloud uses services from more than one cloud provider; hybrid deployment combines private infrastructure with public cloud resources. Motivations include regulatory requirements, acquisition history, resilience, customer proximity, bargaining power, and specialised services. These benefits must be weighed against duplicated operations, inconsistent identity models, networking complexity, data transfer charges, skills requirements, and weaker economies of scale.

Portability exists at multiple layers. Container images may be portable while storage semantics, load balancers, identity systems, databases, observability tools, and managed queues are not. A design that uses only the lowest common denominator may achieve technical portability but lose useful provider capabilities.

Deployment patterns include:

• active-active: multiple locations serve traffic simultaneously;
• active-passive: one location serves while another is prepared for failover;
• cloud bursting: secondary capacity is used during demand peaks;
• distributed data ownership: each region serves a defined data domain;
• hybrid extension: selected services remain on-premises while others run in cloud infrastructure.

Disaster recovery must be tested rather than inferred from replication. Recovery time objective (RTO) is the maximum acceptable time to restore service; recovery point objective (RPO) is the maximum acceptable amount of data loss measured in time. A system with an RPO of 15 minutes may tolerate losing up to 15 minutes of recent changes, subject to the organisation’s definition and measurement method. Backups, replicas, failover automation, DNS behaviour, identity availability, and human procedures all affect actual recovery.

9. Observability, Reliability, and Security

Observability uses logs, metrics, and traces to infer internal system behaviour from external outputs. Logs provide event detail; metrics support aggregation and alerting; traces connect a request across services. Correlation identifiers should be propagated consistently, but they must not contain sensitive personal data.

Useful service-level indicators include request latency, error rate, throughput, queue age, saturation, and availability. Service-level objectives define target performance over a period. An error budget can guide release decisions: if reliability is worse than the objective, teams may prioritise remediation over risky feature delivery.

Security should be layered. Important controls include image scanning, signed artifacts, admission policies, network segmentation, runtime monitoring, workload identity, encrypted transport, patch management, audit logging, and incident response. Kubernetes control-plane access is especially sensitive. The API server, cluster credentials, admission configuration, and etcd backups must be protected.

Compliance may require data residency, retention controls, access records, deletion procedures, segregation of duties, and evidence that controls operate continuously. Privacy risks arise when telemetry captures request bodies, tokens, health data, or customer identifiers. Data minimisation and redaction should be designed before production deployment.

10. Illustrative Case: A Retail Ordering Platform

Background

Consider an illustrative online retailer whose order, payment, inventory, and notification functions have been separated into microservices. The original system runs on manually configured VMs. Deployments are slow, environment differences cause defects, and a failed VM requires an engineer to reconstruct configuration.

Problem

The retailer needs frequent releases, automatic recovery, auditability, and seasonal scaling. Payment information must be protected, while inventory data requires durable storage. The team is considering Kubernetes and a second cloud provider for disaster recovery.

Analysis

Not every component requires identical infrastructure. Stateless order and notification APIs can run as replicated containers. The payment service requires strict identity, network controls, and careful audit logging. Inventory requires a managed database with tested backups rather than an assumption that a StatefulSet alone provides adequate durability. Batch reconciliation is a good candidate for a Job or serverless workload.

Approach and solution

The team creates reproducible multi-stage images and stores signed digests in a private registry. IaC provisions the network, cluster, identities, database, monitoring, and backup policies. A Deployment manages API replicas; Services provide internal discovery; readiness probes prevent unready Pods from receiving traffic. Configuration is stored separately, while credentials are retrieved from an external secret manager using workload identity.

A progressive delivery pipeline first deploys to a test namespace, runs integration and security checks, and then performs a canary release. Metrics include checkout error rate, payment latency, inventory reservation failures, and queue age. The payment service uses mutual TLS and explicit authorisation; network policies restrict access to the database. The platform team initially avoids a full service mesh because the organisation lacks the operational capacity to manage it, but documents the conditions under which it may later be adopted.

For disaster recovery, the database is backed up and restored into a separate environment during scheduled exercises. Container images and IaC modules are replicated, but provider-specific networking and identity procedures are documented separately. DNS failover is tested with realistic client caching behaviour.

Outcome

The design reduces configuration drift and improves deployment repeatability. It also exposes costs and responsibilities that the earlier VM design concealed: cluster upgrades, image maintenance, secret rotation, database recovery testing, and observability ownership become explicit operational work.

Learning points

1. Containers improve packaging but do not replace reliability engineering.
2. Kubernetes is valuable when its automation and ecosystem justify its complexity.
3. Managed stateful services may be safer than self-hosted databases.
4. Portability requires planning for identity, data, networking, and operations—not only images.
5. Progressive delivery and measurable objectives make rollback decisions safer.

11. Best Practices and Limitations

A mature platform begins with the simplest service that satisfies reliability, security, and scaling requirements. Teams should standardise golden paths for image construction, deployment, logging, identity, and alerts, while allowing justified exceptions. Every workload should document ownership, dependencies, resource assumptions, recovery expectations, data classification, and support procedures.

Common anti-patterns include using latest in production, storing secrets in Git, granting cluster-admin privileges to applications, placing all services in one unrestricted network, retrying indefinitely, using liveness probes as dependency tests, and adopting Kubernetes or a service mesh solely because they are fashionable. Another failure is treating cloud-provider abstraction as free: abstraction layers can conceal useful capabilities and create difficult debugging boundaries.

AI-assisted coding and operations can help generate manifests, identify configuration anomalies, summarise incidents, or propose capacity changes. Such systems must be constrained by review, policy validation, least privilege, and protection of confidential telemetry. An automatically generated deployment manifest remains an operational change and requires human accountability.

Chapter Summary

Cloud-native infrastructure provides the operational foundation for microservices. Containers package applications consistently, but they depend on secure images, correct runtime permissions, and disciplined lifecycle design. Kubernetes applies declarative reconciliation to scheduling, service discovery, scaling, health management, and progressive deployment. Configuration should be externalised, secrets protected through dedicated controls, and persistent data managed with explicit durability and recovery strategies.

Infrastructure as code and immutable infrastructure improve repeatability and reduce drift, while GitOps extends reconciliation principles to deployment workflows. Service meshes can provide identity, encryption, routing, and telemetry, but their complexity must be justified. Serverless functions and managed containers are useful alternatives for event-driven or variable workloads. Multi-cloud and hybrid designs may improve resilience or satisfy governance requirements, but portability is limited by data, identity, networking, and operational differences.

Successful cloud-native engineering is therefore not a product-selection exercise. It is the coordinated design of packaging, orchestration, security, observability, recovery, governance, and team capability.

Key Terms

Container: An isolated process environment sharing a host kernel.

Container image: An immutable, layered package containing application files and runtime metadata.

Registry: A repository that stores and distributes container images.

Orchestration: Automated scheduling, networking, scaling, and lifecycle management of workloads.

Kubernetes: A declarative platform for orchestrating containerised workloads.

Pod: Kubernetes’ smallest schedulable unit, containing one or more containers.

Deployment: A Kubernetes resource that manages replicated, usually stateless, application Pods.

Service: A stable Kubernetes network endpoint selecting Pods by labels.

ConfigMap: A Kubernetes object for non-sensitive configuration data.

Secret: A Kubernetes object intended for sensitive data, requiring additional protection.

Infrastructure as code: Version-controlled declarative management of infrastructure resources.

Immutable infrastructure: Replacing deployed artifacts instead of manually modifying them.

GitOps: A practice in which version-controlled declarations drive automated reconciliation.

Service mesh: A communication layer that manages service-to-service traffic and policy.

Serverless: A model in which the provider manages much of the underlying server capacity.

Readiness probe: A health check determining whether a workload should receive traffic.

Liveness probe: A health check used to determine whether a container should be restarted.

RTO: The maximum acceptable time to restore a service after disruption.

RPO: The maximum acceptable time interval of data loss after disruption.

Review Questions

Short Answer Questions

1. Why is a container different from a virtual machine?
2. What is the purpose of a Kubernetes Service?
3. Distinguish a readiness probe from a liveness probe.
4. Why should secrets not be embedded in container images?
5. State one benefit and one limitation of infrastructure as code.

Long Answer Questions

1. Explain Kubernetes control-plane and worker-node responsibilities.
2. Compare rolling, blue-green, and canary deployment strategies.
3. Discuss how persistent state should be managed in a containerised system.
4. Evaluate the benefits and costs of adopting a service mesh.
5. Explain why multi-cloud portability involves more than using portable container images.

Analytical / Application Questions

1. A service has four replicas operating at 80% average CPU utilisation, with a target of 50%. Use the chapter’s proportional estimate to calculate the suggested replica count and explain why the result is not a complete capacity plan.
2. Design a deployment approach for a payment API that must protect credentials, support rollback, and provide auditable access. Identify image, identity, network, secret, and observability controls.
3. An organisation wants Kubernetes, a service mesh, active-active multi-cloud deployment, and self-hosted databases for a small application team. Analyse the likely operational risks and recommend a proportionate alternative.

Multiple Choice Questions with Answers

Question 1

Which Kubernetes resource normally provides a stable network endpoint for a changing set of Pods?

A. Service

B. ConfigMap

C. Job

D. PersistentVolumeClaim

Correct Answer: A. Service

Question 2

What is the primary purpose of a readiness probe?

A. To determine whether a container should receive traffic

B. To encrypt traffic between clusters

C. To build a container image

D. To allocate persistent storage

Correct Answer: A. To determine whether a container should receive traffic

Question 3

Which practice best supports reproducible production deployments?

A. Using the mutable latest image tag

B. Building images manually on production nodes

C. Referencing tested images by digest

D. Embedding environment secrets in the image

Correct Answer: C. Referencing tested images by digest

Question 4

If four replicas have 80% average utilisation and the target is 50%, what replica estimate follows the chapter’s formula?

A. 4

B. 5

C. 6

D. 7

Correct Answer: D. 7

Question 5

What does infrastructure as code primarily provide?

A. Version-controlled, declarative management of infrastructure

B. Automatic removal of all operational responsibilities

C. Guaranteed portability across every cloud provider

D. A replacement for application testing

Correct Answer: A. Version-controlled, declarative management of infrastructure

Question 6

Which statement about Kubernetes Secrets is most accurate?

A. They are automatically safe regardless of cluster configuration

B. They require access control and appropriate encryption protections

C. They are intended only for public configuration

D. They cannot be accessed by workloads

Correct Answer: B. They require access control and appropriate encryption protections

Question 7

What is a major function of a service mesh?

A. Replacing all application databases

B. Managing service-to-service traffic, identity, and telemetry

C. Compiling source code into container images

D. Providing physical data-centre cooling

Correct Answer: B. Managing service-to-service traffic, identity, and telemetry

Question 8

Which workload is often a suitable candidate for serverless execution?

A. An event-triggered image-processing task

B. A database requiring custom kernel modules

C. A permanently connected storage controller

D. A cluster control plane

Correct Answer: A. An event-triggered image-processing task

Question 9

What does RPO describe?

A. The maximum acceptable service restoration time

B. The maximum acceptable interval of data loss

C. The number of replicas required for scaling

D. The time needed to build an image

Correct Answer: B. The maximum acceptable interval of data loss

Question 10

Why can a multi-cloud design remain difficult even when it uses portable containers?

A. Containers cannot communicate over networks

B. Identity, storage, networking, and managed services differ across providers

C. Kubernetes cannot schedule more than one Pod

D. Container images always include physical hardware drivers

Correct Answer: B. Identity, storage, networking, and managed services differ across providers