Cloud Infrastructure, Containers, and Orchestration

Introduction and Context

Serverless applications are often described as applications without servers. This is convenient, but technically incomplete. Servers, networks, storage systems, operating systems, identity services, and monitoring platforms still exist; the cloud provider operates much of them on the customer’s behalf. A sound understanding of infrastructure is therefore essential for designing reliable serverless and microservice systems.

This chapter examines the infrastructure layer beneath modern applications. It connects virtual networking, identity, storage, containers, orchestration, service discovery, configuration, secrets, and infrastructure as code. It also explains how managed serverless functions can coexist with containerized workloads when an application requires different execution models.

Learning Objectives

By the end of this chapter, you should be able to:

• describe the major components of cloud infrastructure;
• design basic network, identity, and storage arrangements;
• distinguish containers from virtual machines and serverless functions;
• explain orchestration, scheduling, scaling, and service discovery;
• separate configuration from secrets and manage both safely;
• express infrastructure using infrastructure as code;
• select an appropriate hybrid serverless-container architecture;
• identify reliability, security, cost, governance, and operational risks.

1. The Infrastructure Abstraction Stack

Cloud computing provides layered abstractions. At the physical layer are data centres, processors, memory, disks, and network equipment. Virtualization converts physical resources into virtual machines, networks, and volumes. Container runtimes isolate application processes while sharing a host operating system kernel. Orchestration platforms schedule and manage containers across clusters. Platform services provide databases, queues, identity, event processing, and serverless execution.

A useful principle is that each abstraction reduces operational responsibility but also reduces direct control. A virtual machine permits extensive operating-system customization, whereas a function-as-a-service platform offers rapid deployment but imposes limits on runtime, duration, networking, and execution state.

The shared-responsibility model is central. A provider may secure the physical data centre and managed control plane, while the customer remains responsible for identity policies, application vulnerabilities, data classification, and correct configuration.

2. Essential Cloud Networking

2.1 Logical network structure

A virtual private cloud or equivalent virtual network is an isolated address space. It is commonly divided into subnets associated with availability zones or fault domains. Public subnets have a route to an internet gateway; private subnets do not accept unsolicited internet traffic. A private workload may still reach external services through a network address translation gateway or an approved proxy.

A typical application network contains:

1. an edge layer for a load balancer or API gateway;
2. an application layer for functions, containers, or virtual machines;
3. a data layer for databases and internal storage;
4. routing rules, firewall rules, and monitoring controls.

Security groups or equivalent stateful firewalls usually control traffic at the resource level. Network access-control lists may provide subnet-level, often stateless, filtering. Least privilege applies to networks: permit only the protocols, ports, sources, and destinations required by the design.

2.2 Addressing and routing

IPv4 addresses contain a network prefix and a host portion. In classless notation, 10.20.0.0/16 identifies a network with 16 prefix bits. The number of addresses in an IPv4 prefix of length p is:

Here, N is the total number of IPv4 addresses and p is the prefix length. This calculation assumes ordinary IPv4 addressing and counts network and broadcast addresses where those conventions apply. For a /24 network, N = 2^(32−24) = 256 addresses; traditional subnet conventions commonly leave 254 usable host addresses because one address identifies the network and one the broadcast address. Cloud platforms may reserve additional addresses, so provider documentation must be consulted.

Routing tables determine the next hop for a destination. Domain Name System (DNS) maps names to addresses and enables service endpoints to change without requiring application code to embed IP addresses. Load balancers distribute requests and can perform health checks, TLS termination, and path-based routing.

2.3 Network design principles

Use multiple availability zones for production services where the platform supports them. Keep databases private, encrypt traffic in transit, avoid unnecessary public IP addresses, and record flow logs. Design for failure of an individual instance, zone, or dependency. Network segmentation is useful, but segmentation alone is not authorization: an application must still authenticate and authorize every sensitive operation.

3. Identity and Access Management

Identity and Access Management (IAM) answers three questions: who or what is making a request, what action is being requested, and which resource is affected. Authentication establishes identity; authorization evaluates permissions; auditing records relevant events.

Human users should normally authenticate through centralized identity federation and receive temporary credentials. Workloads should use roles or workload identities rather than long-lived access keys. A policy can be understood as a tuple of subject, action, resource, and conditions. For example, a reporting service may be permitted to read a particular object-storage prefix but not delete objects or access unrelated data.

The principle of least privilege means granting the minimum permissions necessary for a defined task. Permissions should be reviewed because identities, services, and business requirements change. Separate development, testing, and production accounts or projects reduce blast radius. Multi-factor authentication, privileged-access workflows, key rotation, denial policies, and audit logging are important controls.

Identity is not limited to infrastructure. A microservice must also enforce application-level authorization. A network-accessible endpoint is not necessarily an authorized endpoint. Defence in depth combines identity, network policy, encryption, input validation, and monitoring.

4. Cloud Storage Choices

Storage should be selected according to access pattern, durability, latency, consistency, cost, and data lifecycle.

Durability describes the probability that data will not be lost over a period; availability describes the probability that it can be accessed when requested. They are different properties. Replication improves resilience but does not automatically protect against accidental deletion, corruption, or unauthorized access. Versioning, immutable backups, retention policies, encryption, and tested restoration are therefore required.

For an object, a useful conceptual storage-cost estimate is:

The terms represent the provider’s storage, request, data-transfer, and operational charges. This is a planning model rather than a universal billing equation; actual pricing varies by provider, region, tier, and discount arrangement.

5. Containers and Images

A container packages an application and its user-space dependencies while sharing the host kernel. It is not a miniature virtual machine. Namespaces provide process, filesystem, and network isolation; control groups limit and account for resources. A virtual machine includes a guest operating system and generally provides stronger kernel isolation at the cost of additional overhead.

A container image is an immutable, versioned package assembled from layers. A Dockerfile or equivalent build specification should use a trusted minimal base image, pin important dependencies, exclude credentials, run as a non-root user, and produce reproducible builds. Images should be scanned for vulnerabilities and signed or attested where supported. The image digest, rather than an unqualified mutable tag such as latest, provides a stronger deployment reference.

Containers are portable, but not perfectly portable. Differences in CPU architecture, kernel features, storage drivers, networking, permissions, and managed-platform behaviour can affect execution. Containerization also does not remove the need for patching: the image, dependencies, runtime, and host platform all require security management.

6. Orchestration

Orchestration manages many containers across a pool of compute resources. A platform such as Kubernetes commonly includes a control plane, worker nodes, a scheduler, an API server, a controller system, and a cluster state store. Users declare desired state—for example, three replicas of a web service—and controllers attempt to make actual state converge toward that desired state.

A deployment typically creates replica-managed workloads. A service provides a stable virtual endpoint while selecting healthy workload instances. An ingress or gateway exposes selected services externally. ConfigMaps or equivalent objects hold non-sensitive configuration, while secret mechanisms hold sensitive values. Persistent-volume abstractions connect workloads to durable storage.

Orchestration performs several related tasks:

• scheduling workloads according to resource requests, limits, affinity, and policy;
• restarting failed containers;
• replacing unhealthy replicas;
• rolling out new versions and rolling back failed releases;
• scaling replicas or compute nodes;
• applying network and identity policies.

Resource requests influence scheduling; limits constrain consumption. If requests are too low, a node may become overloaded. If limits are too high, utilization and cost may suffer. Health checks should distinguish a liveness check, which asks whether a process should be restarted, from a readiness check, which asks whether it should receive traffic.

Autoscaling is not magic. It depends on useful metrics, sufficient capacity, startup time, queue behaviour, and stable control rules. A service that scales on CPU but is constrained by database connections will remain unhealthy unless the scaling signal reflects the actual bottleneck.

7. Service Discovery and Communication

In a distributed system, instances are ephemeral. Service discovery provides a stable name or registry entry that resolves to currently available instances. Platform-native service abstractions, DNS, registries, and service meshes are common approaches.

Synchronous HTTP or gRPC communication is appropriate when the caller needs an immediate result. Queues and event streams are preferable when work can be delayed, retried, buffered, or processed independently. Every remote call should have a timeout. Retries must be bounded and should use exponential backoff with jitter; otherwise, a failing dependency can create a retry storm. Retry only operations that are safe to repeat or protected by idempotency keys.

A useful availability approximation for independent serial components is:

A_i is the availability of component i. The approximation assumes independence and a request path that requires every component. Real systems violate these assumptions through correlated failures, redundancy, caching, and partial operation. For example, three serial components each with availability 0.99 yield approximately 0.99 × 0.99 × 0.99 = 0.9703, or 97.03% for that path—not 99%.

8. Configuration and Secrets

Configuration includes values that vary between environments or deployments: feature flags, endpoint names, logging levels, and timeout settings. Secrets include passwords, private keys, tokens, and credentials. Both should be externalized from application code, but secrets require stronger controls.

A secret-management service should provide encryption, access policies, auditing, versioning, and rotation. Secrets must not be committed to source control, embedded in images, printed in logs, or exposed through diagnostic endpoints. Environment variables are convenient but can leak through process inspection, crash reports, or careless logging; a mounted secret or runtime retrieval mechanism may be preferable.

Configuration changes should be validated at startup and, where safe, dynamically reloaded. A failed configuration should cause a clear and observable failure rather than silent use of an insecure default. Separate configuration namespaces and credentials by environment.

9. Infrastructure as Code

Infrastructure as Code (IaC) expresses networks, identities, storage, clusters, and services in declarative or programmable files. Tools such as Terraform, OpenTofu, CloudFormation, and provider-specific systems allow infrastructure to be reviewed, versioned, tested, and reproduced.

A controlled IaC workflow generally includes:

1. define resources and dependencies;
2. format and statically validate files;
3. generate and review an execution plan;
4. run policy, security, and cost checks;
5. apply through an approved pipeline;
6. record state and outputs securely;
7. monitor drift between declared and actual resources.

State files may contain sensitive values and must be encrypted, access-controlled, versioned, and locked to prevent concurrent updates. Modules improve reuse, but excessive abstraction can hide important security settings. Avoid manually changing managed resources unless the change is reconciled into code. Test destructive changes and define explicit retention policies for data resources.

10. Hybrid Serverless-Container Architectures

Serverless functions are effective for short-lived, event-driven, variable workloads. Containers are preferable when an application requires a custom runtime, long execution, specialized libraries, predictable background processing, or fine-grained process control. A hybrid architecture uses both rather than forcing every component into one model.

For example, an API gateway can invoke a function for authentication and request validation. The function can place a job on a queue. A container service can process computationally intensive jobs, store results in object storage, and publish a completion event. Another function can notify users. This design separates burst-oriented orchestration from sustained processing.

Selection should consider execution duration, startup latency, concurrency, operating-system requirements, deployment frequency, workload variability, observability, and cost. A container does not automatically cost less than a function, and a function does not automatically scale without limits. Estimate usage using measured workload characteristics and provider pricing.

Applied Case: Illustrative Online Document Processing Platform

Background

An educational publisher receives document uploads from authors. Each document must be scanned, converted to searchable text, checked for malware, and made available to authorized editors.

Problem

Uploads are unpredictable, conversion tools have large dependencies, and processing may take several minutes. Editors require secure access, status updates, and reliable recovery after failures.

Analysis

A synchronous upload request should not wait for conversion. Documents are sensitive intellectual property, so public object access is inappropriate. Conversion workers require a custom container image and more execution time than a typical function. The system must prevent duplicate processing when events are delivered more than once.

Approach

An API gateway invokes an authentication function. The document is uploaded using a short-lived, restricted upload authorization. Object-storage creation emits an event to a queue. A containerized worker service consumes queue messages, retrieves the document using a workload identity, performs malware scanning and conversion, and writes results to a private results bucket. A database records status and an idempotency key based on the document version and job identifier. A notification function publishes completion information to the editor portal.

The worker has a readiness check, bounded retries, a dead-letter queue, resource limits, and access only to the required storage prefixes. Encryption is enabled in transit and at rest. Infrastructure is created through IaC, and audit logs record access to documents and administrative actions.

Outcome

The API remains responsive during upload bursts. The queue absorbs uneven demand, while the container service scales according to queue depth and processing capacity. Failed jobs can be investigated and replayed without re-uploading documents. Functions handle lightweight event coordination, while containers handle the specialized processing workload.

Learning Points

The case demonstrates that hybrid architecture is a division of responsibilities, not merely the use of two technologies. Reliable design depends on identity, private storage, idempotency, observability, failure handling, and controlled infrastructure—not only on selecting a runtime.

Challenges, Governance, and Contemporary Practice

Infrastructure complexity can become a serious operational burden. Common risks include excessive permissions, insecure images, unbounded retries, untested backups, public storage, configuration drift, noisy monitoring, and uncontrolled cloud expenditure. Governance should define ownership, tagging, data classification, retention, approved regions, incident response, and change authority.

Privacy and regulatory obligations may require data residency, access records, deletion procedures, encryption-key control, and breach reporting. Organizations should understand where logs, backups, replicas, and support systems store data. Accessibility and responsible design also matter: infrastructure choices influence whether services remain available to users with limited connectivity or assistive technologies.

Contemporary platforms increasingly provide managed container execution, serverless containers, confidential computing, policy-as-code, software-supply-chain attestations, and platform engineering portals. AI-assisted tools can generate IaC, summarize logs, or suggest configurations, but generated changes require human review, testing, least-privilege analysis, and protection against disclosure of proprietary data. Automation increases speed; it does not transfer accountability.

Best-Practice Checklist

• Use private subnets and explicit, minimal network paths.
• Prefer temporary workload identities over embedded keys.
• Encrypt sensitive data and test restoration from backups.
• Build small, patched, signed, and reproducible container images.
• Define health checks, resource requests, limits, and disruption policies.
• Use timeouts, bounded retries, idempotency, and dead-letter handling.
• Keep configuration external and secrets in a managed secret system.
• Review IaC plans, protect state, scan dependencies, and detect drift.
• Monitor latency, errors, saturation, queue depth, cost, and security events.
• Select functions or containers according to workload characteristics rather than fashion.

Chapter Summary

Cloud applications depend on an infrastructure stack ranging from virtual networks and identity to storage, containers, orchestration, and managed application services. Networking provides segmentation, routing, naming, and controlled connectivity. IAM authenticates principals and authorizes narrowly defined actions. Storage selection must consider access patterns, durability, availability, security, and lifecycle cost.

Containers package processes and dependencies but share a host kernel; orchestration schedules, heals, scales, and exposes them. Service discovery, timeouts, bounded retries, and idempotency are essential because distributed components fail independently. Configuration should be externalized, while secrets require specialized storage and access control. IaC makes infrastructure reviewable and reproducible but introduces state, drift, and change-management responsibilities.

Hybrid architectures combine serverless functions for event-driven coordination with containers for specialized or sustained processing. Professional design balances performance, reliability, security, governance, operational effort, and cost.

Key Terms

Cloud infrastructure: Compute, network, storage, identity, and platform resources used to run applications.

Virtual private cloud: Logically isolated cloud network.

Subnet: A subdivided network address range.

Route table: Rules determining how traffic reaches destinations.

IAM: Identity and Access Management; controls authentication and authorization.

Least privilege: Granting only the permissions required for a task.

Object storage: API-based storage of objects identified by keys.

Container: Isolated user-space process sharing a host kernel.

Container image: Versioned package containing application code and dependencies.

Orchestration: Automated scheduling and management of distributed workloads.

Replica: An independently running copy of a workload.

Service discovery: Finding currently available service instances through stable names or registries.

Idempotency: Property that repeating an operation produces the same intended result.

Configuration: Non-secret values controlling application behaviour.

Secret: Sensitive credential, token, key, or password.

Infrastructure as Code: Managing infrastructure through versioned machine-readable definitions.

Drift: Difference between declared infrastructure and actual infrastructure.

Serverless function: Provider-managed, event-triggered execution unit.

Hybrid architecture: A design combining serverless and container-based components.

Review Questions

Short Answer Questions

1. What is the difference between a public subnet and a private subnet?
2. Why should workload identities generally be preferred to long-lived access keys?
3. How does a container differ from a virtual machine?
4. What is the purpose of a readiness check?
5. Why should secrets not be stored in source-control repositories?

Long Answer Questions

1. Explain how networking, IAM, and storage controls combine to protect a cloud application.
2. Describe the principal components and responsibilities of a container orchestration platform.
3. Compare object, block, and file storage, including suitable application examples.
4. Explain a secure and reviewable infrastructure-as-code workflow.
5. Discuss the circumstances in which a hybrid serverless-container architecture is appropriate.

Analytical / Application Questions

1. Design a network and identity arrangement for an API, worker service, and private database. Justify each major control.
2. A queue-based document processor receives duplicate messages and experiences occasional worker failures. Propose mechanisms for idempotency, retries, dead-letter handling, and observability.
3. A team wants to run every workload as a serverless function. Evaluate this proposal against execution duration, dependencies, startup behaviour, scaling, security, and operational requirements.

Multiple Choice Questions with Answers

Question 1

Which component normally provides a stable endpoint while selecting healthy container instances?

A. A container image

B. A service abstraction

C. A secret store

D. A block volume

Correct Answer: B. A service abstraction

Question 2

How many total IPv4 addresses are represented by a /24 prefix under the standard address calculation?

A. 24

B. 128

C. 256

D. 512

Correct Answer: C. 256

Question 3

Which practice best follows the principle of least privilege?

A. Giving every service administrator access

B. Using one shared account for all environments

C. Granting a worker read access only to its required storage prefix

D. Embedding a permanent root key in the container image

Correct Answer: C. Granting a worker read access only to its required storage prefix

Question 4

What is the main purpose of a readiness check?

A. To decide whether a workload should receive traffic

B. To encrypt a container image

C. To allocate an IP address permanently

D. To create a database backup

Correct Answer: A. To decide whether a workload should receive traffic

Question 5

Which workload is most suitable for a custom container rather than a short-lived function?

A. A simple event validation lasting milliseconds

B. A specialized conversion process requiring large native libraries and several minutes

C. A single lightweight notification

D. A small authorization lookup

Correct Answer: B. A specialized conversion process requiring large native libraries and several minutes

Question 6

Which item should normally be stored in a managed secret-management system?

A. A public application name

B. A logging level

C. A database password

D. A non-sensitive region identifier

Correct Answer: C. A database password

Question 7

What problem does infrastructure as code primarily address?

A. It removes the need for application testing

B. It makes infrastructure declarative, reviewable, and reproducible

C. It guarantees zero cloud cost

D. It replaces all identity controls

Correct Answer: B. It makes infrastructure declarative, reviewable, and reproducible

Question 8

Why are bounded retries and backoff important in distributed systems?

A. They prevent a failing dependency from being overwhelmed by retry storms

B. They guarantee that every request succeeds

C. They eliminate the need for timeouts

D. They make all operations idempotent automatically

Correct Answer: A. They prevent a failing dependency from being overwhelmed by retry storms

Question 9

Three independent serial components each have availability of 0.99. What is the approximate availability of the path requiring all three?

A. 0.03

B. 0.9703

C. 0.99

D. 1.03

Correct Answer: B. 0.9703

Question 10

In the illustrative document platform, why is a queue used between upload handling and conversion?

A. To make the storage bucket public

B. To absorb uneven demand and decouple the request from long processing

C. To remove the need for authentication

D. To ensure conversion never fails

Correct Answer: B. To absorb uneven demand and decouple the request from long processing