Cloud Infrastructure and Distributed SystemsIntroduction and ContextCloud computing is built on a physical and logical infrastructure that allows computing, storage, networking, and platform services to be delivered elastically over a network. Although cloud services appear to users as programmable resources, they depend on highly engineered data centres, virtualization layers, distributed control systems, and carefully designed failure-management mechanisms. A postgraduate understanding of cloud computing therefore requires more than familiarity with virtual machines or a cloud provider’s console. It requires analysis of how infrastructure is organized, how workloads are placed and isolated, how distributed components coordinate, and how performance, consistency, availability, and cost are traded against one another. This chapter examines the infrastructure foundations of cloud systems. It connects data-centre architecture with regions and availability zones, explains virtualization and containers, analyses orchestration and scheduling, and introduces the principal design trade-offs of distributed systems. Learning ObjectivesBy the end of this chapter, the learner should be able to: 1. Explain the physical and logical architecture of a modern cloud data centre. 2. Distinguish regions, availability zones, edge locations, and points of presence. 3. Compare hardware virtualization, hypervisors, operating-system containers, and virtual machines. 4. Explain container orchestration, scheduling, service discovery, and autoscaling. 5. Analyse distributed-system trade-offs involving latency, throughput, consistency, availability, and partition tolerance. 6. Apply reliability models to replicated infrastructure. 7. Evaluate fault-tolerance, governance, security, and operational risks in cloud deployments. 1. Data-Centre ArchitectureA cloud data centre is a facility containing computing, storage, networking, power, cooling, physical security, and management systems. It is not merely a large server room. Its design must support high utilization, rapid provisioning, predictable performance, maintenance without widespread interruption, and recovery from hardware and software failures. 1.1 Physical layersThe principal physical layers are: - Compute: servers containing processors, memory, local disks, accelerators, and network interfaces.
- Storage: local solid-state or magnetic storage, network-attached storage, and distributed storage clusters.
- Network: top-of-rack switches, aggregation layers, spine-leaf fabrics, routers, firewalls, load balancers, and links to external networks.
- Power: utility feeds, uninterruptible power supplies, generators, power distribution units, and redundant circuits.
- Cooling: air cooling, liquid cooling, heat exchangers, and environmental monitoring.
- Facilities management: physical access control, fire suppression, monitoring, maintenance, and inventory systems.
Modern facilities commonly use a spine-leaf network. Each leaf switch connects servers in a rack, while spine switches provide high-bandwidth paths between leaf switches. This structure supports relatively predictable east-west traffic, which is traffic between services inside the data centre. North-south traffic refers to traffic entering or leaving the facility. 1.2 Resource pools and control planesCloud providers abstract physical equipment into resource pools. A compute pool may contain thousands of servers, while a storage pool may distribute data across many machines. A control plane accepts user requests, authenticates them, allocates resources, tracks state, applies policies, and exposes management APIs. A data plane performs the actual packet forwarding, computation, or storage operations. The distinction is operationally significant. A control-plane outage may prevent the creation of new virtual machines even when existing applications continue running in the data plane. Conversely, a data-plane failure can interrupt customer workloads while management interfaces remain available. 1.3 Infrastructure efficiencyData-centre efficiency is influenced by server utilization, cooling, power distribution, and workload placement. A commonly used facility metric is Power Usage Effectiveness (PUE): PUE = Total facility energy / IT equipment energy The numerator and denominator must refer to the same time period and energy units. A PUE of 1.4 means that for each unit of energy consumed by IT equipment, 0.4 additional units are consumed by cooling, power distribution, lighting, and other facility systems. PUE is useful for facility comparison, but it does not by itself measure application efficiency, carbon intensity, or service quality. 2. Regions, Availability Zones, and Edge InfrastructureCloud providers divide their infrastructure geographically and logically to balance latency, resilience, regulation, and operational complexity. 2.1 RegionsA region is a geographic area containing one or more isolated data-centre groupings. Regions are usually separated by sufficient distance to reduce the probability that a local disaster affects all of them. Region selection affects network latency, data sovereignty, service availability, taxation, and disaster-recovery strategy. A region is not necessarily a single building. It is a provider-defined failure and service boundary. Services available in one region may not be available in another, and regional APIs may have different capacity, pricing, and feature constraints. 2.2 Availability zonesAn availability zone (AZ) is an isolated infrastructure location within a region, normally consisting of one or more data centres with independent or partially independent power, cooling, and networking arrangements. The purpose of zones is to reduce correlated failures. A workload replicated across zones is generally more resilient than one replicated only across racks in the same zone. Availability zones are not guaranteed to be perfectly independent. They may share regional services, network dependencies, operational personnel, or external utilities. Consequently, zone-aware architecture reduces risk but does not eliminate correlated failure. 2.3 Edge locations and points of presenceEdge locations and points of presence (PoPs) place caching, content delivery, DNS, security filtering, or computation closer to users. They improve latency and reduce traffic sent to central regions. However, edge systems often have smaller resource capacities, more complex deployment models, and weaker consistency guarantees than centralized databases. 2.4 Choosing a deployment topologyA single-zone deployment is simple and may be suitable for development or noncritical workloads. Multi-zone deployment is appropriate for production services requiring resilience to a zone failure. Multi-region deployment is justified when requirements include regional disaster recovery, global latency reduction, regulatory separation, or business continuity across major geographic events. Multi-region architecture introduces difficult issues: data replication delay, conflict resolution, higher network cost, identity synchronization, operational complexity, and potentially divergent software versions. Geographic distribution should therefore be driven by a documented failure model rather than by an assumption that “more regions” is always better. 3. Virtualization and HypervisorsVirtualization allows multiple logical computing environments to share a physical server while maintaining controlled isolation. A virtual machine (VM) presents a guest operating system with virtual CPUs, memory, disks, and network interfaces. 3.1 Hypervisor typesA Type 1 hypervisor, or bare-metal hypervisor, runs directly on the physical host and manages guest operating systems. It is common in data centres because it reduces dependence on a general-purpose host operating system. A Type 2 hypervisor runs as an application on a conventional host operating system. It is common on developer workstations and laboratory systems, where convenience is more important than maximum efficiency. The hypervisor performs CPU scheduling, memory management, device virtualization, and isolation. Hardware-assisted virtualization extensions allow guest operating systems to execute many instructions efficiently while privileged operations are mediated by the hypervisor. 3.2 CPU and memory virtualizationA VM is assigned virtual CPUs (vCPUs), but a vCPU is not necessarily equivalent to a dedicated physical core. The hypervisor schedules vCPUs on physical processing units. Oversubscription occurs when the number of assigned virtual resources exceeds the immediately available physical resources, relying on the fact that workloads are not all fully active simultaneously. Memory virtualization maps guest physical addresses to host physical addresses. Techniques such as extended page tables reduce translation overhead. Memory ballooning, page sharing, and swapping may help reclaim memory, but swapping to storage can produce severe performance degradation. 3.3 Virtual I/O and isolationVirtual network interfaces and virtual disks are connected to software or hardware-assisted switching and storage subsystems. Isolation mechanisms restrict one tenant from reading another tenant’s memory or storage. Strong isolation is essential in public clouds because unrelated customers share hardware. Virtualization has overhead. CPU virtualization may be close to native performance for many workloads, but intensive I/O, memory pressure, device access, and noisy neighbours can produce measurable costs. Performance-sensitive applications should use benchmarking rather than assume that a VM type’s advertised capacity maps directly to application performance. 4. Containers and Micro-virtualizationA container is an isolated user-space environment that shares the host operating-system kernel. Unlike a VM, a container does not normally contain a complete guest kernel. This makes containers lightweight and fast to start, but it also means that kernel vulnerabilities or incompatibilities can have broader consequences than in strongly isolated VM environments. Linux containers commonly use namespaces for process, network, mount, and user isolation, and control groups (cgroups) for resource accounting and limits. A container image packages application code, libraries, configuration defaults, and metadata in an immutable distribution format. At runtime, writable changes are often placed in a temporary layer, while persistent data is stored separately. 4.1 Containers versus virtual machinesVMs provide stronger isolation boundaries and can run different operating systems on the same physical host. Containers generally provide higher density, faster startup, and convenient packaging, but share the host kernel. In practice, cloud platforms frequently run containers inside VMs, combining a VM boundary between tenants with container-level portability and operational flexibility. Containers do not automatically solve dependency, security, or operational problems. An image can contain vulnerable libraries, excessive privileges, embedded secrets, or outdated base components. Secure container practice includes minimal images, image signing, vulnerability scanning, non-root execution, read-only filesystems where possible, and controlled supply chains. 5. Orchestration and SchedulingRunning one container manually is straightforward; operating thousands of containers across changing infrastructure is a distributed-systems problem. An orchestrator maintains the desired state of applications and attempts to make the actual state converge toward it. 5.1 Core orchestration functionsAn orchestrator typically provides: - placement of workloads on eligible nodes;
- health checking and restart mechanisms;
- service discovery and internal load balancing;
- rolling updates and rollback;
- secret and configuration distribution;
- resource quotas and isolation;
- autoscaling;
- persistent-volume attachment;
- policy enforcement and identity integration.
The desired state might specify that five replicas of a service should run, each with defined CPU and memory requests. Controllers continuously compare desired and observed state. If a container exits, a controller may create a replacement; if a node fails, workloads may be rescheduled elsewhere, subject to capacity and storage constraints. 5.2 Scheduling decisionsScheduling is constrained by resource capacity, affinity and anti-affinity, topology, security policy, data locality, licensing, and performance objectives. A simplified resource-feasibility condition for a node is: For each resource type r, allocated(r) + requested(r) ≤ capacity(r) This is a necessary condition, not a complete performance guarantee. CPU and memory requests may be satisfied while network bandwidth, storage IOPS, cache capacity, or accelerator availability becomes a bottleneck. Schedulers often use filtering followed by scoring. Filtering removes nodes that violate hard constraints. Scoring ranks the remaining nodes according to objectives such as balanced utilization, locality, spreading replicas across failure domains, or minimizing energy consumption. 5.3 AutoscalingHorizontal scaling changes the number of service instances; vertical scaling changes the resources assigned to an instance. Autoscaling based only on CPU can be misleading. A database may be limited by storage latency, while a queue consumer may be limited by message arrival rate. A robust autoscaling policy considers demand, latency, queue depth, error rate, and startup time. It should also include stabilization windows to prevent oscillation. Scaling cannot compensate for a fundamentally inefficient algorithm or an unavailable downstream dependency. 6. Distributed-System Models and Trade-offsA distributed system consists of components that communicate over a network and coordinate to provide a unified service. Network communication differs from local procedure calls: messages can be delayed, reordered, duplicated, or lost; nodes can crash; and clocks are imperfect. 6.1 Failure modelsCommon failure models include: - Crash failure: a process stops responding.
- Omission failure: a message is not sent or received.
- Timing failure: a response is too late to be useful.
- Byzantine failure: a component behaves arbitrarily or maliciously.
- Correlated failure: multiple components fail because of a shared dependency.
Cloud engineering generally assumes crash, network, and correlated infrastructure failures. Systems requiring protection against malicious infrastructure or arbitrary behaviour may require Byzantine fault-tolerant techniques, which are substantially more complex. 6.2 CAP perspectiveThe CAP theorem concerns a distributed data system that may experience a network partition. Under such a partition, a system cannot simultaneously guarantee both linearizable consistency for all operations and availability for all requests. The theorem does not mean that systems permanently choose only two of three informal properties, nor does it say that consistency and availability are irrelevant when no partition exists. During a partition, a design may: - reject or delay some operations to preserve consistency;
- accept operations on multiple sides to preserve availability, later reconciling conflicts;
- provide weaker consistency such as eventual consistency;
- use application-specific semantics, such as commutative updates.
The correct choice depends on business requirements. A payment authorization service may prioritize correctness and avoid accepting conflicting decisions. A social-media “like” counter may accept temporary divergence to remain responsive. 6.3 Consistency modelsStrong or linearizable consistency gives the illusion that each operation occurs atomically at a single point between its invocation and completion. It is easier for application developers to reason about but may require coordination and incur latency. Sequential consistency preserves a single order consistent with each client’s program order, but that order need not correspond to real-time ordering across all clients. Causal consistency preserves the order of causally related operations while allowing concurrent operations to appear in different orders. Eventual consistency guarantees that, if updates stop and communication continues, replicas converge to the same value. It says nothing about how quickly convergence occurs or what intermediate values clients observe. Consistency is not a single global switch. A system may use strong consistency for account balances, causal consistency for conversations, and eventual consistency for search indexes. 7. Replication, Fault Tolerance, and ReliabilityReplication stores multiple copies of data or service instances. It can improve availability and read capacity, but it creates coordination, storage, and consistency costs. For independent components with availabilities A1, A2, ..., An, the availability of a serial composition is: A_series = product of Ai for i = 1 to n For independent parallel components where the service succeeds if at least one component is available: A_parallel = 1 − product of (1 − Ai) for i = 1 to n These equations assume statistical independence and clearly defined component boundaries. Real cloud failures are often correlated, so calculated availability may overstate actual performance. Worked example: replicated serviceSuppose two independent service instances each have availability 0.99, and a request succeeds if either instance is available. Then: A_parallel = 1 − (1 − 0.99)(1 − 0.99) = 1 − (0.01 × 0.01) = 0.9999 The idealized availability is therefore 99.99%. If both instances share one database or one network gateway, the overall service will not achieve this value because the shared dependency remains a single point of failure. 7.1 QuorumsMany replicated stores use quorum rules. If there are N replicas, a read quorum R and write quorum W are commonly selected so that R + W > N, ensuring overlap between a read and a write quorum. If W > N/2, two successful writes cannot be disjoint under the simplified quorum model. Quorum overlap does not automatically provide linearizability. Version selection, conflict resolution, replica freshness, failure handling, and read-repair policies also matter. 7.2 Failure detection and recoveryFailure detectors use timeouts, heartbeats, health probes, and observed progress. A timeout is evidence of possible failure, not proof: the service may be alive but slow, partitioned, or overloaded. Aggressive timeouts can cause false failovers and cascading load; conservative timeouts delay recovery. Recovery mechanisms include retries, exponential backoff, circuit breakers, replication, checkpointing, write-ahead logging, failover, and graceful degradation. Retries must be bounded and used carefully. Retrying a non-idempotent payment request can create duplicate effects unless an idempotency key or transactional mechanism is used. 8. Detailed Applied Case: Multi-Zone Online Examination PlatformThe following is an illustrative case. BackgroundA university operates an online examination platform used by 20,000 students. The system includes authentication, question delivery, answer submission, grading, reporting, and audit logging. Examinations are high-value events: loss of an answer or inconsistent submission timestamp could create academic disputes. ProblemThe initial design uses one VM, one relational database, and periodic backups. During a previous examination, a storage failure caused an outage. The university requires improved resilience while preserving correct answer records and controlling cost. AnalysisThe most important assets are submitted answers, examination state, identity records, and audit logs. Read-heavy question delivery can tolerate caching and eventual refresh. Answer submission requires stronger guarantees. A single-zone deployment exposes the entire service to a zone-level failure. A multi-region active-active design would improve geographic resilience but introduce conflict resolution and complex academic-record governance. Approach and solutionThe revised architecture uses: 1. A load balancer across three availability zones. 2. Stateless application containers managed by an orchestrator. 3. Anti-affinity rules so replicas are distributed across zones. 4. A multi-zone relational database with synchronous protection within the region. 5. An append-oriented audit log with immutable retention controls. 6. Object storage for examination material and encrypted backups. 7. A content-delivery layer for static question assets. 8. Idempotency keys for answer submissions. 9. Queue-based processing for noncritical grading and reporting tasks. 10. Monitoring for latency, failed submissions, replication lag, and queue depth. The application acknowledges an answer only after it has passed validation and been durably recorded according to the database’s transaction policy. The user interface may cache the last confirmed answer locally, but local browser state is not treated as the authoritative record. OutcomeA zone failure can be handled by shifting traffic to healthy application instances, provided the database service and network paths remain available. If the primary database becomes unavailable, the documented recovery procedure promotes a protected replica or restores from a verified backup. The platform does not claim zero data loss without evidence; instead, it defines a recovery point objective (RPO) and recovery time objective (RTO). Learning pointsThe case illustrates that resilience is not obtained merely by adding replicas. Data semantics, idempotency, shared dependencies, recovery procedures, testing, and governance determine whether the architecture is genuinely dependable. Strong consistency is appropriate for accepted answer records, whereas cached static content can use weaker consistency. 9. Security, Governance, and Operational PracticeInfrastructure design has security and governance implications. Multi-tenant environments require isolation of memory, storage, networks, identities, and administrative privileges. Critical controls include encryption in transit and at rest, key management, least privilege, network segmentation, vulnerability management, secure image pipelines, tamper-evident logging, and independent auditing. Data placement across regions may be restricted by privacy, financial, health, or national-security regulations. Organizations should identify data classifications, retention periods, legal transfer mechanisms, and deletion requirements before selecting regions or replication policies. Operationally, infrastructure should be managed as code, reviewed through change control, and tested using failure injection, backup restoration, capacity tests, and disaster-recovery exercises. Service-level objectives should be measurable and tied to user-visible outcomes. A provider’s service-level agreement is not a substitute for an organization’s own architecture, monitoring, or recovery plan. Artificial intelligence is increasingly used for anomaly detection, capacity forecasting, workload placement, and incident correlation. Such systems can reduce operational burden, but automated remediation must be bounded by policy. Poor training data, opaque decisions, and feedback loops can cause inappropriate scaling or conceal emerging failures. Human oversight remains necessary for high-impact infrastructure decisions. 10. Contemporary Developments and Critical IssuesSeveral developments are reshaping cloud infrastructure: - Confidential computing uses hardware-supported trusted execution environments to protect data during processing, although side-channel and operational risks remain.
- Serverless and managed platforms shift infrastructure management to providers but do not remove concerns about latency, quotas, observability, or vendor dependence.
- Service meshes provide traffic management, identity, and telemetry between services, but add proxies and operational complexity.
- Disaggregated infrastructure separates compute, memory, and storage resources to improve utilization, while requiring high-speed fabrics and advanced orchestration.
- Edge computing supports low-latency applications but complicates deployment, consistency, physical security, and fleet management.
- Green cloud computing considers carbon intensity, renewable-energy availability, workload scheduling, and hardware lifecycle, rather than energy consumption alone.
A mature architecture evaluates these technologies against workload requirements rather than adopting them as universal solutions. Best-Practice Design Principles1. Define failure domains explicitly and place replicas across independent domains. 2. Separate stateless application scaling from stateful data management. 3. Match consistency guarantees to business consequences. 4. Treat timeouts, retries, and idempotency as part of the application protocol. 5. Measure actual performance under representative load. 6. Eliminate hidden single points of failure, including identity, DNS, databases, and deployment systems. 7. Use least privilege and secure software supply chains. 8. Automate provisioning, testing, monitoring, and rollback. 9. Test backups by restoring them and test failover under controlled conditions. 10. Document RPO, RTO, service-level objectives, dependencies, and escalation procedures. Chapter SummaryCloud infrastructure combines physical data centres, networks, storage systems, virtualization, containers, and distributed control planes. Regions and availability zones provide geographic and failure-domain organization, but neither guarantees complete independence from correlated failures. Hypervisors virtualize compute, memory, storage, and networking, while containers provide lightweight process isolation by sharing an operating-system kernel. Orchestrators maintain desired application state through scheduling, health checking, service discovery, scaling, and controlled updates. Scheduling must consider more than CPU and memory; topology, storage, network capacity, security, and data locality are also important. Distributed systems must tolerate delay, loss, crashes, and correlated failures. Consistency, availability, latency, and partition tolerance involve engineering trade-offs rather than universally correct choices. Replication and quorum techniques improve resilience but introduce coordination and operational costs. Reliability calculations are useful only when their independence and dependency assumptions are realistic. Professional cloud architecture requires security, governance, observability, tested recovery, and explicit service objectives. The most robust design is not necessarily the most replicated or technologically complex; it is the design whose guarantees, costs, and failure behaviour are understood and aligned with business requirements. Key TermsRegion: A geographic cloud infrastructure area containing one or more isolated deployment groupings. Availability zone: An isolated infrastructure location within a region designed to reduce correlated failures. Edge location: A location near users that provides caching, networking, or computation. Data plane: The infrastructure path that performs application traffic processing or resource operations. Control plane: Systems that manage configuration, provisioning, policy, and resource state. Virtual machine: A software-defined computer running a guest operating system on virtualized hardware. Hypervisor: Software or firmware that creates and manages virtual machines. Container: An isolated user-space environment that generally shares the host operating-system kernel. Namespace: An operating-system mechanism that isolates processes or resources for containers. Cgroup: A mechanism for controlling and accounting for resource use by processes. Orchestrator: A system that deploys, monitors, scales, and repairs application workloads. Scheduler: A component that selects suitable infrastructure for workload placement. Oversubscription: Allocation of more virtual resources than are simultaneously physically available. Consistency: The guarantees governing what values and operation orders distributed clients observe. Linearizability: A consistency model in which operations appear to occur atomically in real-time order. Eventual consistency: A guarantee that replicas converge if updates stop and communication continues. Quorum: A sufficiently large subset of replicas required for an operation to succeed. Idempotency: The property that repeating an operation produces the same intended effect as performing it once. RPO: Recovery point objective, or the maximum acceptable amount of data loss measured in time. RTO: Recovery time objective, or the target time for restoring a service after disruption. Review QuestionsShort Answer Questions1. What is the difference between a region and an availability zone? 2. Why can a virtual CPU not always be treated as a dedicated physical CPU core? 3. State one major difference between a container and a virtual machine. 4. What is the purpose of anti-affinity in workload scheduling? 5. Define eventual consistency. Long Answer Questions1. Explain the architecture of a modern cloud data centre, including compute, storage, networking, power, cooling, and control planes. 2. Compare Type 1 and Type 2 hypervisors and discuss the performance and isolation implications of virtualization. 3. Explain the major functions of a container orchestrator and the constraints involved in scheduling workloads. 4. Discuss the CAP perspective and compare linearizable, causal, and eventual consistency. 5. Explain how replication, recovery procedures, idempotency, and dependency analysis contribute to fault tolerance. Analytical / Application Questions1. Design a deployment topology for a banking transaction service that must survive a single availability-zone failure. Identify its consistency, replication, and recovery requirements. 2. A service has two independent instances, each with availability 0.99, but both depend on one database with availability 0.995. Analyse why the service availability cannot be assumed to be 0.9999 and identify architectural improvements. 3. An online marketplace wants low-latency multi-region writes for product-stock updates. Evaluate the trade-offs between strong coordination, regional availability, conflict resolution, and overselling risk. Multiple Choice Questions with AnswersQuestion 1Which statement best describes an availability zone? A. A global content-delivery cache B. An isolated infrastructure location within a region C. A virtual machine image format D. A software container registry Correct Answer: B. An isolated infrastructure location within a region Question 2What is the primary function of a hypervisor? A. To compile application source code B. To manage virtual machines and mediate access to hardware C. To replace all physical networking equipment D. To provide database indexing Correct Answer: B. To manage virtual machines and mediate access to hardware Question 3Which resource mechanism is commonly used to limit and account for container resource consumption? A. Cgroups B. DNS zones C. Object buckets D. Message queues Correct Answer: A. Cgroups Question 4If two independent service instances each have availability 0.99 and either can serve a request, what is the idealized combined availability? A. 0.9801 B. 0.9900 C. 0.9999 D. 1.9800 Correct Answer: C. 0.9999 Question 5Which condition is a common quorum overlap rule for N replicas? A. R + W = 0 B. R + W > N C. R − W > N D. R × W = N − 1 Correct Answer: B. R + W > N Question 6What does eventual consistency guarantee under its basic definition? A. Every read immediately returns the newest value B. All operations are totally ordered in real time C. Replicas converge if updates stop and communication continues D. No network partition can occur Correct Answer: C. Replicas converge if updates stop and communication continues Question 7Why are idempotency keys useful in distributed applications? A. They increase processor clock speed B. They help prevent repeated requests from causing duplicate effects C. They eliminate the need for authentication D. They guarantee zero network latency Correct Answer: B. They help prevent repeated requests from causing duplicate effects Question 8Which action is a control-plane operation? A. Forwarding an application packet through a switch B. Executing a user’s application code C. Provisioning a new virtual machine D. Reading a file from a local disk Correct Answer: C. Provisioning a new virtual machine Question 9What is the principal risk of treating a timeout as definite proof that a service has failed? A. It may cause false failover while the service is merely slow B. It guarantees stronger consistency C. It prevents all correlated failures D. It removes the need for monitoring Correct Answer: A. It may cause false failover while the service is merely slow Question 10Which deployment most directly improves resilience to a single availability-zone failure? A. Placing all replicas in one rack B. Running replicas across multiple availability zones C. Using one larger virtual machine D. Disabling health checks Correct Answer: B. Running replicas across multiple availability zones
|