Compute, Virtualization, and Serverless Infrastructure

Introduction and Context

Compute infrastructure provides the processing capability required to run applications, databases, analytics systems, and automated workloads. In cloud computing, compute is delivered through several abstraction levels: virtual machines (VMs), containers, managed instance groups, dedicated hosts, and serverless functions. Each option offers a different balance of control, portability, operational responsibility, scalability, performance, and cost.

A cloud practitioner must therefore do more than create a server. The practitioner must select an appropriate compute model, choose a suitable machine size, define an image and configuration process, design for failure, and select a pricing model that matches workload behaviour. The central design question is:

Which compute abstraction provides the required performance and control at the lowest sustainable operational and financial cost?

This chapter examines the major compute models and develops a practical framework for provisioning and comparing them.

Learning Objectives

By the end of this chapter, you should be able to:

1. Explain virtualization and the role of the hypervisor.
2. Provision and configure virtual machines using images and instance specifications.
3. Compare machine sizing, pricing models, dedicated resources, and placement options.
4. Design highly available and autoscaling groups.
5. Explain containers and serverless functions as alternative compute abstractions.
6. Select event-driven compute for suitable application workloads.
7. Estimate compute capacity and cost using appropriate assumptions.
8. Evaluate operational, security, governance, and sustainability implications.

1. Compute Abstraction in the Cloud

A compute abstraction hides some physical infrastructure while exposing a programmable interface. The higher the abstraction, the less infrastructure the customer manages, but the less direct control the customer normally has.

This model is often called a shared-responsibility model. Moving from VMs to functions usually reduces infrastructure administration, but application design must accommodate limits such as execution duration, statelessness, and restricted runtime environments.

2. Virtualization Fundamentals

Virtualization enables several logical machines to share one physical host. A hypervisor allocates processor time, memory, storage, and network access to guest virtual machines.

2.1 Hypervisors

A Type 1, or bare-metal, hypervisor runs directly on the physical server. It is common in data centres because it provides efficient resource control. A Type 2 hypervisor runs above a conventional host operating system and is common on desktop computers and development systems.

The hypervisor presents virtual hardware to each guest. The guest operating system behaves as though it owns a complete computer, although its instructions and device requests are mediated by the virtualization layer. Modern processor extensions reduce the overhead of virtualization, but performance is not automatically identical to bare metal.

2.2 Virtual Machine Components

A VM specification normally includes:

• virtual CPUs, often expressed as vCPUs;
• memory, measured in GiB;
• a boot volume and optional data volumes;
• virtual network interfaces;
• security groups or equivalent firewall rules;
• an operating-system image;
• metadata, startup scripts, and identity permissions.

A VM is not merely a size. It is the combination of a machine type, image, storage, network placement, identity, security policy, and lifecycle configuration.

2.3 Images

An image is a reusable template containing an operating system, installed packages, configuration, and sometimes application code. Images may be supplied by a provider, created by an organisation, or built automatically through an image pipeline.

A reliable image process should:

1. begin with a trusted and supported base image;
2. apply security updates;
3. install only required packages;
4. remove credentials, temporary files, and debugging secrets;
5. apply baseline configuration;
6. run automated tests;
7. record the image version and build evidence;
8. digitally sign or otherwise protect the image where supported.

Immutable-image practice means that a deployed VM is replaced with a new image rather than manually modified repeatedly. This improves reproducibility and reduces configuration drift.

3. Provisioning Virtual Machines

A typical VM provisioning sequence is:

1. Select a region and availability zone.
2. Select a trusted image.
3. Choose vCPU, memory, storage, and network performance.
4. Attach an appropriate identity or role.
5. Configure firewall rules using least privilege.
6. Add encrypted storage and backup policy.
7. Apply startup configuration or configuration management.
8. Register the VM with monitoring, logging, and asset inventory.
9. Test health, connectivity, performance, and recovery.

Infrastructure as Code (IaC) expresses these choices in version-controlled configuration. IaC makes environments repeatable and supports code review, automated testing, and controlled change. It does not, however, remove the need to understand the underlying infrastructure.

Example: Selecting a VM

Suppose a web application requires 4 vCPUs, 16 GiB of memory, 100 GiB of persistent storage, and moderate network throughput. A suitable general-purpose instance may be selected if its documented specification meets or exceeds these requirements. The decision should also consider:

• whether the workload is CPU- or memory-intensive;
• the expected peak request rate;
• storage IOPS and throughput;
• availability-zone capacity;
• licensing restrictions;
• regional price and currency;
• compatibility with the selected operating system.

The smallest machine that passes a realistic load test is generally preferable to a machine chosen only by guesswork.

4. Machine Sizing and Capacity Planning

Sizing should be based on measured or estimated demand rather than nominal application requirements. Important metrics include CPU utilization, memory pressure, disk latency, network throughput, request rate, queue depth, and response time.

A simple capacity estimate for stateless servers can be expressed as:

where:

• N is the number of instances;
• D_peak is peak demand, such as requests per second;
• C_safe is the safe capacity of one instance, measured in the same units;
• ⌈x⌉ means round upward to the next whole instance.

The formula assumes that instances are functionally equivalent and that demand is distributed adequately. C_safe should not normally equal the absolute laboratory maximum. It should include a safety margin and should be established through load testing.

For availability, if one instance must be allowed to fail while still serving demand, a simple planning expression is:

where F is the planned number of simultaneous instance failures. This is a planning approximation, not a guarantee. Failure domains, startup time, dependency limits, and correlated failures must also be considered.

Worked Example

A service receives a peak of 1,800 requests per second. Load testing shows that one instance safely handles 600 requests per second. The design requires capacity for one instance failure.

Four instances provide three instances of nominal serving capacity after one failure. In practice, the architect should validate the assumption under realistic traffic, cache behaviour, database limits, and deployment conditions.

Vertical and Horizontal Scaling

Vertical scaling increases the size of an instance, for example by adding vCPUs or memory. It is simple but eventually reaches machine limits and may require interruption. Horizontal scaling adds instances. It improves elasticity and fault tolerance but requires stateless design, load balancing, service discovery, and shared or externalized state.

A common cloud architecture uses horizontal scaling for the application tier and a separately designed data tier. Session state may be stored in a managed database or cache rather than local VM memory.

5. Autoscaling Groups

An autoscaling group, also called a managed instance group in some platforms, maintains a population of similar instances. It usually includes:

• a launch template or instance template;
• minimum, desired, and maximum capacity;
• health checks;
• a scaling policy;
• a load balancer or service registration mechanism;
• placement across multiple failure domains.

Autoscaling is not simply “adding servers when CPU is high.” A good policy chooses a meaningful metric and accounts for monitoring delay, instance startup time, warm-up behaviour, and workload queues.

Scaling Policies

Common policies include:

• Target tracking: maintain a target such as 60% average CPU utilization.
• Step scaling: add different quantities for different alarm ranges.
• Scheduled scaling: adjust capacity at known times, such as business opening hours.
• Predictive scaling: forecast demand from historical patterns.
• Queue-based scaling: add workers as queue depth or message age increases.

CPU is suitable for CPU-bound applications but may be misleading for I/O-bound services. Request latency, queue age, active connections, or custom business metrics may be better indicators.

Autoscaling has a control-loop nature: metrics are observed, a policy calculates a desired change, instances start or stop, and the system is measured again. Excessively aggressive policies may cause oscillation, while slow policies may fail to respond to a sudden demand spike.

Availability Design

Instances should be distributed across at least two failure domains where the provider and workload requirements permit. A load balancer should remove unhealthy instances. Graceful termination allows existing requests to complete. Startup scripts must be idempotent, because they may execute repeatedly on replacement instances.

Autoscaling cannot repair an application that stores essential state only on local disk or memory. It also cannot overcome a saturated database, a provider quota, or an incorrectly configured health check.

6. Pricing and Dedicated Resources

Cloud providers commonly offer several pricing approaches:

A simplified compute cost estimate is:

where each term represents the relevant cost component for the billing period. A basic instance estimate is:

where p is the price per instance-hour, h is the number of billed hours per instance, and n is the number of instances. The units must be consistent; for example, dollars per hour multiplied by hours gives dollars.

If a workload runs continuously for 720 hours with four instances at $0.08 per instance-hour:

This excludes storage, data transfer, operating-system licensing, monitoring, and other services. Actual billing may use per-second or per-minute granularity and may include discounts or minimum charges.

Dedicated resources can mean a dedicated host, a dedicated instance, or a bare-metal server. They may be justified by regulatory isolation, software licensing tied to physical servers, predictable performance, or specialised hardware. They are not automatically more secure; access controls, patching, encryption, and monitoring remain necessary.

7. Containers as an Intermediate Abstraction

Containers package an application and its dependencies while sharing the host kernel. Compared with VMs, containers generally start faster and use resources more efficiently, but they provide a different isolation boundary. A container image is immutable, layered, and normally promoted through development, testing, and production environments.

Container orchestration platforms schedule containers, restart failed workloads, expose services, and support rolling deployments. They introduce additional concepts such as clusters, nodes, pods or tasks, service discovery, ingress, secrets, and resource requests.

Containers are useful when an organisation needs portability and rapid deployment but still requires control over runtime behaviour. They do not eliminate patching: the image, dependencies, host, orchestration platform, and application must all be maintained.

8. Serverless Functions

Function as a Service (FaaS) executes small units of code in response to events. The provider manages the servers, operating-system maintenance, capacity provisioning, and much of the scaling process. The customer supplies function code, configuration, permissions, and dependencies.

A function is commonly characterized by:

• a trigger, such as an HTTP request or queue message;
• a runtime and memory allocation;
• a maximum execution duration;
• an identity and permission policy;
• environment variables or managed configuration;
• concurrency limits;
• logging and tracing settings.

Functions should generally be stateless. Persistent state belongs in a database, object store, cache, or queue. Temporary local storage may exist but should not be treated as durable unless the platform explicitly guarantees it.

Event-Driven Architecture

An event-driven system separates event producers from consumers. A producer publishes an event to a queue, topic, or event bus. A function consumes the event and performs an action. This arrangement supports loose coupling and asynchronous processing.

Important design concerns include:

• At-least-once delivery: a message may be delivered more than once, so handlers should be idempotent.
• Retries: transient failures may be retried automatically.
• Dead-letter handling: repeatedly failing events should be isolated for investigation.
• Ordering: ordering is not always guaranteed and may reduce scalability when required.
• Concurrency: too many simultaneous functions can overload downstream systems.
• Observability: correlation identifiers, logs, metrics, and traces are essential.

A useful idempotency strategy is to assign each event a unique identifier and record processed identifiers in durable storage before applying a non-repeatable business action.

Serverless Cost and Performance

Function pricing commonly depends on invocation count and execution duration, sometimes multiplied by allocated memory or another resource dimension. A generic estimate is:

The approximation is provider-specific and must be checked against current pricing documentation. It is especially important to include database operations, API gateways, queues, logging, and data transfer.

Serverless systems may experience cold starts when a new execution environment must be initialized. Provisioned or pre-warmed capacity can reduce latency but may increase cost. Functions are attractive for irregular workloads, automation, file processing, scheduled jobs, and lightweight APIs. Long-running, highly stateful, latency-sensitive, or specialized workloads may be better suited to VMs or containers.

9. Comparing Compute Models

Selection should consider workload characteristics rather than fashion.

A workload may use several models simultaneously. For example, a web front end may run in an autoscaling VM or container group, an image-resizing task may use functions, and a licensed database may use a dedicated host.

10. Applied Case: Online Retail Image Processing

The following is an illustrative case, not a report of a particular company.

Background

A small online retailer allows vendors to upload product images. The existing system runs image resizing synchronously on one VM. During marketing campaigns, upload latency increases and the VM becomes overloaded.

Problem

The retailer needs to process unpredictable bursts, retain the original images, create three resized versions, and prevent a failed image from blocking other uploads. The budget is limited, and the team has little experience administering servers.

Analysis

Image resizing is computationally independent for each upload and does not require a long-lived session. The original image can be stored in object storage. An object-created event can publish a message to a queue. A function can consume the message, create the derivatives, and write them back to object storage.

The queue provides buffering during bursts. A dead-letter queue isolates messages that fail repeatedly. The function must use an idempotency key based on the object identifier and version, because retries may occur. Concurrency must be bounded so that the function does not exhaust downstream storage or account quotas.

Approach and Solution

The retailer replaces synchronous processing with the following flow:

1. The user uploads the original image.
2. Object storage emits an event.
3. A queue stores the processing request.
4. A function retrieves the image and creates three derivatives.
5. The function writes derivatives and records completion metadata.
6. Failed messages are retried and eventually moved to a dead-letter queue.
7. Monitoring reports queue age, invocation errors, duration, and processing backlog.

The web application itself remains on a small autoscaling group because it serves user requests and requires predictable HTTP behaviour. The image-processing function scales independently.

Outcome

Upload requests no longer wait for resizing. Bursts are absorbed by the queue, and processing capacity increases with backlog. The retailer pays for function execution and related services rather than maintaining a large always-on image-processing VM. The design introduces new responsibilities: event observability, retry analysis, permissions, and cost controls.

Learning Points

The case demonstrates that compute selection is workload-specific. Serverless is valuable because the task is stateless, event-driven, and independently scalable—not merely because it is fashionable. The autoscaling web tier and event-driven processing tier are complementary rather than competing choices.

11. Security, Governance, and Professional Practice

Compute infrastructure creates significant security and governance responsibilities. Recommended practices include:

• use least-privilege instance and function identities;
• restrict management access through controlled channels rather than public administrative ports;
• encrypt data at rest and in transit;
• apply operating-system and dependency patches;
• scan images for vulnerabilities;
• separate development, test, and production accounts or projects;
• record administrative activity in audit logs;
• enforce quotas, budgets, and spending alerts;
• remove unused resources and stale images;
• define backup, retention, and deletion policies;
• test disaster recovery rather than merely documenting it.

Regulatory obligations may affect geographic placement, retention, access logging, and isolation. Dedicated resources may support a compliance requirement, but compliance is a system-level property involving people, processes, configurations, and evidence.

Environmental responsibility is also relevant. Rightsizing, autoscaling, efficient architectures, workload scheduling, and use of managed services can reduce idle capacity and energy consumption. However, serverless does not mean zero environmental impact; the provider still operates physical infrastructure, and inefficient invocation patterns can increase resource use.

12. Contemporary Developments and Limitations

Current cloud platforms increasingly combine VMs, containers, functions, managed accelerators, and specialized hardware. Confidential computing uses hardware-supported techniques to protect data while it is being processed. Arm-based instances may offer different price-performance characteristics from x86 instances, but application compatibility must be tested. AI workloads may require GPUs or other accelerators, whose availability and pricing differ substantially from general-purpose compute.

AI-assisted operations can help detect anomalies, recommend rightsizing, summarize incidents, or forecast demand. Such tools should support—not replace—human review. Automated recommendations may be wrong when monitoring data is incomplete, and sensitive operational data must be governed carefully.

Important limitations remain:

• autoscaling reacts after signals are observed and cannot always handle instant demand;
• spot capacity may be interrupted;
• functions can suffer cold starts and platform limits;
• distributed systems make testing and debugging more difficult;
• managed services can create vendor dependence;
• low per-invocation prices can conceal high aggregate cost;
• dedicated resources may be underutilized;
• images and dependencies can become outdated or vulnerable.

The professional response is not to seek a universally best compute model. It is to establish measurable requirements, test realistic workloads, document assumptions, and revisit the design as demand changes.

Best-Practice Decision Framework

A practical decision sequence is:

1. Classify the workload: interactive, batch, stream, scheduled, or event-driven.
2. Identify state, latency, duration, concurrency, and failure requirements.
3. Estimate baseline and peak demand.
4. Choose the lowest suitable abstraction that meets constraints.
5. Select regions, failure domains, machine families, and pricing options.
6. Define images, identities, security controls, and observability.
7. Test performance, failure recovery, and cost under realistic conditions.
8. Automate deployment and establish rollback procedures.
9. Review utilization, reliability, security, and cost continuously.

Chapter Summary

Cloud compute ranges from highly controlled dedicated hardware and VMs to highly abstracted serverless functions. Virtual machines provide broad compatibility and control, while images and Infrastructure as Code improve repeatability. Machine sizing should use measured safe capacity, and horizontal scaling with health checks and multiple failure domains improves resilience. Autoscaling policies must use meaningful metrics and account for startup delays and downstream limits.

Pricing choices include on-demand, committed, spot, and dedicated options. Total cost includes more than the advertised compute rate. Containers provide a portable intermediate abstraction, while serverless functions are particularly suitable for stateless, event-driven, independently scalable tasks. Event-driven systems require careful treatment of retries, idempotency, ordering, concurrency, and dead-letter processing.

Professional compute design combines technical fit, cost, security, governance, observability, sustainability, and operational capability. No compute model is universally superior; the correct choice follows from workload requirements and validated evidence.

Key Terms

Virtual machine: A software-defined computer that runs an operating system on virtualized hardware.

Hypervisor: Software that creates and manages virtual machines on physical hosts.

Type 1 hypervisor: A hypervisor that runs directly on physical hardware.

Image: A reusable template containing an operating system and configuration.

Immutable infrastructure: Infrastructure replaced from versioned definitions rather than modified manually.

vCPU: A virtual processor allocated to a virtual machine.

Vertical scaling: Increasing the resources of one compute instance.

Horizontal scaling: Adding or removing instances to change capacity.

Autoscaling group: A managed set of similar instances that adjusts capacity according to policy.

Launch template: A reusable specification for creating instances.

Health check: A test used to determine whether a resource can serve traffic.

Spot or preemptible instance: Discounted compute that may be interrupted by the provider.

Dedicated host: A physical server allocated for a particular customer or tenancy requirement.

Container: An isolated application environment that shares a host kernel.

Function as a Service: Execution of application functions without customer management of underlying servers.

Event-driven architecture: An architecture in which events trigger processing by decoupled components.

Idempotency: The property that repeating an operation produces the same intended result as performing it once.

Cold start: Initialization delay when a new function execution environment is created.

Infrastructure as Code: Management of infrastructure through version-controlled machine-readable definitions.

Rightsizing: Matching allocated resources to actual workload requirements.

Review Questions

Short Answer Questions

1. What role does a hypervisor play in virtualization?
2. Distinguish vertical scaling from horizontal scaling.
3. Why are images important in VM provisioning?
4. What is the purpose of a dead-letter queue?
5. Name two situations in which dedicated resources may be justified.

Long Answer Questions

1. Explain the process of provisioning a secure and observable virtual machine.
2. Compare on-demand, committed-use, spot, and dedicated pricing approaches.
3. Explain how an autoscaling group supports availability and elasticity.
4. Compare VMs, containers, and functions in terms of control, operations, scaling, and workload fit.
5. Describe the main design requirements of a reliable event-driven function system.

Analytical / Application Questions

1. A service receives 2,400 requests per second, and one instance safely handles 800 requests per second. How many instances are required if the design must tolerate one instance failure? State the assumptions behind your calculation.
2. A company is deciding between a continuously running VM and a function-based image-processing system. Analyse the decision using workload duration, burstiness, state, operations, latency, and total cost.
3. An autoscaling application shows normal CPU utilization but increasing request latency and queue age. Recommend a revised scaling strategy and explain why CPU alone is an inadequate metric.

Multiple Choice Questions with Answers

Question 1

Which component creates and manages virtual machines on a physical host?

A. Hypervisor

B. Load balancer

C. Object store

D. Message queue

Correct Answer: A. Hypervisor

Question 2

A service has peak demand of 1,800 requests per second. Each instance safely handles 600 requests per second, and one instance failure must be tolerated. How many instances should be planned using the chapter’s approximation?

A. 3

B. 4

C. 5

D. 6

Correct Answer: B. 4

Question 3

Which scaling approach adds more instances rather than increasing the size of an existing instance?

A. Vertical scaling

B. Horizontal scaling

C. Static scaling

D. Kernel scaling

Correct Answer: B. Horizontal scaling

Question 4

Which pricing option normally provides a lower price in exchange for a commitment to predictable usage?

A. On-demand

B. Spot

C. Reserved or committed use

D. Unallocated capacity

Correct Answer: C. Reserved or committed use

Question 5

Why should a serverless function generally avoid storing essential state in local memory?

A. Functions cannot execute code in memory

B. Execution environments may be replaced or run concurrently

C. Memory is always encrypted incorrectly

D. Local memory is shared with every customer

Correct Answer: B. Execution environments may be replaced or run concurrently

Question 6

What is the primary purpose of a dead-letter queue?

A. To permanently accelerate successful messages

B. To store messages that repeatedly fail processing

C. To replace all application databases

D. To allocate additional VM memory

Correct Answer: B. To store messages that repeatedly fail processing

Question 7

Which workload is generally a strong candidate for function-based compute?

A. A long-running stateful database server

B. A task triggered by an object-upload event

C. A host requiring a specialized kernel continuously

D. A licensed application tied to one physical server

Correct Answer: B. A task triggered by an object-upload event

Question 8

What property ensures that processing the same event more than once does not create an unintended repeated business effect?

A. Idempotency

B. Overprovisioning

C. Virtualization

D. Compression

Correct Answer: A. Idempotency

Question 9

Which metric may be more appropriate than CPU utilization for scaling an asynchronous worker service?

A. Queue age

B. Image filename length

C. Number of availability zones

D. Operating-system name

Correct Answer: A. Queue age

Question 10

What is a major advantage of immutable images?

A. They eliminate the need for security patches

B. They make deployments more reproducible and reduce configuration drift

C. They guarantee unlimited storage capacity

D. They prevent all hardware failures

Correct Answer: B. They make deployments more reproducible and reduce configuration drift