Virtualization, Cloud, Edge and AI Hardware

Introduction

Computer systems are no longer designed only as standalone desktops or traditional servers. Modern computing is distributed across data centres, cloud platforms, network edges, sensors, mobile devices and specialised artificial-intelligence (AI) accelerators. These environments depend on hardware mechanisms that provide isolation, efficient resource sharing, high-throughput data movement, security and energy efficiency.

Virtual machines (VMs) allow several operating systems to share one physical computer. Containers provide lighter-weight isolation by sharing an operating-system kernel. Cloud servers combine virtualisation, high-speed networking, large memory systems and automated management. Edge and Internet of Things (IoT) platforms move computation closer to data sources, reducing latency and bandwidth use. Trusted execution technologies protect sensitive processing, while GPUs and neural-processing units accelerate parallel and AI workloads.

This chapter examines the computer-architecture principles behind these technologies and their practical role in digital transformation.

Learning Objectives

After studying this chapter, a learner should be able to:

1. Explain hardware support for virtual machines and containers.
2. Distinguish full virtualisation, paravirtualisation and containerisation.
3. Describe the architecture of cloud servers and data centres.
4. Compare cloud, edge and IoT computing environments.
5. Explain trusted execution environments and hardware roots of trust.
6. Compare CPUs, GPUs, FPGAs and neural-processing accelerators.
7. Select suitable platforms for enterprise and AI workloads.
8. Identify performance, security, energy, governance and operational limitations.

1. Virtualisation as a Computer-Architecture Technique

Virtualisation is the abstraction of physical computing resources so that software can use them as logical resources. A virtual machine appears to have processors, memory, storage and devices, although these resources are supplied by a physical host and controlled by a virtual-machine monitor (VMM), also called a hypervisor.

The main objectives are isolation, consolidation, portability and controlled allocation. For example, a physical server may run a web server, database server and development environment as separate VMs. A failure or software conflict in one VM should not directly compromise the others.

1.1 Hypervisor types

A Type 1 hypervisor runs directly on the hardware. It is common in enterprise servers because it has a small intermediary layer and can control hardware efficiently. A Type 2 hypervisor runs as an application on a conventional host operating system and is common for desktop testing and teaching.

The hypervisor performs several functions:

• schedules virtual CPUs on physical CPU cores;
• maps guest virtual addresses to host physical addresses;
• emulates or exposes virtual devices;
• controls privileged operations;
• allocates memory, storage and network bandwidth; and
• records and enforces isolation and policy decisions.

1.2 CPU support

An operating system normally executes privileged instructions that control memory management, interrupts and devices. If a guest operating system could execute these instructions without supervision, it could interfere with other guests. Hardware-assisted virtualisation introduces a less-privileged guest execution mode and a more-privileged hypervisor mode. A transition, often called a VM exit, occurs when the guest performs an operation requiring hypervisor control.

CPU virtualisation extensions maintain a control structure containing guest and host processor state, interrupt controls and execution permissions. The hypervisor can therefore enter a guest efficiently and regain control when necessary.

Virtual CPU scheduling is related to, but not identical with, ordinary process scheduling. A virtual CPU may be paused because the guest is waiting for I/O, because another VM has a higher priority, or because the hypervisor is managing an overcommitted host.

1.3 Memory virtualisation

A guest operating system believes it controls guest-physical memory, but the machine must ultimately access host-physical memory. This introduces a second address-translation stage:

Modern processors support nested or two-dimensional page translation. Translation lookaside buffers (TLBs) cache recent translations; without such caching, the additional translation would impose significant overhead.

Memory overcommitment allows the total virtual memory assigned to guests to exceed installed RAM, provided that inactive pages can be reclaimed or moved to storage. Techniques include ballooning, page sharing and swapping. Overcommitment improves utilisation but may cause severe performance degradation if many guests simultaneously require their full allocations.

1.4 I/O virtualisation

Virtual devices may be emulated entirely in software, but emulation can be expensive. Paravirtualised drivers allow a guest to communicate through an interface designed for virtualisation, reducing unnecessary emulation. Direct assignment and Single Root I/O Virtualisation (SR-IOV) allow a device, or a virtual function of a device, to be assigned more directly to a VM. This improves performance but can reduce portability and make live migration more difficult.

A virtual switch connects guest network interfaces. It may enforce virtual LANs, security groups, bandwidth policies and software-defined networking rules.

1.5 Migration and availability

Live migration transfers a running VM from one host to another. Memory pages are copied while the source continues executing; pages changed during copying may be transferred again. The VM is briefly paused to copy the remaining state and redirect execution. Migration requires compatible CPU features, shared or replicated storage, adequate network capacity and careful handling of device state.

Virtualisation improves availability through snapshots, replication and automated restart, but it does not eliminate failure. A faulty storage system, hypervisor vulnerability or overloaded shared host can affect many VMs simultaneously.

2. Containers and Operating-System-Level Isolation

A container packages an application and its dependencies while sharing the host operating-system kernel. Unlike a VM, a container generally does not contain a complete guest kernel. This makes containers start quickly and use fewer resources, but it also means that kernel vulnerabilities or incompatible kernel requirements can affect isolation.

Container isolation commonly uses:

• namespaces, which provide separate views of processes, networks, mounts and users;
• control groups (cgroups), which limit and measure CPU, memory, I/O and process resources;
• capabilities, which divide traditional superuser privileges; and
• mandatory access-control systems, which restrict operations according to security policy.

A container image is normally an immutable, layered package. A runtime creates a container from the image, while an orchestration system schedules containers across hosts, restarts failed instances, exposes services and scales workloads.

Containers and VMs can be combined: a cluster may run containers inside VMs to obtain both orchestration convenience and a stronger infrastructure boundary.

3. Cloud Server Architecture

Cloud computing supplies computing resources as an on-demand service. Its technical foundation is a pool of physical servers, storage systems and network fabrics managed by software-defined control planes.

A cloud server commonly contains:

• multicore CPUs with virtualisation extensions;
• large-capacity, often error-correcting (ECC), memory;
• local solid-state storage for high-speed temporary data;
• high-bandwidth network interface cards;
• accelerators such as GPUs or FPGAs;
• redundant power and cooling; and
• management controllers for remote monitoring and recovery.

The data-centre fabric connects servers through leaf-and-spine or similar topologies. East–west traffic occurs between services inside the data centre, while north–south traffic connects users or external systems to the data centre. Modern applications often generate substantial east–west traffic, so network bandwidth and latency are architectural concerns rather than mere connectivity details.

3.1 Service models

Infrastructure as a Service (IaaS) provides virtual machines, networks and storage. Platform as a Service (PaaS) provides managed execution environments, databases or development platforms. Software as a Service (SaaS) provides complete applications. Serverless platforms hide server provisioning and charge according to requests or execution resources, although physical servers remain underneath.

Cloud resource allocation involves multiplexing. A provider uses scheduling, quotas and admission control to share hardware among customers. Isolation is enforced by hypervisors, virtual networks, storage access controls and identity systems. Multi-tenancy improves utilisation but creates shared-resource risks, including noisy-neighbour effects and possible side channels.

3.2 Performance and capacity reasoning

A useful first estimate of average utilisation is:

Both times must use the same unit and interval. For a server with 8 cores, each observed as busy for 60% of a measurement interval, aggregate busy capacity is approximately 4.8 core-equivalents, assuming workloads are divisible and there is no scheduling overhead. This is an estimate, not a guarantee of application performance: memory bandwidth, cache misses, I/O and synchronisation may be limiting factors.

For a workload requiring 2,400 CPU-hours per month, a host offering 8 effective CPU cores continuously would provide approximately:

The theoretical average utilisation would be 2,400 ÷ 5,760 ≈ 41.7%. Actual capacity planning must include peak demand, failures, maintenance and reserve capacity.

4. Edge and IoT Computing

Cloud computing centralises processing; edge computing places processing near users, machines or sensors. The edge may be a cellular site, industrial gateway, retail branch, vehicle, hospital room or factory controller. IoT devices include sensors, actuators, cameras, meters and embedded controllers.

An edge platform usually contains a processor, memory, storage, network interfaces and a device-management agent. More capable gateways may include a GPU, NPU or FPGA. Edge systems often operate intermittently, under restricted power, with limited physical protection and variable connectivity.

Cloud–edge division of work

The cloud is well suited to global coordination, long-term storage, model training, fleet analytics and large-scale orchestration. The edge is well suited to immediate control, filtering, local inference and operation during disconnection. A practical design distributes functions rather than treating cloud and edge as competing alternatives.

Latency has several components:

This additive model applies when the stages occur sequentially. If stages overlap, the total is closer to the critical-path duration than to the simple sum. Local processing can reduce communication time and bandwidth consumption, but edge devices may have less processing capacity and weaker physical security.

IoT systems should use secure boot, signed firmware, unique device identities, encrypted communication, patch management and lifecycle decommissioning. Devices that cannot be updated safely become long-term security liabilities.

5. Trusted Execution and Hardware Security

A trusted execution environment (TEE) is an isolated execution area designed to protect code and data from other software, including parts of the operating system. A TEE typically combines processor-enforced isolation, protected memory, secure boot and cryptographic attestation.

A hardware root of trust is an initially trusted component, such as a secure element, trusted platform module or processor security subsystem. It stores or protects keys and helps verify the next stage of boot software. Secure boot establishes a chain in which each stage verifies the signature or measurement of the next.

Attestation allows a remote party to evaluate the software state of a platform. In measurement-based attestation, cryptographic hashes of firmware and software are recorded in protected registers; a signed report communicates selected measurements to a verifier. Attestation proves a measured state, not that the software is bug-free or that the policy is appropriate.

Confidential computing extends this idea to protect data while it is being processed. Memory-encryption technologies and isolated execution modes can reduce exposure to cloud operators or compromised host software. Limitations include side-channel attacks, denial of service, complex key management, performance overhead and the need to trust the processor vendor and firmware supply chain.

Security is therefore layered: hardware isolation should be combined with least privilege, patching, network controls, secure coding, monitoring and governance.

6. Hardware for AI and Parallel Workloads

AI workloads include training, inference, recommendation, image analysis, speech processing and scientific simulation. They commonly perform large numbers of operations on vectors and matrices. The best processor depends on the workload, precision, memory movement and latency target.

6.1 CPUs

CPUs provide sophisticated control flow, large caches, virtual-memory support and broad software compatibility. They are effective for operating systems, preprocessing, irregular algorithms and small or latency-sensitive inference tasks. Their generality often produces lower peak arithmetic throughput than specialised accelerators.

6.2 GPUs

GPUs contain many arithmetic units designed to execute similar operations across large data sets. Their programming model is effective for matrix multiplication, convolution, graphics and simulation. A GPU system includes device memory and a host-to-device communication path; data-transfer overhead can dominate when tasks are small.

6.3 FPGAs

Field-programmable gate arrays configure hardware datapaths after manufacture. They can provide application-specific pipelines, predictable latency and energy efficiency, but development and verification are more demanding than conventional software development.

6.4 Neural-processing units and tensor accelerators

NPUs and tensor accelerators specialise in operations such as matrix multiplication, convolution and activation functions. They often support reduced-precision arithmetic, including 16-bit or 8-bit formats, to improve throughput and energy efficiency. Reduced precision is useful only when model accuracy remains acceptable and the software stack supports the required operations.

A simplified computational estimate for a dense matrix multiplication of an m × k matrix by a k × n matrix is approximately:

The estimate counts one multiplication and one addition for each term and assumes conventional arithmetic without exploiting sparsity. It excludes data movement, memory stalls and hardware-specific fused operations. For m = k = n = 1,000, the estimate is approximately 2 × 1,000 × 1,000 × 1,000 = 2 × 10⁹ arithmetic operations.

Performance should not be measured only in operations per second. Relevant measures include latency, throughput, joules per inference, memory capacity, memory bandwidth, utilisation, accuracy and total cost of ownership.

7. A Practical Architecture Decision Framework

A platform decision should begin with requirements rather than with a fashionable device. Important questions include:

1. Is the workload interactive, batch-oriented or real-time?
2. What latency and availability are required?
3. How much data must be moved, stored and retained?
4. Is the workload regular and parallel, or irregular and branch-heavy?
5. Can the model use reduced precision without unacceptable error?
6. What security boundary and compliance obligations apply?
7. What power, cooling, physical-space and maintenance limits exist?
8. Is the software ecosystem mature for the chosen accelerator?

A CPU-only solution may be best for low-volume, diverse workloads. A GPU may be suitable for large parallel inference or training. An NPU may be preferable for efficient local inference in a camera or mobile device. An FPGA can be valuable where deterministic latency and a custom streaming pipeline are more important than programming convenience.

8. Applied Case: Illustrative Smart-Factory Inspection System

Background

Consider a fictional manufacturing company that uses cameras to inspect products on a conveyor. The original system uploads every high-resolution image to a central cloud service. The factory has an unreliable wide-area connection, and inspection decisions must be made within 100 milliseconds.

Problem

Uploading all images produces bandwidth costs and variable response time. A cloud outage can stop production. However, central storage is useful for audit records, model improvement and cross-site analysis.

Analysis

The response path includes image capture, preprocessing, inference, communication and actuator control. Sending raw images to the cloud makes communication a large and variable component. The workload is highly parallel and therefore suitable for a local GPU or NPU. The device must also continue operating during temporary disconnection.

Approach

The company installs an edge gateway containing a multicore CPU and an NPU. The gateway performs image resizing, defect inference and immediate actuator control. It sends only defect metadata and selected images to the cloud. The cloud stores records, manages model versions and retrains models using approved data. Signed model packages are verified before deployment. A secure boot chain and encrypted device identity protect the gateway.

A container packages the inference service, while the gateway operating system and device-management layer remain controlled by the manufacturer. Resource limits prevent an analytics container from starving the real-time service. A fallback rule-based inspection mode operates if the model or accelerator fails.

Outcome

The design reduces wide-area bandwidth and makes response time more predictable. It introduces new responsibilities: model monitoring, local patching, physical tamper protection, calibration and safe rollback. The solution is successful because cloud and edge functions are deliberately divided rather than because one location is universally superior.

Learning points

• Latency-sensitive control should not depend entirely on a remote cloud.
• Centralised systems remain valuable for fleet management and training.
• Security must cover boot, identity, software supply chain and operations.
• Accelerator selection must consider data movement and maintainability.
• A safe fallback is essential in industrial systems.

9. Limitations, Governance and Professional Practice

Virtualisation can create performance overhead, resource contention and a large management plane. Containers reduce overhead but do not provide the same isolation assumptions as separate kernels. Cloud platforms can produce vendor lock-in, uncertain data location, variable costs and concentration risk. Edge systems increase the number of physically exposed devices and complicate updates.

AI accelerators can increase energy demand, and their manufacture has environmental costs. Organisations should measure utilisation, idle power, cooling overhead and equipment life rather than comparing only peak performance. Energy efficiency can be expressed as useful inferences per joule, although the measurement must specify model, precision, batch size and accuracy.

Governance should address:

• data protection and lawful processing;
• auditability of automated decisions;
• software and hardware supply-chain assurance;
• access control and separation of duties;
• incident response and vulnerability disclosure;
• retention and deletion of sensor data;
• safety certification for industrial or medical systems; and
• fairness, reliability and human oversight in AI applications.

Best practice includes capacity testing under realistic load, hardware and firmware inventories, staged deployments, signed artefacts, vulnerability scanning, observability, backup and recovery testing, and documented exit plans for cloud services.

Contemporary Developments

Current architecture trends include confidential virtual machines, composable infrastructure, disaggregated memory and storage, smart network interface cards, chiplet-based accelerators, open instruction-set designs, sparse and quantised AI models, and cloud services that expose specialised accelerators on demand. These developments increase flexibility but also make system integration more complex. Performance increasingly depends on the complete path from application and compiler to memory, interconnect, accelerator and cooling system.

Chapter Summary

Virtualisation abstracts processors, memory, storage and devices so that multiple isolated environments can share hardware. Hypervisors rely on privileged execution modes, nested address translation, virtual devices and scheduling. Containers provide lighter isolation by sharing the host kernel and using namespaces, control groups and security policies.

Cloud servers combine virtualisation, high-capacity hardware, software-defined networks, pooled storage and automated control planes. Edge and IoT platforms move selected processing near data sources to reduce latency, bandwidth use and dependence on connectivity. Trusted execution uses secure boot, protected memory, roots of trust and attestation to establish stronger security boundaries, although it does not remove software or supply-chain risk.

CPUs provide versatility, GPUs provide parallel throughput, FPGAs provide configurable pipelines, and NPUs provide efficient neural computation. Platform selection must consider latency, workload structure, data movement, energy, security, software maturity and total cost. Effective digital transformation is therefore an architectural and governance decision, not simply a decision to purchase newer hardware.

Key Terms

Virtualisation: Abstraction that presents physical resources as logical computing resources.

Hypervisor: Software or firmware that creates and manages virtual machines.

Type 1 hypervisor: Hypervisor that runs directly on physical hardware.

Type 2 hypervisor: Hypervisor that runs on a host operating system.

Virtual machine: Isolated software environment containing a guest operating system.

VM exit: Transfer of control from a guest to the hypervisor.

Nested page translation: Hardware-assisted translation from guest virtual addresses to host physical addresses.

Container: Isolated application environment sharing the host kernel.

Namespace: Operating-system mechanism providing an isolated resource view.

Control group: Mechanism for limiting and measuring process resources.

Cloud computing: On-demand provision of pooled computing resources over a network.

Edge computing: Processing performed near the source or consumer of data.

IoT: Network of physical devices that sense, communicate or actuate.

Trusted execution environment: Hardware-supported isolated execution area for protected code and data.

Attestation: Evidence about the measured software state of a platform.

GPU: Processor designed for highly parallel computation.

NPU: Accelerator specialised for neural-network operations.

FPGA: Reconfigurable hardware device used to implement custom datapaths.

Quantisation: Representing values with reduced numerical precision.

Noisy neighbour: Performance interference caused by another tenant sharing resources.

Review Questions

Short Answer Questions

1. What are the main responsibilities of a hypervisor?
2. Why does a VM require a second stage of memory translation?
3. How do containers differ from virtual machines in their treatment of the operating-system kernel?
4. State two advantages and two limitations of edge computing.
5. What is the purpose of remote attestation?

Long Answer Questions

1. Explain how CPU, memory and I/O virtualisation work together to support isolated VMs.
2. Compare cloud, edge and IoT platforms for a latency-sensitive monitoring application.
3. Discuss the security benefits and limitations of secure boot, hardware roots of trust and TEEs.
4. Compare CPUs, GPUs, FPGAs and NPUs for AI workloads, including software and energy considerations.
5. Explain how a cloud provider manages multi-tenancy and discuss the risks of resource contention.

Analytical / Application Questions

1. A company must classify images within 80 milliseconds while its network connection is unreliable. Design a cloud–edge architecture and justify where capture, inference, storage and model training should occur.
2. A host has 8 effective CPU cores and a workload requiring 2,400 CPU-hours in a 30-day month. Calculate the theoretical average utilisation and explain why the result should not be used as the only capacity-planning measure.
3. An organisation plans to place containers from several customers on one host. Identify the isolation mechanisms required and recommend circumstances in which the workloads should instead be placed in separate VMs.

Multiple Choice Questions with Answers

Question 1

Which component directly controls the execution of guest operating systems?

A. Hypervisor

B. Compiler

C. Device battery

D. Application database

Correct Answer: A. Hypervisor

Question 2

What is the main operating-system resource mechanism used to limit container CPU and memory consumption?

A. Namespaces

B. Control groups

C. Page colouring

D. Instruction pipelining

Correct Answer: B. Control groups

Question 3

Which sequence correctly represents virtual-memory translation in a virtual machine?

A. Host physical → guest virtual → guest physical

B. Guest physical → host physical → guest virtual

C. Guest virtual → guest physical → host physical

D. Guest virtual → host virtual → device physical

Correct Answer: C. Guest virtual → guest physical → host physical

Question 4

Which workload is generally most suitable for a GPU?

A. Highly parallel matrix operations

B. A single sequential control routine

C. Firmware signature storage

D. Mechanical power regulation

Correct Answer: A. Highly parallel matrix operations

Question 5

What is the principal reason for placing inference at the edge in the smart-factory case?

A. To eliminate all need for model updates

B. To reduce communication delay and dependence on connectivity

C. To guarantee that no device can fail

D. To replace every cloud function

Correct Answer: B. To reduce communication delay and dependence on connectivity

Question 6

What does remote attestation primarily provide?

A. Proof that a program has no bugs

B. Evidence about a platform's measured software state

C. Automatic replacement of failed hardware

D. Unlimited memory for a VM

Correct Answer: B. Evidence about a platform's measured software state

Question 7

A server has 8 effective cores available continuously for 30 days. Ignoring reserve capacity, how many core-hours does it provide using 24-hour days?

A. 240 core-hours

B. 1,920 core-hours

C. 5,760 core-hours

D. 8,640 core-hours

Correct Answer: C. 5,760 core-hours

Question 8

Which statement best describes an NPU?

A. A storage device for container images

B. A processor specialised for neural-network operations

C. A hypervisor scheduling algorithm

D. A network encryption protocol

Correct Answer: B. A processor specialised for neural-network operations

Question 9

What is a major limitation of containers compared with VMs?

A. Containers cannot run applications

B. Containers always require more memory

C. Containers share the host kernel and therefore have a different isolation boundary

D. Containers cannot be networked

Correct Answer: C. Containers share the host kernel and therefore have a different isolation boundary

Question 10

For a conventional dense m × k by k × n matrix multiplication, what does the estimate 2mkn represent?

A. Approximate arithmetic operations

B. Number of storage devices required

C. Network latency in milliseconds

D. Number of operating-system kernels

Correct Answer: A. Approximate arithmetic operations