Computer Architecture, Operating Systems and Virtualisation

Introduction

Computer systems are best understood as layered machines. At the lowest level, electronic circuits represent and manipulate binary values. Above these circuits, processors execute instructions, memory stores active data, and input/output devices connect the computer to users and networks. Operating systems coordinate these resources and provide services to applications. Virtualisation adds another abstraction layer, allowing one physical computer to operate as several logical computers or isolated execution environments.

For information technology professionals, understanding these layers is essential. It supports decisions about server sizing, operating-system selection, cloud deployment, troubleshooting, cybersecurity, performance engineering and infrastructure cost. It also explains why a program may be logically correct but slow, why adding memory can improve performance more than adding processor cores, and why a container is not equivalent to a virtual machine.

Learning Objectives

By the end of this chapter, you should be able to:

• explain the functions of processors, memory, storage and input/output systems;
• distinguish instruction-set architecture from microarchitecture;
• describe operating-system services, process management and file systems;
• compare virtual machines, containers and physical deployment;
• calculate and interpret basic capacity and performance measures;
• identify bottlenecks and apply systematic performance-improvement practices;
• evaluate security, governance and operational implications of virtualised systems.

1. Computer Architecture as a Layered System

Computer architecture describes the organisation and behaviour visible to software, while implementation details are often described as computer organisation or microarchitecture. The instruction-set architecture (ISA) specifies instructions, registers, data types, addressing modes and memory behaviour. Examples include x86-64, ARMv8-A and RISC-V. Microarchitecture describes how a particular processor implements that ISA using pipelines, caches, execution units, branch predictors and control logic.

A program is ultimately transformed into machine instructions. The processor fetches an instruction from memory, decodes it, obtains required operands, executes an operation and writes the result. Modern processors overlap these stages through pipelining and may execute independent instructions out of order. These techniques improve throughput, but they also introduce hazards involving data dependencies, branches and shared resources.

Figure: Abstraction layers in a computer system.

Processors

The central processing unit (CPU) contains arithmetic and logic units, registers, control logic and often several cores. A core can execute an instruction stream; simultaneous multithreading may allow one core to maintain state for multiple threads, although two logical threads do not necessarily provide the performance of two physical cores.

Clock frequency is measured in hertz and indicates cycles per second, not completed instructions per second. A useful simplified model is:

Here, instruction count is the number of executed instructions, CPI is average cycles per instruction, measured in cycles/instruction, and clock rate is cycles/second. The model assumes that the measured CPI includes relevant stalls and that clock rate is stable during the interval. It is an approximation because modern processors vary frequency and execute instructions speculatively or concurrently.

For example, a workload executing 3.0 × 10⁹ instructions at an average CPI of 1.5 on a 3.0 × 10⁹ cycles/second processor requires:

CPU\ time = (3.0 × 10⁹ × 1.5)/(3.0 × 10⁹) = 1.5\ s

Increasing frequency is not the only way to improve performance. Reducing instruction count, improving compiler output, reducing cache misses and increasing useful parallelism may be more effective.

Memory hierarchy

Memory is organised as a hierarchy because fast technologies are expensive and limited in capacity. Registers are fastest and smallest. Processor caches use SRAM and are typically divided into levels. Main memory usually uses DRAM. Persistent storage, such as solid-state drives and hard disks, is slower but retains data when power is removed.

A cache stores copies of recently or frequently used data. Its effectiveness depends on temporal locality, in which recently used data is likely to be reused, and spatial locality, in which nearby data is likely to be accessed. A cache hit is served from the cache; a miss requires data from a lower level.

A simplified average memory access time is:

Hit time is the time to check and access the cache, miss rate is the fraction of accesses that miss, and miss penalty is the additional time to obtain data from the next level. The formula assumes a single cache level and an average, independent penalty; real multilevel caches require an expanded model.

For a cache with a 2 ns hit time, a 5% miss rate and a 60 ns miss penalty, AMAT = 2 + 0.05 × 60 = 5\ ns. This shows why a small reduction in miss rate can be valuable.

Storage and input/output

Storage technologies differ in latency, throughput, durability, cost and failure behaviour. Hard disk drives use rotating magnetic media and have mechanical seek and rotational delays. Solid-state drives use flash memory and have no moving parts, although write amplification, garbage collection and finite program/erase endurance affect performance. Storage systems may use redundancy, such as RAID, but redundancy is not a substitute for backup because deletion, corruption and ransomware can be replicated.

Input/output (I/O) involves communication between the CPU, memory and peripherals. A device controller manages device-specific operations. Direct memory access (DMA) allows a controller to transfer blocks between a device and memory with limited CPU intervention. Interrupts notify the processor when an event requires attention. Polling repeatedly checks device status and may be simpler, but it can waste CPU cycles.

Important I/O measures include latency, throughput, operations per second and queue depth. A system may have high sequential throughput but poor performance for small random requests. Therefore, a storage specification should match the workload rather than rely on a single headline speed.

2. Operating-System Functions

An operating system (OS) is a resource manager and an extended machine. It manages hardware resources while presenting applications with stable abstractions such as processes, files, sockets and virtual memory. Common OS responsibilities include:

• process and thread management;
• memory allocation and protection;
• file and storage management;
• device and driver management;
• networking;
• authentication, authorisation and auditing;
• error handling and accounting;
• user interfaces and system-call services.

The kernel is the privileged core of the OS. User applications run in user mode with restricted access. A system call provides a controlled transition into kernel mode, for example to open a file, create a process or send data through a network socket. This separation limits the damage caused by defective or malicious application code.

Processes and threads

A process is a protected execution environment containing a program, address space, resources and at least one thread. A thread is an execution path within a process. Threads in one process share code, data and open files, but each thread has its own stack and processor state.

A process commonly moves through new, ready, running, blocked and terminated states. A scheduler selects a ready thread for execution. Pre-emptive scheduling allows the OS to interrupt a running thread, improving responsiveness and fairness. Scheduling policies include first-come, first-served; shortest-job approaches; priority scheduling; round robin; and multilevel feedback queues. Each policy involves trade-offs among response time, throughput, fairness and overhead.

Concurrency creates risks. A race condition occurs when the result depends on the timing of unsynchronised accesses. A critical section is code that accesses shared state and must be protected. Mutexes, semaphores, monitors and atomic operations provide synchronisation. Deadlock can arise when processes hold resources while waiting indefinitely for others. The four necessary conditions are mutual exclusion, hold and wait, no pre-emption and circular wait. Prevention, avoidance, detection and recovery are distinct strategies.

Virtual memory

Virtual memory gives each process an address space that is translated to physical memory. Pages provide fixed-size units of virtual memory; page tables map virtual pages to physical frames. A translation lookaside buffer (TLB) caches recent translations. If a referenced page is not in physical memory, a page fault occurs and the OS may load it from storage.

Virtual memory improves protection and enables programs to use an address space larger than available RAM, but excessive paging causes thrashing. Performance depends on working-set size, memory pressure, page-replacement policy and storage latency. Adding RAM may therefore improve performance when the workload is paging, but not when the bottleneck is CPU or I/O bandwidth.

File systems

A file system organises persistent data into files and directories and maintains metadata such as ownership, permissions, size and modification time. File-system designs differ in allocation methods, journaling, consistency, snapshots, encryption and scalability.

A typical file operation passes through several layers: application library, system call, virtual file system, file-system driver, block layer, device driver and storage controller. Buffering and caching may make a write appear complete before data is physically persistent. Applications requiring durability should use appropriate flush or synchronisation mechanisms and understand the guarantees of the storage stack.

Permissions commonly follow a least-privilege principle: users and services receive only the access required for their duties. Access-control lists can provide more detailed rules than simple owner-group-other permissions. Encryption at rest protects stored media, while encryption in transit protects communication; neither alone prevents a compromised authorised process from reading data.

3. Virtualisation

Virtualisation separates a logical computing environment from the physical resources implementing it. A hypervisor, also called a virtual-machine monitor, presents virtual CPUs, memory, disks and devices to guest operating systems. A Type 1 hypervisor runs directly on hardware; a Type 2 hypervisor runs above a host operating system.

A virtual machine (VM) contains a guest OS and applications. The hypervisor schedules virtual CPUs on physical CPUs, maps guest memory to host memory and mediates device access. Hardware-assisted virtualisation improves performance and isolation, but virtualisation still introduces overhead through scheduling, address translation, device emulation and contention.

Memory overcommit allows assigned virtual memory to exceed immediately available physical memory. Techniques such as ballooning, compression and swapping may reclaim memory, but excessive overcommit can cause severe performance degradation. CPU overcommit may be acceptable for bursty workloads but dangerous for latency-sensitive systems.

Containers provide process-level isolation while sharing the host kernel. Namespaces isolate process identifiers, networks, mounts and users; control groups limit and account for CPU, memory and other resources. Containers generally start faster and use fewer resources than VMs because they do not contain a separate guest kernel. However, a kernel vulnerability can have broader consequences for containers sharing that kernel, and containers are not automatically secure merely because they are packaged images.

Orchestration platforms schedule containers across clusters, provide service discovery and restart policies, and support rolling deployment. They do not remove the need for capacity planning, monitoring, identity management, patching and backup.

4. Performance Engineering

Performance analysis should begin with evidence rather than assumptions. Define the service objective, establish a baseline, identify the limiting resource, change one relevant variable where possible, and measure again. Useful indicators include CPU utilisation, run-queue length, memory pressure, page faults, cache-miss rate, disk latency, I/O queue depth, network retransmissions and application response time.

A simple utilisation relationship is:

Both times must use the same units, and the result is normally expressed as a fraction or percentage. A CPU busy for 45 seconds during a 60-second interval has utilisation 45/60 = 0.75 = 75\%. High utilisation is not automatically bad; sustained high utilisation combined with rising response time is stronger evidence of saturation.

Throughput and response time are related but not identical. Throughput measures completed work per unit time, while response time measures delay experienced by a request. Queueing can increase response time before maximum throughput is reached. Amdahl’s law estimates the maximum speed-up from improving only part of a workload:

Here, p is the fraction of execution time that can benefit from an improvement, and s is the speed-up of that part. The formula assumes a fixed workload and no secondary overhead. If 70% of a workload can be accelerated by a factor of 4, the maximum speed-up is 1/((1−0.70)+(0.70/4)) ≈ 2.105. The remaining 30% limits total improvement.

Common optimisation errors include benchmarking unrealistic workloads, ignoring warm-up effects, comparing different software versions, measuring only averages, and changing several variables simultaneously. Percentiles, especially the 95th and 99th, often reveal user experiences hidden by averages.

5. Applied Case: Consolidating a University Laboratory Server

Background

This illustrative case concerns a university laboratory operating separate physical servers for a learning-management database, a web application, a source-code repository and student virtual laboratories. The servers are underutilised at night, but the database experiences slow response during assessment submission periods.

Problem and analysis

Monitoring shows average CPU utilisation below 35%, memory utilisation near 90% and frequent storage queueing on the database server. The initial proposal is to add CPUs. Analysis indicates that the immediate constraints are memory pressure and random storage latency, not processor capacity. The team also identifies weak backup testing and excessive administrative privileges.

Approach

The university deploys a Type 1 hypervisor cluster with redundant hosts and enterprise solid-state storage. The web application and repository are placed in separate VMs. The learning platform is containerised inside a controlled VM to simplify deployment while retaining a distinct guest-kernel boundary. Database memory is reserved, storage I/O is monitored, and resource limits prevent one laboratory workload from consuming the cluster.

Backups follow a separate schedule and are periodically restored to a test environment. Administrative access uses individual accounts, multifactor authentication and role-based permissions. A capacity policy defines alert thresholds for memory pressure, storage latency and backup failure.

Outcome

Database response time improves because sufficient memory is available for caching and storage latency is reduced. Hardware utilisation rises, reducing energy and maintenance costs. However, the university recognises that consolidation increases the impact of a host failure, so it maintains host redundancy, tested backups and documented recovery procedures.

Learning points

The case demonstrates that performance diagnosis must precede procurement, that virtualisation improves flexibility but introduces shared-failure risks, and that operational resilience depends on governance as well as technology. It also shows why containers and VMs should be selected according to isolation, compatibility, portability and operational requirements rather than fashion.

6. Security, Governance and Contemporary Practice

Virtualised environments create additional control points: hypervisors, management consoles, image registries, orchestration systems, virtual networks and secrets stores. A compromised management account may affect many workloads. Best practice includes strong identity controls, network segmentation, signed and scanned images, timely patching, secure configuration baselines, central logging, vulnerability management and tested disaster recovery.

Images should be minimal, versioned and reproducible. Secrets should not be embedded in images or source repositories. Resource limits reduce denial-of-service risk, while admission policies can block privileged containers or untrusted images. Hardware security features, confidential-computing technologies and trusted-boot mechanisms may reduce exposure of workloads, but they introduce compatibility and management trade-offs.

Energy efficiency is also an architectural issue. Consolidation can improve utilisation, while unnecessary overprovisioning increases energy consumption. AI workloads may require accelerators such as GPUs or specialised inference processors. These devices introduce additional scheduling, memory-transfer and virtualisation considerations. Their value should be assessed through measured workload performance, acquisition cost, power use and operational complexity.

A sound professional decision considers total cost of ownership, licensing, portability, skills, compliance, data sovereignty, recovery objectives and supplier dependence. Virtualisation is an enabling technique, not a complete operating model.

Chapter Summary

Computer architecture is a layered system in which processors execute instructions, memory hierarchy reduces access cost, storage preserves data and I/O connects devices. Processor performance depends on instruction count, CPI, clock rate, parallelism and memory behaviour rather than clock frequency alone. Operating systems abstract hardware through processes, virtual memory, file systems, drivers and system calls while enforcing protection and resource allocation.

Processes and threads require scheduling and synchronisation; race conditions and deadlocks must be controlled. File systems manage data, metadata, permissions and persistence. Virtual machines provide guest operating systems through a hypervisor, whereas containers isolate processes while sharing a host kernel. VMs generally offer stronger kernel separation, while containers offer rapid deployment and lower overhead.

Performance engineering requires baselines, appropriate metrics and evidence-based diagnosis. AMAT, utilisation and Amdahl’s law provide useful simplified models, but their assumptions must be recognised. Secure virtualised operations require least privilege, patching, image governance, segmentation, monitoring, backup and recovery testing.

Key Terms

ISA: The instruction-set contract between software and a processor.

Microarchitecture: The internal implementation of an ISA.

Pipeline: Overlapping stages of instruction processing.

Cache: Fast memory storing copies of recently or frequently used data.

CPI: Average processor cycles required per executed instruction.

DMA: Device-controlled transfer between memory and an I/O device.

System call: A controlled request by an application for an OS service.

Kernel: The privileged core of an operating system.

Process: A protected execution environment containing a program and resources.

Thread: An execution path within a process.

Deadlock: Indefinite waiting caused by competing resource dependencies.

Virtual memory: Address translation that separates process addresses from physical memory.

File system: Software that organises files, directories and persistent storage.

Hypervisor: Software or firmware that creates and manages virtual machines.

Virtual machine: A logical computer containing a guest operating system.

Container: An isolated process environment sharing the host kernel.

Namespace: An isolation mechanism for selected system resources.

Control group: A mechanism for limiting and accounting for resource use.

Throughput: Completed work per unit of time.

Amdahl’s law: A model showing how a partial optimisation limits total speed-up.

Review Questions

Short Answer Questions

1. Distinguish instruction-set architecture from microarchitecture.
2. What is the purpose of a processor cache?
3. Explain the difference between a process and a thread.
4. What role does a hypervisor perform?
5. Why is a container not equivalent to a virtual machine?

Long Answer Questions

1. Explain the computer memory hierarchy and discuss how locality affects performance.
2. Describe the principal services provided by an operating system.
3. Compare process scheduling policies and explain the trade-offs among fairness, throughput and response time.
4. Explain virtual memory, page faults and thrashing.
5. Compare physical machines, virtual machines and containers in terms of isolation, overhead and operational use.

Analytical / Application Questions

1. A service executes 2.4 × 10⁹ instructions with an average CPI of 2 on a 3.2 × 10⁹ cycles/second processor. Calculate CPU time and identify one practical way to reduce it.
2. A system has a cache hit time of 3 ns, a miss rate of 4% and a miss penalty of 50 ns. Calculate AMAT and explain the effect of reducing the miss rate to 2%.
3. A proposed server consolidation plan places database, authentication and student workloads on one host. Develop a recommendation addressing resource allocation, failure recovery, security isolation and performance monitoring.

Multiple Choice Questions with Answers

Question 1

Which component defines the instructions and registers visible to software?

A. Instruction-set architecture

B. Cache controller

C. Power supply

D. File allocation table

Correct Answer: A. Instruction-set architecture

Question 2

A processor executes 3 × 10⁹ instructions at a CPI of 2 on a 3 × 10⁹ Hz processor. What is the approximate CPU time?

A. 0.5 s

B. 1 s

C. 2 s

D. 6 s

Correct Answer: C. 2 s

Question 3

What is the main benefit of temporal locality?

A. It increases storage durability

B. It makes recently used data likely to be found in a cache

C. It prevents all page faults

D. It eliminates processor interrupts

Correct Answer: B. It makes recently used data likely to be found in a cache

Question 4

Which OS mechanism provides a controlled transition from user mode to kernel mode?

A. System call

B. Cache miss

C. DMA channel

D. Directory entry

Correct Answer: A. System call

Question 5

Which condition is necessary for deadlock?

A. Unlimited resources

B. Circular wait

C. Read-only files

D. Sequential execution only

Correct Answer: B. Circular wait

Question 6

What does DMA primarily reduce?

A. The need for persistent storage

B. CPU involvement in block data transfers

C. The number of processor registers

D. The size of virtual addresses

Correct Answer: B. CPU involvement in block data transfers

Question 7

Which statement best describes a container?

A. It always contains a complete guest kernel

B. It shares the host kernel while isolating processes and resources

C. It cannot be networked

D. It is a physical replacement for RAM

Correct Answer: B. It shares the host kernel while isolating processes and resources

Question 8

A cache has a 2 ns hit time, a 5% miss rate and a 60 ns miss penalty. What is the AMAT?

A. 2 ns

B. 3 ns

C. 5 ns

D. 62 ns

Correct Answer: C. 5 ns

Question 9

Which practice most directly improves the reliability of backup operations?

A. Increasing CPU clock frequency

B. Testing restoration periodically

C. Disabling access controls

D. Using identical passwords

Correct Answer: B. Testing restoration periodically

Question 10

According to Amdahl’s law, what limits the total speed-up of an optimisation?

A. The part of the workload that cannot be improved

B. The number of file names

C. The size of the monitor

D. The existence of directories

Correct Answer: A. The part of the workload that cannot be improved