8. Interconnects, Input/Output, and Storage Systems

Introduction and Context

A computer system is not merely a collection of processors and memory modules. Its performance depends on how efficiently these components communicate with one another and with the external world. Interconnects carry data, addresses, control information, and synchronization signals. Input/output (I/O) mechanisms connect processors and memory to devices such as networks, displays, disks, sensors, and accelerators. Storage systems preserve information across power cycles and provide a hierarchy of capacity, latency, cost, and reliability.

In early computer systems, a shared bus provided a relatively simple communication medium. As processor counts, memory bandwidth, and peripheral speeds increased, shared buses became bottlenecks. Modern systems therefore combine hierarchical buses, point-to-point links, switched fabrics, and networks-on-chip (NoCs). At the same time, operating systems and virtualization layers must coordinate interrupts, direct memory access, device controllers, storage queues, and protection mechanisms.

The central design problem is a trade-off. A low-latency interconnect may be expensive or difficult to scale. A high-throughput interface may introduce queueing delay and consume substantial power. Redundancy can improve reliability but increases cost and operational complexity. Understanding these trade-offs is essential when designing servers, embedded systems, data centers, high-performance computers, and cloud platforms.

Learning Objectives

After studying this chapter, the reader should be able to:

1. Explain the operation and limitations of buses, point-to-point links, switched fabrics, and NoCs.

2. Distinguish latency, bandwidth, throughput, transaction rate, and queueing delay.

3. Describe peripheral interfaces, device controllers, programmed I/O, interrupts, and DMA.

4. Analyze storage hierarchies, RAID organizations, SSD behavior, and flash-management techniques.

5. Explain I/O virtualization and the role of isolation, queues, and device assignment.

6. Evaluate interconnect and storage choices using performance, scalability, reliability, security, and cost criteria.

From Shared Buses to Scalable Interconnects

A bus is a shared communication path used by multiple agents. A traditional system bus may contain address lines, data lines, and control lines. An initiator, such as a processor, requests a read or write transaction; a target, such as memory or a peripheral controller, responds. Because multiple devices may request access simultaneously, bus arbitration is required. Arbitration may be centralized, distributed, fixed-priority, round-robin, or fairness-aware.

The attractiveness of a shared bus lies in its simplicity. A common protocol and physical medium allow several devices to communicate without dedicated links between every pair. However, only a limited number of transfers can occur concurrently. Electrical loading increases as more devices are attached, and propagation, arbitration, and turnaround delays become significant. A high-speed shared bus can also suffer from head-of-line blocking: a delayed transaction prevents later transactions from using the medium.

A useful first approximation for the time to transfer a message is

T = T_startup + T_serialization + T_propagation + T_queue,

where startup time includes protocol and arbitration overhead, serialization time is message size divided by link bandwidth, propagation time is the signal travel delay, and queueing time reflects contention. Increasing bandwidth primarily reduces serialization time; it does not necessarily reduce startup or queueing delay.

Point-to-point interconnects avoid a single shared electrical path. Each link connects two endpoints, and multiple transfers may proceed simultaneously. A switched fabric uses routers or switches to establish paths between endpoints. Examples of design concepts include separate lanes, packet framing, credit-based flow control, virtual channels, and quality-of-service classes. Packetization improves scalability and fault isolation, although headers, buffering, routing, and reassembly introduce overhead.

A fabric must address flow control. In credit-based flow control, a sender transmits only when the receiver advertises available buffer space. In backpressure systems, a congested receiver signals the sender to slow down. Without flow control, a fast source can overrun a slower destination, causing data loss or requiring expensive retransmission.

Interconnect performance should not be described by bandwidth alone. Latency matters for fine-grained synchronization and small messages; bandwidth matters for bulk transfers. A system may have high peak bandwidth but poor application throughput if transactions are small, queues are congested, or software overhead dominates. Designers therefore consider average and tail latency, sustained throughput, fairness, utilization, and energy per transferred bit.

Networks-on-Chip

A multicore processor may contain dozens or hundreds of cores, cache banks, memory controllers, accelerators, and I/O agents. A conventional shared bus does not scale well in this environment. A network-on-chip treats on-chip communication as a miniature packet network. Routers connect using topologies such as mesh, torus, ring, tree, or custom hierarchical structures.

A two-dimensional mesh is popular because it is regular and relatively easy to place physically. Its disadvantages include increasing hop count for distant nodes and possible congestion near heavily used memory controllers. Topology selection therefore depends on traffic patterns, floorplan constraints, wiring resources, and fault-tolerance requirements.

NoC routing may be deterministic, adaptive, minimal, or non-minimal. Deterministic routing is simple and predictable, whereas adaptive routing can avoid congested or faulty links. Virtual channels allow packets sharing a physical link to be separated logically, reducing certain forms of deadlock and improving traffic isolation. Deadlock avoidance requires careful ordering of resource acquisition or a routing algorithm that prevents cyclic dependencies.

NoC design also exposes a systems issue: computation and communication are inseparable. Adding cores without sufficient network bandwidth may produce little application benefit. Conversely, an overprovisioned network consumes area and power. Contemporary designs may use traffic monitoring, dynamic voltage and frequency scaling, approximate communication, and application-specific accelerators to balance energy and performance.

Peripheral Interfaces and Device Controllers

Peripheral interfaces define how external devices communicate with a computer. They specify electrical signaling, packet or frame formats, addressing, timing, error detection, reset behavior, and software-visible registers. Examples include serial links, memory-mapped peripheral buses, storage interfaces, and high-speed expansion fabrics.

A device controller is the hardware intermediary between a peripheral and the processor-memory system. It commonly contains configuration registers, status registers, command queues, data buffers, error flags, and interrupt-generation logic. The operating system driver translates general operations—such as read, write, open, or configure—into controller-specific commands.

Memory-mapped I/O assigns device registers addresses in the processor’s address space. Loads and stores to those addresses cause device transactions rather than ordinary memory accesses. This approach simplifies the programming model but requires correct ordering, access permissions, and handling of side effects. A read of a status register may clear a flag, while a write to a command register may start an irreversible operation.

Programmed I/O requires the processor to poll device status and move data itself. Polling is straightforward and may be efficient when events are extremely frequent and predictable, but it wastes processor cycles when devices are slow or idle. Interrupt-driven I/O allows a controller to notify the processor when service is needed. Interrupts reduce unnecessary polling but introduce context-switch, prioritization, and interrupt-handling overhead.

An interrupt controller prioritizes, masks, routes, and sometimes coalesces interrupts. Interrupt moderation groups multiple events into one notification, reducing overhead at the cost of additional latency. This is useful for high-speed networking and storage, where one interrupt per packet or request would overwhelm the processor.

DMA permits a controller to transfer data directly between a device and memory. The processor configures a descriptor or command, and the DMA engine performs the transfer. After completion, the controller may issue an interrupt. DMA substantially reduces processor involvement, but it raises correctness and security issues. Caches must remain coherent with DMA buffers, memory barriers may be required, and an untrusted device must not access arbitrary physical memory.

An IOMMU translates device-visible addresses and enforces access permissions. It provides isolation between devices or virtual machines and can support scatter-gather transfers, in which non-contiguous physical memory is presented as a logically continuous buffer. DMA is therefore not simply a performance feature; it is part of the memory-protection architecture.

Storage Hierarchy and Its Design Logic

Storage is organized as a hierarchy because no single technology simultaneously provides minimal latency, very large capacity, low cost, persistence, and high reliability. Registers and caches are fastest but small and volatile. Main memory offers greater capacity but remains volatile. Solid-state drives and hard disks provide persistence. Remote and archival storage offer capacity and durability with higher access latency.

The hierarchy works because programs exhibit locality. Temporal locality means recently used data is likely to be used again; spatial locality means nearby data is likely to be accessed. Caches exploit both forms. Storage systems also use buffering, read-ahead, write-back caching, request scheduling, and batching.

Important measures include:

• Latency: time from request submission to completion.
• Bandwidth: maximum data-transfer rate.
• IOPS: completed input/output operations per second.
• Queue depth: number of outstanding requests.
• Tail latency: a high percentile, such as the 99th percentile, of response time.
• Durability: ability to retain data over time.
• Availability: probability that the service is accessible when required.
• Endurance: amount of data that can be written before a device reaches its specified wear limit.

A small random read may be latency-bound, while a large sequential transfer may be bandwidth-bound. Increasing queue depth can improve device utilization and throughput, but excessive queueing increases response time. This tension is especially important in interactive and real-time systems.

RAID and Dependable Storage

Redundant Array of Independent Disks (RAID) combines multiple drives to improve capacity, performance, reliability, or some combination of these. RAID 0 stripes data across drives without redundancy; it can deliver high throughput but provides no protection against drive failure. RAID 1 mirrors data, allowing continued operation after a drive failure but requiring approximately twice the raw capacity. RAID 5 uses distributed single parity and can tolerate one drive failure. RAID 6 uses dual parity and can tolerate two drive failures, although parity calculations and rebuilds add overhead.

RAID 10 combines mirroring and striping. It generally offers strong random-write performance and relatively straightforward recovery, but usable capacity is lower than in parity-based arrangements. RAID is not a substitute for backup. It protects against selected hardware failures, not accidental deletion, malware, application corruption, or site-wide disasters.

Rebuild behavior is a critical design concern. After a drive failure, the system reconstructs missing data from surviving data and parity. During rebuilding, performance may decline and the array may be exposed to additional failures. Larger drives increase rebuild time and therefore influence the practical reliability of parity arrays. Reliability analysis must consider failure independence, unrecoverable read errors, latent sector errors, monitoring quality, and backup strategy.

Solid-State Drives and Flash Translation

SSDs use nonvolatile flash memory rather than rotating magnetic media. They avoid seek and rotational delays and can service many concurrent requests. However, flash cells cannot generally be overwritten in place. Data is written in pages, while erasure occurs in larger blocks. The SSD controller therefore maintains a flash translation layer (FTL) that maps logical block addresses to physical flash locations.

The FTL performs garbage collection, wear leveling, bad-block management, error correction, and sometimes compression or deduplication. Garbage collection copies valid pages from partially used blocks and erases the blocks for reuse. This creates write amplification: the physical amount written may exceed the data requested by the host. Overprovisioning supplies spare space for efficient garbage collection and replacement of worn cells.

Flash memory has finite program/erase endurance. Wear leveling distributes writes across cells, while error-correcting codes compensate for increasing raw bit-error rates. SSD performance may vary substantially with workload, temperature, queue depth, fullness, and the presence of a sustained write stream. A temporary high-speed cache can make short benchmarks misleading; sustained testing is required for engineering decisions.

I/O Virtualization

Virtual machines require access to storage, networks, GPUs, and other devices while remaining isolated from one another. In software-emulated I/O, a hypervisor or host process presents a virtual device and translates guest operations. This provides flexibility and strong control but may impose substantial CPU and latency overhead.

Paravirtualized interfaces expose an efficient guest-aware protocol, often using shared memory rings and descriptor queues. The guest and host exchange references to buffers rather than copying every payload. Hardware-assisted techniques can further reduce overhead. Direct device assignment gives a virtual machine exclusive access to a physical device, while a mediated device may partition one device among several guests.

I/O virtualization must balance performance with isolation. Device assignment can approach native performance but reduces sharing flexibility. Shared devices improve utilization but require robust scheduling, address translation, and protection. Technologies such as IOMMUs, interrupt remapping, hardware queue virtualization, and single-root I/O virtualization address these concerns.

Virtualized storage also introduces multiple queues: application queues, guest queues, hypervisor queues, host kernel queues, controller queues, and device-internal queues. End-to-end latency can therefore be difficult to predict. Performance analysis must identify where queueing occurs rather than attributing all delay to the physical device.

Applied Case: Designing Storage and I/O for a Virtualized Analytics Platform

Background

Consider an organization operating a virtualized analytics platform. Several virtual machines ingest network data, transform it, and write intermediate results. The platform must support high sustained throughput, protect tenant data, and remain available after a drive failure.

Situation and Problem

Initial measurements show acceptable average throughput but poor 99th-percentile write latency. During large ingestion bursts, one tenant’s workload delays others. A single parity-based storage pool is also rebuilding after a failed drive, further reducing performance.

Analysis

The problem is not simply insufficient disk bandwidth. Several queues exist between the guest application and the storage media. Large ingestion requests compete with small metadata writes. Parity updates create read-modify-write operations, and rebuild traffic consumes device and interconnect bandwidth. The virtual machines also share CPU resources for storage processing.

Approach

The design team adopts separate storage classes. High-priority metadata and transactional writes use mirrored SSDs, while large sequential intermediate data uses a striped, parity-protected pool. Virtualized storage queues are assigned per tenant, with limits on queue depth and bandwidth. The host uses interrupt moderation and polling selectively, while DMA buffers are protected by the IOMMU. Monitoring records throughput, IOPS, queue depth, tail latency, write amplification, and rebuild progress.

Backups are maintained independently of RAID. Failure testing includes drive removal, controller restart, virtual-machine migration, and recovery from corrupted data. Workloads are benchmarked under steady-state conditions rather than only during the SSD’s short burst-performance period.

Outcome

Separating workloads prevents metadata operations from being trapped behind large transfers. Queue limits improve fairness, and mirrored devices reduce write latency for sensitive operations. The parity pool remains cost-effective for bulk data, while independent backups address failures that RAID cannot handle.

Learning Points

The case demonstrates that system performance is end-to-end. Faster media alone may not solve queueing, virtualization, parity, or scheduling problems. Storage architecture should be matched to workload characteristics, and reliability must include recovery procedures, not merely redundancy.

Challenges, Risks, and Professional Considerations

Common design mistakes include comparing peak interface bandwidth with application throughput, ignoring tail latency, using RAID as a backup strategy, and testing SSDs only with short benchmarks. Another error is increasing queue depth without measuring the resulting response-time degradation. Poor interrupt affinity can cause a single processor core to become a bottleneck even when the device has unused capacity.

Security risks include malicious or faulty DMA, unauthorized device access, firmware compromise, data remanence, insecure device resets, and leakage between virtual machines. Storage encryption protects confidentiality but introduces key-management responsibilities and may affect performance. Sanitization of flash storage is complicated by overprovisioning and remapped blocks; cryptographic erase, when correctly implemented, can be more dependable than simple overwriting.

Governance requires clear retention policies, audit trails, access controls, incident-response procedures, and tested recovery objectives. Designers should distinguish recovery point objective—the acceptable amount of data loss—from recovery time objective—the acceptable restoration time. Hardware choices must also consider energy consumption, cooling, e-waste, vendor dependence, and lifecycle support.

Current Developments and Best Practices

Contemporary systems increasingly use coherent interconnects, chiplet-based packaging, high-speed memory fabrics, computational storage, programmable data-processing units, and disaggregated storage. NoCs are becoming more adaptive and application-aware. Storage-class memory and persistent-memory technologies seek to reduce the boundary between memory and storage, although programming models, endurance, consistency, and recovery semantics remain important issues.

Best practice is to begin with workload characterization: request-size distribution, read/write ratio, locality, burstiness, concurrency, latency objectives, and failure requirements. The design should then:

• separate control-plane and data-plane traffic where appropriate;
• use DMA and IOMMU protection for high-volume device transfers;
• measure sustained throughput and tail latency;
• control queue depth and apply quality-of-service policies;
• select RAID or replication according to failure and recovery objectives;
• maintain tested, independent backups;
• monitor errors, wear, temperature, rebuilds, and firmware health; and
• validate behavior under overload, failure, migration, and recovery.

Chapter Summary

Interconnects determine how processors, memory, accelerators, and devices exchange information. Shared buses are simple but limited by arbitration, contention, and electrical scalability; point-to-point links and switched fabrics support greater concurrency. Networks-on-chip extend these principles inside multicore processors, where topology, routing, flow control, deadlock avoidance, and power are central concerns.

Peripheral controllers expose registers, queues, buffers, and status information to software. Programmed I/O, interrupts, and DMA provide progressively more efficient ways to manage transfers, while IOMMUs protect memory from unauthorized device access. Storage hierarchies exploit locality and balance latency, throughput, capacity, cost, endurance, and persistence. RAID improves selected forms of availability but is not a replacement for backup. SSDs provide low latency but depend on flash translation, garbage collection, wear leveling, and error correction. I/O virtualization enables sharing and isolation, though it introduces additional queues and software layers. Sound design requires end-to-end measurement, workload-aware architecture, explicit reliability objectives, and security-conscious administration.

Key Terms

Bus: A shared communication medium carrying data, address, and control signals.

Bus arbitration: The process of selecting which requester may use a shared bus.

Switched fabric: An interconnection of switches and links that enables concurrent packet transfers.

Latency: Time between initiating an operation and receiving its completion.

Throughput: The amount of useful work or data completed per unit time.

Network-on-chip: A packet-switched communication network integrated into a processor chip.

Virtual channel: A logically independent channel sharing a physical interconnect link.

Device controller: Hardware that manages a peripheral and exposes a software interface.

Memory-mapped I/O: Access to device registers through addresses in the processor address space.

Interrupt: A notification requesting processor service for a device event.

DMA: Direct movement of data between a device and memory with limited processor involvement.

IOMMU: Hardware that translates and restricts device-generated memory addresses.

Queue depth: The number of outstanding operations awaiting service.

RAID: A method of combining drives for performance, capacity, or redundancy.

Write amplification: The ratio of physical storage writes to host-requested writes.

Wear leveling: Distribution of flash writes to extend device endurance.

Tail latency: A high-percentile response time, such as the 99th percentile.

I/O virtualization: Presentation and management of virtual devices for isolated software environments.

Recovery time objective: The maximum acceptable time to restore a service.

Recovery point objective: The maximum acceptable amount of data loss measured in time.

Review Questions

Short Answer Questions

1. Why does a shared bus become difficult to scale as the number of agents increases?

2. Distinguish latency from throughput in an interconnect.

3. What functions are performed by a device controller?

4. Why is an IOMMU important for DMA security?

5. What is write amplification in an SSD?

Long Answer Questions

1. Explain the progression from shared buses to switched fabrics and networks-on-chip.

2. Compare programmed I/O, interrupt-driven I/O, and DMA with respect to processor overhead and latency.

3. Explain how RAID 0, RAID 1, RAID 5, RAID 6, and RAID 10 differ in performance and fault tolerance.

4. Describe the operation of an SSD flash translation layer, including garbage collection and wear leveling.

5. Discuss the performance and isolation trade-offs of software-emulated, paravirtualized, and directly assigned I/O.

Analytical / Application Questions

1. A storage system has high peak bandwidth but unacceptable 99th-percentile latency. Identify likely causes and propose measurements and remedies.

2. Design an interconnect and storage strategy for a multi-tenant analytics platform with high sequential writes, small metadata updates, and strict tenant isolation.

3. During a RAID rebuild, application latency rises sharply. Analyze the interacting effects of rebuild traffic, queue depth, parity operations, and virtualization, and recommend mitigations.

22. Multiple Choice Questions with Answers

Question 1

Which factor most directly limits concurrency in a traditional shared bus?

A. Every transaction must use the same shared communication medium

B. Flash cells require block erasure

C. The IOMMU translates device addresses

D. RAID mirrors data across drives

Correct Answer: A. Every transaction must use the same shared communication medium

Question 2

A workload transfers large sequential blocks but experiences high queueing delay. Which change is most likely to reduce delay without changing the media?

A. Increase queue depth indefinitely

B. Apply queue limits or quality-of-service scheduling

C. Disable all flow control

D. Replace DMA with programmed I/O

Correct Answer: B. Apply queue limits or quality-of-service scheduling

Question 3

What is the primary security role of an IOMMU?

A. Compressing storage data

B. Scheduling SSD garbage collection

C. Restricting and translating device memory accesses

D. Increasing flash endurance

Correct Answer: C. Restricting and translating device memory accesses

Question 4

Which I/O method allows a device to transfer data directly to memory with limited processor copying?

A. Polling

B. Memory-mapped I/O

C. Interrupt masking

D. DMA

Correct Answer: D. DMA

Question 5

Why can a high queue depth improve throughput but worsen application responsiveness?

A. It eliminates all device parallelism

B. It increases outstanding work and may increase waiting time

C. It prevents controllers from using buffers

D. It converts random operations into sequential ones

Correct Answer: B. It increases outstanding work and may increase waiting time

Question 6

Which RAID level provides striping without redundancy?

A. RAID 0

B. RAID 1

C. RAID 5

D. RAID 6

Correct Answer: A. RAID 0

Question 7

What causes SSD write amplification?

A. The host always writes identical data twice

B. Physical flash management writes more data than the host requested

C. The interface has no packet headers

D. The SSD performs no garbage collection

Correct Answer: B. Physical flash management writes more data than the host requested

Question 8

Which technique most directly reduces interrupt overhead for high-rate devices?

A. Interrupt moderation or coalescing

B. Disabling device buffers

C. Using only polling at all rates

D. Removing controller queues

Correct Answer: A. Interrupt moderation or coalescing

Question 9

What is a principal advantage of a network-on-chip over a shared on-chip bus?

A. It eliminates the need for routing

B. It permits greater communication concurrency through multiple links

C. It guarantees zero latency

D. It requires no flow control

Correct Answer: B. It permits greater communication concurrency through multiple links

Question 10

Why is RAID not a complete backup strategy?

A. RAID cannot improve read performance

B. RAID protects against every software failure

C. RAID does not generally protect against deletion, corruption, or site-wide disasters

D. RAID always reduces storage capacity to zero

Correct Answer: C. RAID does not generally protect against deletion, corruption, or site-wide disasters