I/O Systems and Secondary Storage

Introduction

Input/output (I/O) systems connect the processor and memory to the external world. They include keyboards, displays, network interfaces, sensors, printers, disks, solid-state drives, and persistent-memory devices. Although I/O devices differ greatly in speed, data representation, and control mechanisms, operating systems provide a common framework for using them safely and efficiently.

I/O is challenging because processors operate at nanosecond-scale speeds while many devices operate much more slowly. Devices may also generate events unpredictably, transfer data in blocks or streams, and fail independently of the CPU. The operating system therefore uses device controllers, interrupts, direct memory access, drivers, buffering, caching, spooling, and scheduling algorithms to manage this mismatch.

Secondary storage is especially important because it preserves data when power is removed. This chapter examines the architecture and management of I/O systems, magnetic disks, solid-state storage, RAID, persistent memory, and storage reliability.

Learning Objectives

By the end of this chapter, you should be able to:

1. Explain the roles of device controllers, device drivers, interrupts, and DMA.

2. Distinguish programmed I/O, interrupt-driven I/O, and DMA.

3. Explain buffering, caching, and spooling and select appropriate uses for each.

4. Describe disk and SSD organization and compare scheduling strategies.

5. Explain RAID levels, trade-offs, and failure-recovery mechanisms.

6. Evaluate persistent-memory technologies and storage reliability practices.

7. Analyze an I/O architecture for performance, dependability, and security.

1. The I/O System Architecture

A typical I/O path contains several layers:

• Application: requests operations such as read, write, open, or print.
• System-call interface: validates requests and transfers control into the kernel.
• Device-independent I/O layer: provides common naming, protection, buffering, and error handling.
• Device driver: translates generic operations into device-specific commands.
• Controller: electronic hardware that operates the device and exposes registers or queues.
• Interconnect: such as PCI Express, USB, SATA, or a storage fabric.
• Device: performs the physical or electronic operation.

This layered design allows applications to use a file or socket without knowing whether the underlying hardware is a hard disk, SSD, or network device. The abstraction improves portability, but it can hide performance differences. A sequential read from a magnetic disk, for example, has very different latency characteristics from a random read from an NVMe SSD.

Device controllers

A device controller is hardware that manages a particular device or class of devices. It commonly contains:

• Control registers, through which the operating system issues commands.
• Status registers, indicating readiness, errors, or completion.
• Data registers or queues, holding data or command descriptors.
• Local buffer memory, absorbing speed differences.
• Control logic, implementing the device protocol.

Modern controllers often support command queues, scatter-gather lists, error correction, encryption, and independent processing. The operating system should not assume that completion means data is safely stored on nonvolatile media. A controller or drive may acknowledge data while it remains in volatile cache. Explicit flush or force-unit-access mechanisms are used when durability is required.

Device drivers

A device driver is privileged software that controls a device and presents a standard kernel interface. It initializes the controller, constructs commands, maps buffers, handles interrupts, reports errors, and exposes capabilities.

A good driver must validate all hardware-generated values before using them, prevent user processes from directly controlling privileged devices, and handle malformed or delayed responses. Driver bugs are particularly serious because drivers execute in the kernel and can compromise the entire system. Modern systems increasingly use hardware isolation, signed drivers, input-output memory management units (IOMMUs), and least-privilege architectures to reduce this risk.

2. Methods of Data Transfer

Programmed I/O

With programmed I/O, the CPU repeatedly reads a status register until the device becomes ready, then transfers data through a register. This is simple but wastes processor cycles during waiting. It is suitable for very small or infrequent operations, but inefficient for large transfers or slow devices.

Interrupt-driven I/O

In interrupt-driven I/O, the CPU starts an operation and continues other work. When the device requires attention or completes the operation, its controller sends an interrupt. The processor suspends the current execution context, runs an interrupt handler, acknowledges the device, records the result, and resumes the interrupted work.

Interrupts avoid continuous polling, but they have costs: saving and restoring state, entering privileged code, cache disruption, and scheduling follow-up work. A very high interrupt rate can overload the processor. Operating systems therefore combine interrupt moderation, batching, and deferred processing. Network interfaces, for example, may handle an initial event in a top-half handler and process many packets later in a bottom-half or kernel-thread context.

Interrupts may be maskable or non-maskable. Priority mechanisms determine which event is handled first. Correct interrupt handling requires synchronization because an interrupt handler may access data also used by processes or other kernel threads.

Direct Memory Access

Direct memory access (DMA) permits a controller to transfer a block directly between a device and main memory. The CPU sets up the operation by specifying a memory address, length, direction, and device command. The controller then performs the transfer and normally interrupts the CPU when it completes.

DMA substantially reduces CPU involvement, but it raises protection and consistency issues. The operating system must ensure that the device accesses only authorized memory. An IOMMU translates device-visible addresses and can prevent a faulty or compromised device from reading arbitrary memory. Systems with caches must also maintain coherence between CPU caches and DMA buffers; this may be provided by hardware or by explicit cache flush and invalidation operations.

DMA buffers may be physically contiguous, or the driver may provide a scatter-gather list containing several memory regions. Scatter-gather DMA avoids unnecessary copying when an application buffer is fragmented in physical memory.

Comparison

Programmed I/O is easy to implement but consumes CPU time. Interrupt-driven I/O is efficient for sporadic events and moderate transfers. DMA is generally preferred for large transfers because it minimizes per-byte CPU work. These approaches are not mutually exclusive: a device may use DMA for payload data and interrupts for completion notification.

3. Managing Data Movement

Buffering

A buffer is a temporary memory area used while data moves between components operating at different rates or granularities. Buffering can:

• absorb bursts;
• convert between block and stream representations;
• permit an operation to proceed asynchronously;
• combine small writes into larger transfers; and
• support producer-consumer pipelines.

A single buffer may be adequate when the producer and consumer do not overlap. Double buffering permits one buffer to be filled while another is processed. Ring buffers are effective for continuous streams such as audio, network packets, and sensor data.

Buffering does not make the underlying device intrinsically faster. It improves utilization and responsiveness, but it consumes memory and can increase latency if data waits in a queue.

Caching

A cache stores copies of data that are likely to be reused. A file-system cache may keep recently read disk blocks in RAM, avoiding repeated device access. Caching exploits temporal locality, while read-ahead exploits sequential locality by fetching blocks before they are requested.

Caching creates a consistency problem: the cached copy may differ from the storage copy. Write-through caching sends updates to the device immediately, offering stronger durability but potentially lower performance. Write-back caching delays the device write, improving performance but risking data loss after a crash. A flush operation requests that dirty data be written; however, durable ordering also depends on the device, controller, and storage protocol.

The operating system should avoid redundant caching where possible. For example, database systems may use direct I/O or carefully coordinate with the page cache to prevent two independent caches from consuming memory and making conflicting replacement decisions.

Spooling

Spooling places output in a queue, usually on disk, for a device that services requests sequentially. Printing is the classic example. Applications submit print jobs and continue; a spooler later schedules them for the printer. Spooling decouples application execution from device availability and permits job ordering, cancellation, quotas, and accounting.

A spool contains potentially sensitive information. Access control, encryption, retention limits, and secure deletion policies are therefore important in institutional and cloud environments.

4. Magnetic Disk Storage

A hard disk drive (HDD) stores data magnetically on rotating platters. Each surface contains concentric tracks divided into sectors. A head assembly moves to a selected track, and the platter rotates until the required sector is beneath the head.

Disk access time is commonly analyzed as:

Access time = seek time + rotational latency + transfer time + controller/queueing overhead.

Seek time is the time needed to position the head. Rotational latency is the waiting time for the desired sector to rotate into position. If the disk rotates at a constant speed of R revolutions per minute, the time for one revolution is 60/R seconds. Under the simplifying assumption that requests are uniformly distributed around a track, the average rotational latency is half a revolution:

Average rotational latency = 30/R seconds.

For a 7,200-RPM disk, one revolution takes 60/7200 = 0.00833 seconds, or 8.33 ms. The average rotational latency is therefore approximately 4.17 ms. This estimate excludes seek, transfer, and queueing time.

Disk scheduling seeks to reduce head movement while maintaining fairness. Important algorithms include:

• FCFS: services requests in arrival order. It is fair and simple but may cause excessive movement.
• SSTF: selects the request with the shortest seek distance from the current head position. It often reduces average seek time but can starve distant requests.
• SCAN: moves in one direction servicing requests, then reverses, resembling an elevator.
• C-SCAN: services requests in one direction only, then returns to the beginning without servicing during the return. It offers more uniform waiting times.
• LOOK and C-LOOK: like SCAN and C-SCAN, but reverse or wrap at the last pending request rather than at the physical disk end.

Scheduling is less important for SSDs because they have no mechanical head. Nevertheless, request merging, queue management, fairness, and workload-aware scheduling remain relevant.

Worked scheduling example

Suppose a disk head is initially at cylinder 50 and pending requests are 82, 170, 43, 140, 24, and 16. If FCFS is used, the total movement is:

| Movement | Distance |

|---|---:|

| 50 to 82 | 32 |

| 82 to 170 | 88 |

| 170 to 43 | 127 |

| 43 to 140 | 97 |

| 140 to 24 | 116 |

| 24 to 16 | 8 |

| Total | 468 cylinders |

If SSTF is used, the order is 43, 24, 16, 82, 140, 170. The movement is 7 + 19 + 8 + 66 + 58 + 30 = 188 cylinders. SSTF reduces movement for this snapshot, but a continuous stream of requests near the current head could delay a request at the opposite side of the disk. Thus, performance and fairness must be considered together.

5. Solid-State Drives

An SSD stores data in flash memory rather than rotating magnetic media. NAND flash is organized into pages and erase blocks. Reads and writes occur at page granularity, whereas erasure generally occurs at the larger block granularity. Flash cells wear because program and erase operations gradually degrade them.

An SSD controller uses a flash translation layer (FTL) to map logical block addresses to physical flash locations. The FTL performs wear leveling, garbage collection, bad-block management, error correction, and often encryption. Because a used page cannot simply be overwritten in place, the controller writes new data elsewhere and later erases blocks containing obsolete pages.

Important SSD operations include:

• Wear leveling: distributes writes across cells to avoid premature failure of frequently used locations.
• Garbage collection: copies still-valid pages from partially used blocks and erases the blocks.
• TRIM or discard: informs the device that logical blocks are no longer needed, improving garbage collection.
• Over-provisioning: reserves physical flash capacity for replacement, wear leveling, and efficient garbage collection.
• Error-correcting codes: detect and correct bit errors caused by cell wear and retention loss.

SSD latency is usually much lower than HDD latency, but it is not constant. Queue depth, garbage collection, thermal throttling, write amplification, and sustained workload patterns affect performance. Random writes can be particularly expensive because they may trigger read-modify-write operations and block erasures.

A scheduling policy for SSDs should therefore consider queue depth, fairness, request deadlines, parallel flash channels, and write amplification rather than merely minimizing geometric distance.

6. RAID and Storage Redundancy

RAID, or redundant array of independent disks, combines multiple drives to improve performance, capacity, reliability, or some combination of these objectives. RAID is not a substitute for backup: accidental deletion, malware, corruption, and site-level disasters can affect all members of an array.

• RAID 0: stripes data across drives without redundancy. It improves parallel throughput but has no fault tolerance.
• RAID 1: mirrors data. Usable capacity is approximately half the raw capacity of two equally sized drives, while one drive can fail without data loss.
• RAID 5: uses block-level striping with distributed single parity. It can tolerate one drive failure, but degraded reads and reconstruction impose performance and reliability costs.
• RAID 6: uses dual parity and can tolerate two simultaneous drive failures, at the cost of additional capacity and write overhead.
• RAID 10: mirrors pairs and stripes across them. It offers strong performance and redundancy, although the failure tolerance depends on which drives fail.

For a RAID 5 array with N equal-sized drives, usable capacity is approximately (N − 1) times the capacity of one drive, assuming the array is large enough for the chosen implementation. For RAID 6, it is approximately (N − 2) times one-drive capacity. These simplified capacities ignore formatting and metadata overhead.

Reliability and the rebuild problem

When a drive fails, the array reconstructs its missing contents from surviving data and parity. During reconstruction, the array is exposed to additional failures, and performance is reduced. Large modern drives can make rebuilds lengthy. Unrecoverable read errors, latent sector errors, and a second drive failure can prevent successful recovery, especially in single-parity systems.

Professional storage design therefore combines RAID with monitoring, hot spares, scrubbing, tested backups, and failure-domain awareness. Replication across independent servers or locations may be preferable to a single large local array.

7. Persistent Memory and Storage-Class Memory

Persistent memory occupies a conceptual position between conventional RAM and storage. It is byte-addressable or comparatively low-latency like memory, yet retains data without power. Technologies and products have included battery-backed DRAM, nonvolatile DIMMs, and emerging resistive, phase-change, and other memory technologies.

Persistent memory creates new design questions. A program may need to ensure that modified cache lines reach the persistence domain and that writes become durable in the correct order. Atomicity, crash consistency, wear, encryption, and access permissions must be addressed. A data structure that is consistent in volatile memory may be inconsistent after a crash if only some of its updates became persistent.

Operating systems may expose persistent memory as a fast block device or as a directly mapped byte-addressable region. The block interface preserves familiar file-system abstractions; direct access can reduce copying and system-call overhead but places greater responsibility on applications and libraries.

Persistent memory is relevant to AI and ML systems because model checkpoints, embedding indexes, feature stores, and large datasets may benefit from faster restart and lower data-movement overhead. However, its value depends on workload locality, durability requirements, cost, and software support rather than latency alone.

8. Storage Reliability, Durability, and Security

Reliability means performing correctly over time; availability means being ready for use; durability means preserving acknowledged data despite failures. These properties are related but distinct. A highly available replica may still contain corrupted data, and a durable backup may not provide immediate availability.

Best practices include:

1. Use tested backups. Follow a policy such as multiple copies on different media, with at least one copy isolated or offline. A backup is not reliable until restoration has been tested.

2. Use end-to-end integrity checks. Checksums, cryptographic hashes, and file-system metadata can detect silent corruption. Detection should be paired with a mechanism for repair.

3. Monitor health. Track SMART or vendor telemetry, error rates, temperature, wear indicators, latency, and rebuild status.

4. Scrub redundant storage. Periodically read data and verify parity or replicas before a failure makes recovery urgent.

5. Respect flush and ordering semantics. Applications requiring durability must use appropriate synchronization operations and confirm that the hardware honors them.

6. Protect data. Apply encryption at rest, secure key management, access control, audit logging, and secure device retirement.

7. Plan for failure domains. Separate replicas across controllers, racks, power systems, or geographic sites when the threat model requires it.

Storage security includes confidentiality, integrity, availability, and controlled destruction. Sanitization methods depend on the medium. Overwriting may be appropriate for some magnetic media, but flash translation layers can make simple overwriting incomplete. Cryptographic erasure—destroying encryption keys—may be more dependable when encryption was enabled correctly from the beginning. Organizations must also consider privacy regulations, retention requirements, data residency, and legal holds.

9. Applied Case: An Illustrative University AI Research Cluster

Background

A university AI laboratory stores training datasets, experiment logs, model checkpoints, and research results. Its first-generation system used a single large file server with HDDs. Researchers reported slow dataset loading, lost checkpoints after power interruptions, and occasional unexplained file corruption.

Problem analysis

Measurements showed that many jobs performed concurrent sequential reads, while checkpoint creation generated large bursts of writes. The server used write-back caches but did not consistently issue flush operations before reporting checkpoints as complete. The storage array had redundancy, but no regular scrubbing schedule or restore testing. A single network link and one controller also represented failure points.

Approach

The laboratory redesigned the system as follows:

• Frequently accessed training data was placed on a striped, redundant SSD pool, while archival data remained on HDD-based capacity storage.
• Checkpoint software used temporary files, integrity hashes, atomic rename, and explicit durability operations before declaring a checkpoint complete.
• RAID 6 was selected for the large-capacity HDD tier because capacity and tolerance of two drive failures were more important than maximum write performance.
• SSDs were monitored for endurance, temperature, and latency; HDD arrays were scrubbed periodically.
• Snapshots and geographically separate backups protected against accidental deletion and site failure.
• The network path was made redundant, and access was controlled through identity-based permissions and encryption.
• Restore drills were scheduled each term.

Outcome

Dataset throughput improved because the active working set used SSD parallelism and the network was no longer a single bottleneck. Checkpoint loss after power events was eliminated in testing because completion was reported only after integrity and durability requirements were met. The laboratory also discovered that one backup policy had been unusable until a restore drill exposed a permissions error.

Learning points

The case demonstrates that I/O performance is a system property. Faster media alone cannot solve queueing, network, caching, or software-ordering problems. Reliability requires redundancy, integrity verification, monitoring, and tested recovery. In AI environments, reproducibility and checkpoint integrity are as important as raw throughput.

10. Challenges and Contemporary Directions

Modern I/O systems face increasing scale and heterogeneity. NVMe devices support many queues and high parallelism; cloud storage exposes distributed semantics and variable latency; computational storage moves selected processing closer to data; and storage-class memory reduces the boundary between memory and storage.

These developments create challenges. Applications must cope with tail latency rather than averages, multi-tenant interference, evolving durability guarantees, energy consumption, and supply-chain risks in firmware and drivers. AI workloads add large sequential datasets, high checkpoint traffic, and accelerator-host data movement. Techniques such as asynchronous I/O, zero-copy transfer, batching, data locality, compression, and prefetching can help, but each may trade CPU time, memory, latency, and reliability.

A sound engineering decision begins with a workload description: read/write ratio, request size, sequentiality, concurrency, latency objective, durability requirement, failure model, and budget. Benchmark results are meaningful only when they reproduce the intended workload and include warm-up behavior, queue depth, tail latency, and recovery scenarios.

Chapter Summary

I/O systems bridge fast processors and diverse, independently operating devices. Controllers provide hardware interfaces, drivers translate operating-system requests, interrupts report events, and DMA transfers large data blocks with limited CPU intervention. Buffering absorbs rate differences, caching exploits locality, and spooling decouples applications from sequential devices.

Magnetic-disk performance depends on seek, rotational latency, transfer, and queueing. Disk scheduling algorithms such as FCFS, SSTF, SCAN, and C-SCAN balance movement, fairness, and response time. SSDs eliminate mechanical movement but introduce flash translation, garbage collection, wear leveling, write amplification, and error correction. RAID improves selected combinations of performance, capacity, and fault tolerance, but it does not replace backups.

Persistent memory offers nonvolatile, low-latency access but requires careful crash consistency and persistence ordering. Reliable storage combines redundancy with integrity checking, monitoring, scrubbing, tested backups, correct durability operations, security controls, and recovery planning. For AI and ML systems, effective I/O design must consider data locality, checkpoint durability, throughput, tail latency, and the complete failure domain.

Key Terms

Device controller: Hardware that operates a device and exposes control, status, and data interfaces.

Device driver: Privileged software that translates generic operating-system requests into device-specific commands.

Interrupt: A signal that requests processor attention for an event such as I/O completion.

DMA: Direct memory access; controller-managed transfer between a device and main memory.

IOMMU: Hardware that translates and restricts addresses used by devices during DMA.

Buffer: Temporary storage used to absorb rate or format differences during data transfer.

Cache: A store of copies of data kept to exploit likely reuse.

Spooling: Queuing device output, often on disk, for later sequential service.

Seek time: Time required for an HDD head to move to the required cylinder.

Rotational latency: Time waiting for the required disk sector to rotate into position.

SSTF: Disk scheduling method that selects the request with the shortest current seek distance.

SCAN: Elevator-style disk scheduling that services requests while moving in one direction, then reverses.

SSD: Solid-state drive using nonvolatile flash or related memory rather than rotating platters.

FTL: Flash translation layer mapping logical block addresses to physical flash locations.

Wear leveling: Distributing flash writes to reduce uneven cell wear.

RAID: Organization of multiple drives to provide selected performance, capacity, or redundancy benefits.

Parity: Redundant information used to reconstruct missing data in some RAID levels.

Persistent memory: Nonvolatile memory that can provide comparatively low-latency access.

Durability: Preservation of acknowledged data despite specified failures.

Scrubbing: Periodic verification of stored data and redundant information to detect latent errors.

Review Questions

Short Answer Questions

1. What are the principal responsibilities of a device driver?

2. How does DMA reduce CPU involvement in data transfer?

3. Distinguish buffering from caching.

4. Why is average rotational latency approximately half a disk revolution?

5. Why is RAID not a replacement for backup?

Long Answer Questions

1. Compare programmed I/O, interrupt-driven I/O, and DMA, including their performance and protection implications.

2. Explain the operation of an SSD’s flash translation layer, including garbage collection, wear leveling, and TRIM.

3. Compare FCFS, SSTF, SCAN, and C-SCAN disk scheduling with respect to movement, fairness, and starvation.

4. Compare RAID 0, RAID 1, RAID 5, RAID 6, and RAID 10 in terms of capacity, performance, and fault tolerance.

5. Explain how an operating system and applications can provide durable, crash-consistent storage.

Analytical / Application Questions

1. A storage server experiences high write bursts, long tail latency, and premature SSD wear. Identify likely causes and propose measurements and remedies.

2. Design an I/O and storage architecture for an AI laboratory that requires high-throughput training data, durable checkpoints, and recovery from a site-level disaster. Justify the media, redundancy, caching, and backup choices.

3. A RAID 5 array reports one failed drive while a scrub finds an unrecoverable sector on another drive. Analyze the risk and recommend an immediate response and a longer-term redesign.

Multiple Choice Questions with Answers

Question 1

Which component normally translates generic kernel I/O requests into device-specific commands?

A. Device driver

B. Cache controller

C. Compiler

D. File descriptor

Correct Answer: A. Device driver

Question 2

What is the main advantage of DMA for a large data transfer?

A. It eliminates the need for device hardware

B. It transfers data with limited per-byte CPU involvement

C. It guarantees that data is permanently stored

D. It prevents all memory-access errors

Correct Answer: B. It transfers data with limited per-byte CPU involvement

Question 3

A disk rotates at 7,200 RPM. Under the uniform-position assumption, what is its average rotational latency?

A. 0.083 ms

B. 1.20 ms

C. 4.17 ms

D. 8.33 ms

Correct Answer: C. 4.17 ms

Question 4

Which disk scheduling algorithm can reduce seek movement but may starve requests far from the current head position?

A. FCFS

B. SSTF

C. C-SCAN

D. LOOK

Correct Answer: B. SSTF

Question 5

Which SSD operation informs the device that logical blocks are no longer needed?

A. TRIM or discard

B. Seek

C. Spooling

D. Mirroring

Correct Answer: A. TRIM or discard

Question 6

Which RAID level provides distributed single parity and can tolerate one drive failure in the usual design?

A. RAID 0

B. RAID 1

C. RAID 5

D. RAID 10

Correct Answer: C. RAID 5

Question 7

What is the principal purpose of wear leveling in flash storage?

A. To increase disk rotational speed

B. To distribute writes across flash cells

C. To remove the need for error correction

D. To convert flash into volatile RAM

Correct Answer: B. To distribute writes across flash cells

Question 8

Which statement best distinguishes a cache from a buffer?

A. A cache stores likely-to-be-reused copies; a buffer absorbs transfer differences

B. A cache is always persistent; a buffer is always encrypted

C. A cache is used only for output; a buffer only for input

D. There is no functional distinction between them

Correct Answer: A. A cache stores likely-to-be-reused copies; a buffer absorbs transfer differences

Question 9

Why is RAID insufficient as the only data-protection mechanism?

A. RAID cannot improve availability

B. RAID prevents all hardware redundancy

C. Deletion, corruption, malware, or site failure can affect the entire array

D. RAID works only with magnetic tape

Correct Answer: C. Deletion, corruption, malware, or site failure can affect the entire array

Question 10

Which practice most directly verifies that a backup can actually be used after a disaster?

A. Increasing cache size

B. Running restore drills

C. Disabling checksums

D. Using SSTF scheduling

Correct Answer: B. Running restore drills