Data Representation, Storage and File Systems

Introduction

Digital forensics depends on the ability to interpret data at the level at which computers store and process it. A document that appears as readable text to a user is ultimately represented as bits, arranged into bytes, encoded according to a character standard, stored in sectors or flash pages, and organized by a file system. Investigators who understand only application-level views may miss deleted content, hidden partitions, timestamp anomalies, file-system metadata, or evidence distributed across storage structures.

This chapter develops a foundation for examining binary and hexadecimal data, character encoding, timestamps, storage media, partitions, boot structures, allocation units, slack space, deleted data, journaling, metadata, RAID, and solid-state storage. It also emphasizes forensic interpretation: a storage artifact is not automatically evidence of user activity, and every conclusion should be based on validated tools, documented assumptions, and preservation of original data.

Learning Objectives

By the end of this chapter, the reader should be able to:

1. Convert and interpret binary, hexadecimal, decimal, and byte-oriented data.

2. Explain character encoding, endianness, and common timestamp representations.

3. Describe magnetic, optical, flash, and virtualized storage behavior.

4. Analyze partition tables, boot structures, sectors, clusters, and allocation maps.

5. Interpret metadata, slack space, deleted files, journals, and file signatures.

6. Compare FAT-family, NTFS, ext4, APFS, and related file systems.

7. Explain the forensic consequences of RAID, SSD garbage collection, TRIM, encryption, and virtualization.

8. Apply a defensible workflow to a storage-forensics investigation.

1. Data at the Machine Level

1.1 Bits, bytes, and number systems

A bit has a value of 0 or 1. Eight bits form one byte, which can represent 2^8 = 256 distinct patterns, from 00000000 to 11111111. For an unsigned n-bit field, the representable range is 0 through 2^n − 1. This range assumes that all bit patterns represent non-negative integers; signed representations use conventions such as two's complement and therefore have different interpretations.

Binary is useful for understanding individual bits, flags, masks, and permissions. Hexadecimal is more compact: one hexadecimal digit represents four bits, so two hexadecimal digits represent one byte. For example:

• Binary 10101100 = hexadecimal AC = decimal 172.
• Hexadecimal 7F = binary 01111111 = decimal 127.
• Hexadecimal 0x100 represents decimal 256.

A forensic examiner should distinguish a number's value from its display format. The byte sequence 31 32 33 may represent the ASCII characters “123,” three decimal values, part of an instruction, or a field in a proprietary structure. Interpretation requires context, offsets, specifications, and corroborating evidence.

1.2 Endianness

Multibyte integers may be stored in little-endian or big-endian order. In little-endian representation, the least significant byte appears first. Thus, the four-byte sequence 78 56 34 12 represents 0x12345678 when interpreted as a little-endian integer. In big-endian order, the same value is stored as 12 34 56 78.

Endianness applies to numeric fields, not generally to character strings. A forensic tool that displays a field incorrectly may produce an apparently impossible file size, timestamp, or block number. Examiners should consult the relevant file-system or file-format specification and verify interpretations against known values.

1.3 Character encoding

ASCII assigns values to a limited set of English letters, digits, punctuation, and control characters. Unicode provides a much larger character repertoire. UTF-8 represents Unicode characters using one to four bytes and preserves ASCII compatibility for values 0 through 127. UTF-16 commonly uses two-byte code units and may use surrogate pairs for supplementary characters. UTF-32 uses four-byte code units but consumes more storage.

Encoding affects searches and interpretation. The word “password” may appear in ASCII, UTF-8, UTF-16 little-endian, compressed form, encrypted form, or not at all if an application stores it as a structured value. A search that considers only ASCII is therefore incomplete. Investigators should document the encoding assumptions used in keyword searches and reporting.

2. Time, Dates, and Interpretation

Digital systems use several timestamp schemes. Unix time commonly records the number of seconds elapsed since 1970-01-01 00:00:00 UTC, excluding leap seconds in ordinary implementations. Windows FILETIME records 100-nanosecond intervals since 1601-01-01 UTC. FAT timestamps are generally stored in local time with coarse resolution, whereas NTFS timestamps are typically UTC-based FILETIME values.

A timestamp is not automatically a proof of an event. It may describe creation, modification, access, metadata change, synchronization, download, indexing, or a system-generated operation. Time-zone settings, daylight-saving changes, clock drift, virtualization, copying, and anti-forensic manipulation can affect interpretation.

A defensible timeline should record the original value, its format, the assumed time zone, conversion method, source artifact, and confidence level. Correlation with logs, network records, application databases, and independent devices is preferable to relying on one timestamp.

3. Storage Media and Acquisition Implications

3.1 Magnetic disks

Hard disk drives store magnetically encoded bits on rotating platters. Data is accessed through sectors, traditionally 512 bytes, although 4,096-byte physical-sector formats are common. Logical block addressing presents sectors as a linear sequence of addresses, hiding physical geometry from most software.

Magnetic media may retain remnants of deleted data until sectors are overwritten. However, recovery is not guaranteed: files may be fragmented, partially overwritten, encrypted, or affected by bad-sector remapping. Modern disk controllers may also relocate defective sectors, making physical assumptions unreliable.

3.2 Solid-state storage

SSDs use NAND flash organized into pages and erase blocks. Reads and writes occur at different granularities: a page may be programmed, but an entire erase block is generally erased. Flash translation layers map logical block addresses to physical locations, perform wear leveling, and manage bad cells.

Consequently, a logical overwrite may not immediately overwrite the previous physical location. Conversely, garbage collection and TRIM may cause old pages to become inaccessible or be erased without an investigator issuing a conventional delete operation. SSD examination must therefore prioritize timely acquisition, controller-aware methods, and interpretation of limitations. Traditional expectations about recovering deleted files from magnetic disks do not transfer reliably to SSDs.

3.3 Optical, removable, virtual, and network storage

Optical media may be write-once or rewritable and can contain session and volume structures. USB devices may use flash storage behind controllers that report misleading capacities or sector sizes. Virtual disks such as VHDX and QCOW2 contain metadata and may expose snapshots, differencing chains, and unallocated regions. Network-attached storage can distribute data across servers and require collection of both content and service metadata.

The storage device itself is evidence. Investigators should document make, model, serial number, capacity, interface, connection state, encryption status, and acquisition method.

4. Partitions, Volumes, and Boot Structures

A disk may contain a partition table describing regions allocated to volumes. The traditional Master Boot Record (MBR) occupies the first sector and includes boot code, partition entries, and a signature. MBR partition entries use a 32-bit sector addressing model and are limited in ways that become significant for large disks.

GUID Partition Table (GPT) uses a protective MBR for compatibility, a primary GPT header, partition entries, and a backup header near the end of the disk. GPT includes globally unique partition identifiers and integrity checks. The backup structures are valuable when the primary table is damaged, but they must not be treated as automatically authoritative: stale or inconsistent copies may remain after repartitioning.

A volume boot record or volume boot sector contains file-system-specific bootstrap code and parameters. The BIOS Parameter Block or related structures may identify bytes per sector, sectors per cluster, reserved areas, and other geometry information. UEFI systems commonly use an EFI System Partition containing boot files. Boot artifacts can reveal operating-system installations, removable media, recovery environments, or attempts to conceal data in unusual partitions.

Partition boundaries should be validated mathematically and logically. An examiner should compare partition start and end addresses with the disk image size, identify gaps, inspect unallocated regions, and check for overlapping or inconsistent entries.

5. Allocation, Slack, and Deleted Data

File systems map logical files to physical or logical storage units. A sector is a device-level addressable unit; a cluster, block, or allocation unit is a file-system unit composed of one or more sectors. If a file occupies only part of its final cluster, the remainder is file slack. Previously stored bytes may remain in slack, although modern initialization, encryption, compression, and storage behavior can reduce or alter such remnants.

Unallocated space consists of units not currently assigned to active files according to file-system metadata. It may contain deleted file content, remnants of previous partitions, temporary data, or unrelated noise. Deletion usually removes or changes references rather than immediately erasing every content byte. Recovery depends on fragmentation, reuse, overwriting, TRIM, encryption, and file-system behavior.

A file signature, or magic number, is a characteristic byte pattern associated with a format. Signature analysis can identify files whose names or extensions are misleading, but it is not infallible. A valid header does not prove that the entire file is intact, and an attacker can deliberately insert or modify signatures. File carving reconstructs content from unallocated space using headers, footers, internal structures, or statistical techniques. Carving often loses original names, paths, permissions, and timestamps and can produce false positives.

6. File-System Structures and Comparison

6.1 FAT family

FAT12, FAT16, and FAT32 use a File Allocation Table to link clusters. Directory entries contain names, attributes, starting clusters, sizes, and timestamps. Long filenames are represented through associated directory entries. FAT is comparatively simple and widely supported, making it common on removable media, but it provides limited metadata, weak journaling support, and historical constraints on volume and file size.

Examination should include boot parameters, FAT copies, root-directory structures, directory entries, allocation chains, volume labels, and unused directory entries. Deleted entries may preserve portions of names and starting-cluster information, but recovery can fail when clusters have been reused or fragmented.

6.2 NTFS

NTFS uses the Master File Table (MFT), in which files and directories are represented by records containing attributes. Important attributes include file names, data, standard information, security descriptors, and indexes. Small files may be stored resident within MFT records; larger files use data runs that describe extents elsewhere on the volume.

NTFS also provides the USN Change Journal, which records certain changes, and a transaction log supporting metadata consistency. The Recycle Bin, link files, Windows event logs, registry hives, and shadow copies may complement NTFS evidence. MFT timestamps and Windows application timestamps must be interpreted separately; similar-looking times may have different semantics.

6.3 ext-family systems

Linux ext2, ext3, and ext4 use superblocks, block groups, inode tables, data blocks, and directory structures. Inodes store metadata and pointers or extents to file data. ext3 and ext4 support journaling, while ext4 adds extents, delayed allocation, checksums, and other scalability and reliability features. The journal may preserve metadata or recently written information, but it is not a complete historical record of every file operation.

Important artifacts include superblock backups, inode states, directory entries, extended attributes, access-control information, journal contents, and filesystem-specific timestamps. Linux systems may also expose evidence through shell histories, package databases, systemd journals, and application logs.

6.4 APFS and modern file systems

APFS is designed for Apple platforms and supports containers, volumes, copy-on-write metadata, snapshots, space sharing, encryption integration, and clones. Copy-on-write designs can preserve older blocks through snapshots or alternate references, but they also complicate reconstruction because logical identity and physical placement are separated.

Modern file systems increasingly use checksums, snapshots, compression, encryption, deduplication, and copy-on-write structures. These features improve reliability or efficiency but require specialized parsers and careful validation.

7. Journaling, Metadata, and Evidential Value

Journaling records intended changes so that a file system can recover from interruption. Metadata journaling may record directory or allocation changes without recording file content. Full-data journaling, where implemented, offers stronger content preservation but incurs greater overhead. A journal is therefore best understood as a recovery mechanism whose contents may provide historical clues, not as a universal audit log.

Metadata provides context: ownership, permissions, allocation, identifiers, parent-child relationships, extended attributes, alternate data streams, security descriptors, and application-specific records. Metadata can be more probative than visible content, but it is also vulnerable to copying, synchronization, tool conversion, clock errors, and deliberate manipulation.

Forensic reporting should distinguish:

• observed fact: a byte sequence or record was found;
• technical interpretation: the record corresponds to a particular structure;
• investigative inference: the structure is consistent with a user or process action;
• attribution: a specific person performed the action.

The last claim requires evidence beyond file-system metadata.

8. RAID, Encryption, and Virtualization

RAID distributes or duplicates data across multiple drives. RAID 0 stripes data without redundancy; failure of one member can affect the entire logical volume. RAID 1 mirrors data; RAID 5 uses distributed parity and can tolerate one member failure under normal assumptions; RAID 6 uses dual parity and can tolerate two member failures. RAID 10 combines mirroring and striping. These descriptions concern array design, not forensic recoverability.

A RAID examination requires member order, stripe size, parity layout, controller metadata, sector offsets, and sometimes the original controller or a validated reconstruction algorithm. A wrong parameter can produce an apparently plausible but incorrect file system. Investigators should preserve every member and configuration artifact before attempting reconstruction.

Full-disk, volume, file, and application encryption protect different scopes of data. Encryption may leave metadata, headers, key-management artifacts, and unencrypted remnants, but brute-force recovery is not a substitute for lawful key acquisition. Passwords, recovery keys, trusted-platform modules, cloud credentials, and live-system memory may be relevant, subject to legal authority and privacy constraints.

Virtualization introduces layers: physical device, virtual disk, partition, guest file system, snapshots, and host metadata. The same logical file may exist in current, deleted, and snapshot states. Examiners must identify the layer from which each artifact was obtained.

9. A Defensible Storage-Forensics Workflow

A sound workflow begins with authorization, scope definition, and risk assessment. The examiner should document the device state, isolate it from unintended modification, and use a validated write blocker where appropriate. Acquisition should create a bit-for-bit image when feasible, preserve source and destination details, and calculate cryptographic hashes for integrity verification. Hashes demonstrate that two datasets are identical under the chosen algorithm; they do not prove that the source was unaltered before acquisition.

Analysis should proceed from structure to content: identify the image format, inspect partition tables, locate volumes, identify file systems, validate superblocks or boot records, and then examine active, deleted, unallocated, slack, journal, and application artifacts. Automated tools should be supplemented with hex-level inspection and independent validation for important findings.

Reporting should state methods, tool versions, hash values, time-zone assumptions, recovered and unrecovered areas, limitations, and the distinction between observation and interpretation. Reproducibility is strengthened when offsets, record identifiers, extraction procedures, and relevant screenshots or exported records are preserved.

10. Illustrative Case: Suspected Theft of a Design Archive

Background

A company reported that proprietary design files were copied from an engineer's workstation before resignation. The workstation contained a 1 TB SSD using GPT and NTFS. The user claimed that the files had been deleted weeks earlier and that no external drive had been connected.

Problem

The investigation needed to determine whether the files had existed, when they were accessed, whether deletion occurred, and whether the evidence supported copying to removable media. It was not sufficient merely to find a file with the correct name.

Analysis and approach

Investigators documented the powered state, disconnected network access under approved procedures, and acquired the SSD using a validated process. They verified the image hash and identified the EFI System Partition, recovery partition, and main NTFS volume. Examination of the MFT showed records for the design files, while the current directory no longer listed them. Portions of file content were recovered from unallocated clusters, but several files were incomplete.

The NTFS USN Change Journal showed file creation and later deletion events. A Windows shortcut artifact referenced a path on the workstation, and application recent-file records indicated that the files had been opened. However, those artifacts did not independently prove copying. A separate examination of endpoint logs and an attached USB device image found matching filenames and hashes for a subset of the designs. The USB device had a different file-system timestamp resolution, so investigators treated its timestamps as approximate and relied more heavily on matching content, device connection logs, and corroborating access records.

Because the source was an SSD, investigators reported that TRIM and garbage collection could have removed additional remnants. They did not claim that the incomplete recovered files represented the complete deleted archive.

Outcome and learning points

The evidence supported the conclusion that the files had existed on the workstation, had been opened, and had later been deleted. Matching content on the USB device and independent connection evidence supported copying of some files. The evidence did not establish the precise second of copying or exclude every possible alternative actor.

The case demonstrates the value of combining file content, MFT records, journals, shortcut artifacts, endpoint logs, and external-device evidence. It also illustrates why a timestamp or deleted-file recovery result should not be interpreted in isolation.

11. Challenges, Governance, and Contemporary Developments

Storage investigations face increasing use of encryption, cloud synchronization, SSD garbage collection, proprietary controllers, large-scale RAID, compression, deduplication, and ephemeral virtual machines. Tool support may lag behind new file-system versions. Parsing errors, incorrect time-zone conversion, and unvalidated RAID reconstruction can produce confident but false conclusions.

Investigators must obtain lawful authority, minimize collection of unrelated personal data, protect confidential information, and maintain chain of custody. Sensitive material should be access-controlled, encrypted at rest, and retained only as required. AI-assisted triage can prioritize documents, cluster similar artifacts, or identify anomalous structures, but it should not replace byte-level validation. AI outputs require auditability, known datasets, bias assessment, and human review; a model's classification is not itself proof of authorship or intent.

Best practice includes maintaining an evidence register, using write protection, recording tool and parser versions, validating important results with independent methods, preserving original images, documenting uncertainty, and separating investigative hypotheses from demonstrated facts.

Chapter Summary

Digital forensic interpretation begins with bits, bytes, number systems, encoding, and endianness. Storage media impose different behaviors: magnetic disks may retain overwritten remnants, whereas SSD controllers, TRIM, and garbage collection can make logical deletion irreversible. Partitions, boot structures, and volume metadata define how storage is organized, while clusters, allocation maps, slack space, journals, and unallocated regions reveal information beyond visible files.

FAT, NTFS, ext4, and APFS use different structures and provide different forensic opportunities. Metadata and timestamps require contextual interpretation and cannot alone establish user intent. RAID reconstruction, encryption, virtualization, and copy-on-write snapshots add layers that must be identified and validated. A defensible investigation preserves original evidence, verifies integrity, uses validated tools, documents assumptions, and clearly separates observations from inferences.

Key Terms

Bit: A binary value of zero or one.

Byte: A group of eight bits.

Endianness: The order in which bytes of a multibyte value are stored.

Encoding: A rule for representing characters or other information as bytes.

Timestamp: A recorded value associated with an event or state at a particular time.

Sector: A device-level unit addressed on storage media.

Cluster: A file-system allocation unit composed of one or more sectors.

Slack space: Unused space within an allocated final cluster.

Unallocated space: Storage not currently assigned to an active file according to metadata.

File carving: Recovering file content from raw data using structural patterns.

Magic number: A characteristic byte sequence associated with a file format.

Partition table: Metadata describing regions of a disk.

MBR: A traditional first-sector boot and partition structure.

GPT: A modern partitioning scheme using GUIDs and redundant headers.

MFT: NTFS metadata structure containing records for files and directories.

Inode: A Unix-like file-system structure containing file metadata and data references.

Journaling: Recording changes to support file-system recovery.

RAID: A method of combining storage devices for performance, redundancy, or both.

TRIM: A command allowing a storage system to identify logical blocks no longer needed.

Copy-on-write: A method that preserves existing data until modified data is written elsewhere.

Review Questions

Short Answer Questions

1. Why is hexadecimal commonly used in forensic examination?

2. Distinguish file slack from unallocated space.

3. What is the principal forensic effect of endianness?

4. Why can a deleted file sometimes be recovered?

5. What information does a partition table provide?

Long Answer Questions

1. Compare magnetic-disk and SSD deletion behavior and explain the consequences for acquisition.

2. Describe the principal forensic structures of NTFS and their evidential value.

3. Explain how timestamps should be normalized and interpreted in a forensic timeline.

4. Compare MBR and GPT and discuss why both primary and backup structures should be examined.

5. Explain the technical and evidential challenges of reconstructing a RAID array.

Analytical / Application Questions

1. A recovered four-byte field appears as 78 56 34 12. Explain the value under little-endian interpretation and describe how you would test whether that interpretation is correct.

2. An investigator finds a deleted filename in an NTFS record but only partial content in unallocated space. Analyze at least four reasons why recovery may be incomplete and identify corroborating evidence to seek.

3. A case involves a virtual machine stored on a RAID 5 array with an encrypted guest volume. Design a preservation and analysis sequence that minimizes alteration and distinguishes host, virtual-disk, and guest-file-system evidence.

Multiple Choice Questions with Answers

Question 1

How many distinct patterns can be represented by one byte?

A. eight

B. 128

C. 256

D. 512

Correct Answer: C. 256

Question 2

The byte sequence 78 56 34 12 represents which value under little-endian interpretation?

A. 0x78563412

B. 0x12345678

C. 0x123456

D. 0x56341278

Correct Answer: B. 0x12345678

Question 3

Which structure is central to NTFS file and directory records?

A. File Allocation Table

B. Master File Table

C. Superblock journal only

D. EFI System Partition

Correct Answer: B. Master File Table

Question 4

What is file slack?

A. Space between partitions

B. Unused space in an allocated final cluster

C. A backup partition table

D. A RAID parity region

Correct Answer: B. Unused space in an allocated final cluster

Question 5

Which partitioning scheme uses GUID-based partition identifiers and redundant headers?

A. FAT16

B. MBR

C. GPT

D. ext2

Correct Answer: C. GPT

Question 6

What is a major limitation of file carving?

A. It can recover only encrypted data

B. It always preserves original filenames

C. It may lose names, paths, and metadata

D. It works only on optical media

Correct Answer: C. It may lose names, paths, and metadata

Question 7

What is a principal function of journaling?

A. Guaranteeing a complete user activity history

B. Supporting file-system recovery after interruption

C. Encrypting every file automatically

D. Increasing monitor resolution

Correct Answer: B. Supporting file-system recovery after interruption

Question 8

Which RAID level normally uses distributed parity and tolerates one member failure?

A. RAID 0

B. RAID 1

C. RAID 5

D. RAID 10 only

Correct Answer: C. RAID 5

Question 9

Why can TRIM complicate deleted-file recovery from SSDs?

A. It converts all files to ASCII

B. It may allow obsolete logical blocks to be discarded

C. It duplicates every sector

D. It disables partition tables

Correct Answer: B. It may allow obsolete logical blocks to be discarded

Question 10

Which statement is the most defensible forensic conclusion?

A. A timestamp alone proves a person performed an action

B. A file extension proves the file's true format

C. A matching recovered file and independent device records can corroborate copying

D. A hash proves who created the evidence

Correct Answer: C. A matching recovered file and independent device records can corroborate copying