Persistent Data and Storage

Introduction

Containers are designed to be replaceable. A container’s writable layer normally disappears when the container is removed, which is desirable for immutable application deployment but dangerous for databases, uploaded files, audit records, and other stateful data. Persistent storage separates the life of data from the life of a container.

Docker provides three principal mount mechanisms: volumes, bind mounts, and tmpfs mounts. Each has different ownership, portability, security, and operational characteristics. Effective container administration also requires understanding storage drivers, permissions, backup and restoration, capacity management, and the complete data lifecycle.

Learning Objectives

By the end of this chapter, a learner should be able to:

• distinguish container writable layers from persistent storage;
• select and configure volumes, bind mounts, and tmpfs mounts;
• explain the role and limitations of storage drivers;
• manage Linux permissions and ownership for mounted data;
• design basic backup, restore, and validation procedures;
• apply retention, monitoring, security, and deletion practices to stateful workloads.

1. The Storage Model of a Container

A container image is composed of read-only layers. When a container starts, the runtime adds a thin writable layer above those image layers. Changes made inside the container are written to this layer through a union or overlay filesystem. If the container is deleted, these changes are normally deleted as well.

A mount makes a directory in the container a view of storage located outside the container’s writable layer. For example, mounting persistent storage at /var/lib/postgresql/data allows database files to survive replacement of the database container.

Figure: A mount redirects a container directory to storage whose lifecycle is independent of the container.

The writable layer is useful for temporary application changes, but it is generally unsuitable for high-volume or important data. Copy-on-write filesystems can also impose performance overhead when a process modifies files originating in a lower image layer. Persistent data should therefore be placed deliberately rather than relying on the writable layer.

2. Mount Types

2.1 Volumes

A Docker-managed volume is stored in Docker’s storage area and is identified by a name. Docker creates and manages its lifecycle, while the container merely consumes it.

docker volume create app-data
docker run -d --name web \
  --mount type=volume,src=app-data,dst=/var/lib/app \
  my-web-image

The --mount form is explicit and generally preferred in teaching and production documentation. The shorter form is also valid:

docker run -d -v app-data:/var/lib/app my-web-image

Advantages of volumes include Docker-oriented administration, easy attachment to replacement containers, and support for volume drivers that can connect to external storage systems. Volumes are less convenient when administrators need direct access using ordinary host paths. A volume must not be assumed to be a backup: it is live data, not a second copy.

A volume can be declared read-only from the container’s perspective:

docker run --rm \
  --mount type=volume,src=app-data,dst=/data,readonly \
  backup-tool

2.2 Bind mounts

A bind mount maps a specific host path into a container:

docker run --rm \
  --mount type=bind,src=/srv/config,dst=/etc/myapp,readonly \
  myapp:1.0

Bind mounts are useful for development source code, host-managed configuration, certificates, and carefully controlled integration with host files. They provide direct host access, but that is also their principal risk. A container process may alter or delete host files if permissions allow it. Relative paths and ambiguous host locations should be avoided in production. Prefer absolute paths and read-only mounts whenever writing is unnecessary.

2.3 tmpfs mounts

A tmpfs mount stores files in memory (and potentially swap, depending on host configuration) and does not persist when the container stops. It is appropriate for short-lived secrets, caches, and temporary files:

docker run --rm \
  --mount type=tmpfs,dst=/run/secrets,tmpfs-size=64m,tmpfs-mode=0700 \
  myapp:1.0

Tmpfs is not a substitute for encrypted persistent storage. It consumes host memory, may affect resource pressure, and disappears on restart. The exact handling of swap and security depends on the operating system and runtime configuration.

2.4 Comparison

3. Storage Drivers and Performance

A storage driver implements the image layers and container writable layer. Common Linux installations use overlay-based drivers such as overlay2, although availability depends on the operating system, kernel, Docker version, and filesystem. The storage driver is not the same thing as a volume driver.

• The storage driver manages image layers and the container writable layer.
• A volume driver provisions or presents persistent storage.
• A bind mount uses an existing host path and does not require a volume driver.

The writable layer uses copy-on-write behavior: when a container modifies a file that exists in a lower image layer, the runtime may copy that file into the writable layer before modification. This can increase write amplification and latency. Databases should normally write to a dedicated volume or external storage system rather than to the writable layer.

Storage performance depends on workload characteristics, including random versus sequential I/O, synchronization frequency, metadata operations, latency, capacity, and contention. A fast storage medium cannot compensate for an unsuitable database configuration or an unsafe backup design. Benchmark the actual workload, because results vary with filesystem, kernel, hardware, virtualization, and driver.

Inspect the environment before designing storage:

docker info
docker system df
docker volume ls
docker volume inspect app-data

Do not manually edit Docker’s internal storage directories. Use Docker commands or documented storage interfaces; direct manipulation can corrupt metadata and invalidate support assumptions.

4. Permissions, Ownership, and Security

A container process does not automatically gain permission merely because a path is mounted. On Linux, access is evaluated using the process user and group IDs, discretionary permissions, ACLs, and security mechanisms such as SELinux or AppArmor.

For example, an image may run as user ID 1001, while a host directory is owned by root and mode 0750. The process may then be unable to create files. A robust solution is to align ownership and group access deliberately, rather than running the application as root by default.

sudo mkdir -p /srv/myapp/data
sudo chown -R 1001:1001 /srv/myapp/data
sudo chmod 0750 /srv/myapp/data

The correct UID and GID must be obtained from the image documentation or inspection; 1001 is only an example. Recursive ownership changes can be dangerous on shared paths and should be tested before production use.

For shared data, group permissions or ACLs may be more appropriate than making files world-writable. Avoid chmod 777, which grants unnecessary access and can enable tampering. A read-only mount is a strong defense-in-depth measure:

--mount type=bind,src=/srv/certs,dst=/run/certs,readonly

On systems using SELinux, a bind mount may require an appropriate labeling option, such as :Z for a private relabel or :z for a shared relabel, depending on policy and distribution. These options should not be applied casually: relabeling can affect host access and other containers. Rootless Docker changes the privilege model and may impose additional restrictions on paths, networking, and storage.

Sensitive data should be encrypted in transit and at rest where required by organizational policy or law. Mounting a secret into a container does not automatically protect it from a process that has access to that mount. Limit access, avoid embedding secrets in images, and consider a dedicated secrets-management system.

5. Stateful Service Design

A stateful service has data whose loss or inconsistency affects business behavior. Good design separates:

1. Application code, delivered as an image;
2. Configuration, delivered through controlled configuration mechanisms;
3. Persistent data, stored on a volume or external service;
4. Backups, stored independently from the primary data location.

A database volume should normally be mounted at the database’s documented data directory. Mounting over a directory containing initialization files can hide those files, and changing mount points after initialization can make a service appear empty. Follow the database vendor’s procedure for initialization, shutdown, and consistency.

A single volume attached to one container is not automatically highly available. High availability requires redundancy, failure detection, replication or resilient storage, and a tested recovery process. Likewise, a volume snapshot is not necessarily application-consistent: a database may have unflushed or interdependent writes at snapshot time.

6. Backup and Restore

A backup is a recoverable copy created through a defined process. Availability of a volume is not backup, and replication is not always backup: accidental deletion or corruption can replicate to every copy.

A useful backup policy specifies:

• what data is included and excluded;
• frequency and retention period;
• consistency method;
• encryption and access control;
• storage location and geographic separation;
• verification and restoration tests;
• responsible personnel and recovery objectives.

Two common approaches are application-aware backup and filesystem-level backup. An application-aware backup uses the database or service’s export mechanism and can produce a logically consistent artifact. A filesystem-level copy may be efficient, but it requires a quiesced service, a supported snapshot method, or a storage system that guarantees consistency.

A simple quiesced volume export can be demonstrated as follows:

docker stop db

docker run --rm \
  --mount type=volume,src=db-data,dst=/source,readonly \
  --mount type=bind,src=/srv/backups,dst=/backup \
  alpine tar czf /backup/db-data-2025-01-15.tgz -C /source .
docker start db

This is an instructional pattern, not a universal database backup procedure. Some databases require a native dump or a coordinated snapshot. The backup file should be protected from unauthorized access and checked for successful completion.

A restore should be performed into a clean target or controlled replacement environment. For example:

docker volume create db-restore-test
docker run --rm \
  --mount type=volume,src=db-restore-test,dst=/target \
  --mount type=bind,src=/srv/backups,dst=/backup,readonly \
  alpine sh -c 'tar xzf /backup/db-data-2025-01-15.tgz -C /target'

After restoration, start the service with the restored volume, inspect logs, run integrity checks, and verify representative records. A backup that has never been restored is an assumption rather than demonstrated recoverability.

Two operational measures are useful:

• Recovery Point Objective (RPO): the maximum acceptable amount of data loss measured in time.
• Recovery Time Objective (RTO): the maximum acceptable time to restore service.

For example, an RPO of 15 minutes requires a process capable of preserving changes at least that frequently; an RTO of 60 minutes requires the entire restoration and validation workflow to fit within one hour. These are service requirements, not Docker-specific commands.

7. Data Lifecycle and Capacity Management

Data lifecycle management covers creation, use, retention, archival, and destruction. A practical lifecycle policy identifies the data owner, classification, retention duration, legal hold requirements, deletion authority, and evidence of deletion.

Unused volumes can accumulate after deployments and testing. Inspect before deleting:

docker volume ls
docker volume inspect old-volume
docker volume rm old-volume

docker volume prune can remove unused local volumes and should be used only after careful review. Never treat an automated prune command as a backup or as a substitute for inventory.

Monitor capacity, inode consumption, I/O latency, errors, and backup age. A full filesystem can cause database failure, logging failure, and cascading service outages. Alert before capacity becomes critical. Quotas and application-level limits can prevent one workload from exhausting shared storage.

Deletion is irreversible unless a valid backup exists. Secure deletion requirements vary by medium, encryption architecture, and regulation. For highly sensitive data, encryption with controlled key destruction may be more dependable than attempting to overwrite modern flash storage; organizational security policy should govern the decision.

8. Illustrative Case: Containerized Order Service

Background

A small retailer runs an order API and PostgreSQL in Docker Compose. The API image is rebuilt weekly. Initially, PostgreSQL writes to the container writable layer, and uploaded invoice files are stored inside the API container.

Problem

A routine deployment removes and recreates the containers. The application starts successfully, but historical orders and invoices disappear. A later attempt to copy the live database directory produces an inconsistent backup.

Analysis

The design confused container replacement with data retention. It also lacked a defined owner, backup schedule, restore test, and permission model. Copying database files while the database was active did not establish application consistency.

Approach

The team changes the design as follows:

• PostgreSQL data is placed on a named volume, orders-db.
• Invoice files are placed on a separate named volume, invoice-data.
• Database exports run nightly through PostgreSQL’s native backup tooling.
• A weekly restore is tested in an isolated environment.
• The API receives the invoice volume read-write only where required; configuration and certificates are mounted read-only.
• The database container runs as the documented non-root database user.
• Backup files are encrypted, access-controlled, and retained according to business policy.
• Monitoring tracks volume capacity, failed backups, and restore-test results.

Outcome

Replacing containers no longer removes primary data. A deployment rollback can reconnect the existing volumes, while the independent backup system provides recovery from corruption or accidental deletion. The team documents that the volume itself is not a backup and that database exports are the recovery authority.

Learning Points

Persistence requires both technical placement and operational discipline. Separation of code, data, and backups improves replaceability. Application-aware backups and tested restoration are more reliable than copying arbitrary live files. Permissions should be designed with least privilege rather than corrected after an outage.

9. Limitations and Contemporary Practice

Docker volumes are convenient for a single host, but larger environments may use network filesystems, cloud block storage, distributed filesystems, or orchestrator-managed persistent-volume abstractions. These systems introduce their own failure modes, latency, attachment rules, encryption options, and cost models.

Container orchestration platforms commonly separate a workload from a persistent-volume claim or equivalent resource. The conceptual principles remain unchanged: identify the data owner, define access mode and performance requirements, protect credentials, monitor capacity, and test recovery.

Infrastructure as code can make volume declarations and policies repeatable, but it does not guarantee safe data migration. Automated deployment should distinguish destructive operations from reversible changes and require approval for deleting production storage.

AI-assisted operations may help classify logs, forecast capacity, or generate backup scripts, but generated commands must be reviewed. An incorrect cleanup command, path, or retention rule can destroy data at scale. Human approval, least privilege, audit trails, and tested rollback remain essential.

Best-Practice Checklist

1. Keep important data out of the container writable layer.
2. Choose volume, bind mount, or tmpfs according to lifecycle and access needs.
3. Use explicit mount declarations and read-only mounts where possible.
4. Run services with the least privilege practical.
5. Verify UID, GID, ACL, SELinux, and AppArmor requirements.
6. Separate primary data from backup copies.
7. Use application-consistent backup methods for databases.
8. Encrypt sensitive backups and restrict access.
9. Monitor capacity, errors, latency, and backup freshness.
10. Perform documented, regular restore tests.
11. Inventory volumes before pruning or deleting them.
12. Define retention, ownership, legal hold, and secure-destruction rules.

Chapter Summary

Persistent storage decouples data from the replaceable container lifecycle. Docker volumes provide Docker-managed persistence, bind mounts expose selected host paths, and tmpfs mounts provide volatile memory-backed storage. The container writable layer and its storage driver are suitable for transient changes, not as a general database location. Volume drivers and storage drivers solve different problems.

Reliable stateful operation also depends on permissions, least privilege, application-aware consistency, independent backups, tested restoration, capacity monitoring, and explicit data lifecycle governance. A volume is not a backup, replication is not necessarily recovery, and a successful backup is demonstrated only when restoration has been tested. These principles apply from a single Docker host to larger orchestrated platforms.

Key Terms

Container writable layer: The temporary writeable filesystem layer added above image layers.

Volume: Docker-managed persistent storage identified independently of a container.

Bind mount: A mapping of a specified host path into a container.

tmpfs mount: Volatile memory-backed storage that does not normally survive container termination.

Storage driver: Component that manages image layers and container writable-layer behavior.

Volume driver: Component that provisions or presents volume storage, potentially from an external system.

Copy-on-write: A method in which data is copied to a writable layer when modification is required.

Persistence: Survival of data beyond the lifetime of a particular container.

UID/GID: Numeric user and group identifiers used in Unix permission checks.

Least privilege: Granting only the access required to perform a task.

RPO: Maximum acceptable period of data loss, expressed as time.

RTO: Maximum acceptable time to restore service.

Application-consistent backup: A backup coordinated with the application so its data is in a recoverable state.

Retention: The period for which data or backups are kept.

Restore test: A controlled operation that demonstrates a backup can be recovered and used.

Read-only mount: A mount that prevents the container from modifying the mounted data.

Copy-on-write overhead: Additional work caused by modifying data from a lower read-only layer.

Data lifecycle: The stages of data creation, use, retention, archival, and destruction.

Rootless container: A container operated without requiring the daemon or workload to have host-root privileges.

Review Questions

Short Answer Questions

1. Why is the container writable layer usually unsuitable for database data?
2. What is the principal difference between a volume and a bind mount?
3. Give two appropriate uses for a tmpfs mount.
4. Why is a volume not automatically a backup?
5. What do RPO and RTO measure?

Long Answer Questions

1. Compare volumes, bind mounts, and tmpfs mounts in terms of lifecycle, security, and practical use.
2. Explain the distinction between a storage driver and a volume driver.
3. Describe how UID, GID, permissions, ACLs, and security labeling affect mounted data.
4. Design a backup and restore process for a containerized relational database.
5. Explain why data lifecycle governance is necessary in addition to technical persistence.

Analytical / Application Questions

1. A web application loses uploaded files whenever its container is redeployed. Diagnose the design failure and propose a corrected storage architecture.
2. A database backup is created by copying its live volume while the database is under heavy write load. Evaluate the reliability of the backup and recommend a safer method.
3. A host is nearly full because of unused volumes and old backup archives. Develop a risk-controlled cleanup plan that avoids accidental data loss.

Multiple Choice Questions with Answers

Question 1

Which Docker storage option is normally managed by Docker and identified by a name?

A. A named volume

B. A tmpfs mount

C. A container writable layer

D. An image layer

Correct Answer: A. A named volume

Question 2

Which mount type is most appropriate for a host-managed configuration directory that the application only needs to read?

A. A read-only bind mount

B. A writable tmpfs mount

C. The container writable layer

D. An anonymous image layer

Correct Answer: A. A read-only bind mount

Question 3

What happens to ordinary data stored only in a container writable layer when the container is removed?

A. It is normally removed with the container

B. It is automatically copied to a backup volume

C. It is replicated to every Docker host

D. It becomes part of the image automatically

Correct Answer: A. It is normally removed with the container

Question 4

What does a storage driver primarily manage?

A. Image layers and the container writable layer

B. Business retention policy

C. Database transaction semantics

D. TLS certificates

Correct Answer: A. Image layers and the container writable layer

Question 5

Which statement about tmpfs is correct?

A. It is volatile storage backed primarily by memory

B. It always survives host reboot

C. It is the preferred location for permanent databases

D. It is identical to a named volume

Correct Answer: A. It is volatile storage backed primarily by memory

Question 6

A process runs as UID 1001, while a mounted directory is accessible only to root. What is the most direct issue?

A. The process may lack permission to read or write the directory

B. The image layers will be deleted

C. The mount automatically becomes tmpfs

D. The Docker daemon will change the UID automatically

Correct Answer: A. The process may lack permission to read or write the directory

Question 7

Which practice best demonstrates that a backup is usable?

A. Performing a controlled restore test

B. Listing the backup filename

C. Keeping the backup on the same live volume

D. Compressing it without checking the result

Correct Answer: A. Performing a controlled restore test

Question 8

What does RPO describe?

A. The maximum acceptable amount of data loss measured in time

B. The maximum number of containers on a host

C. The time required to build an image

D. The storage driver’s copy-on-write speed

Correct Answer: A. The maximum acceptable amount of data loss measured in time

Question 9

Why should a database usually write to a dedicated volume rather than the writable layer?

A. To separate important data from replaceable container storage and reduce copy-on-write concerns

B. Because volumes cannot be backed up

C. Because writable layers are always read-only

D. To make the database run as root

Correct Answer: A. To separate important data from replaceable container storage and reduce copy-on-write concerns

Question 10

Which action is riskiest without prior inventory and backup verification?

A. Running a command that prunes unused volumes

B. Inspecting a volume

C. Applying a read-only mount

D. Checking Docker disk usage

Correct Answer: A. Running a command that prunes unused volumes