Images, Layers, and Container Lifecycle

Introduction

Docker images and containers are the fundamental artifacts of containerized application delivery. An image is an immutable, distributable package containing an application, its runtime, libraries, configuration defaults, and a filesystem template. A container is a running—or stopped—instance created from an image. Understanding the distinction is essential: images are normally built, stored, tagged, scanned, and transported, whereas containers are created, started, monitored, stopped, removed, and replaced.

This chapter examines how Docker identifies images, how image layers and caching support efficient builds, how containers use copy-on-write storage, and how operators manage the container lifecycle. It also introduces logs, inspection, resource limits, and cleanup practices required for safe professional work.

Learning Objectives

By the end of this chapter, you should be able to:

• distinguish images, repositories, tags, digests, and containers;
• explain image layers, caching, and copy-on-write behavior;
• inspect images and containers using Docker commands;
• describe container lifecycle states and transitions;
• retrieve logs and diagnose basic operational problems;
• apply CPU, memory, and restart controls appropriately;
• remove unused images, containers, networks, and volumes safely; and
• evaluate practical trade-offs in image design and container operations.

1. Images, Repositories, Tags, and Digests

A Docker image is a read-only filesystem template plus metadata. The filesystem may include a Linux user-space environment, executable files, shared libraries, certificates, scripts, and application data intended to be present at startup. Image metadata identifies the default command, entrypoint, environment variables, working directory, exposed ports, user, and other execution settings.

An image is not a virtual machine. It does not normally contain a complete operating-system kernel. Containers created from Linux images share the host kernel through operating-system isolation mechanisms. Consequently, an image should contain user-space components, while the host supplies the kernel and container runtime support.

A repository is a named collection of related image versions. For example, nginx may refer to a repository hosted at Docker Hub, while registry.example.com/payments/api identifies a repository in a private registry. A repository name can include a registry hostname, namespace, and image name.

A tag is a human-readable mutable label associated with an image version. Examples include latest, 1.25, 1.25.4, and production. Tags are convenient, but they are not permanent identities. A registry administrator can move a tag so that it refers to a different image. The tag latest is only a conventional name; it does not necessarily indicate the newest, safest, or production-ready release.

A digest is a content-derived identifier, commonly written as an algorithm name followed by a hexadecimal value, such as sha256:.... A digest identifies the exact content referenced by the registry manifest. Pulling an image by digest provides reproducibility because the reference does not change when a tag is moved.

Typical references include:

Useful commands include:

docker pull nginx:1.25
docker image ls
docker image inspect nginx:1.25
docker image rm nginx:1.25
docker image tag nginx:1.25 registry.example.com/team/web:1.25

The docker image inspect command reports metadata and configuration. It should not be confused with security verification. Inspection can show what an image claims to contain; vulnerability scanning, signature verification, provenance records, and trusted build processes provide additional assurance.

Tags and reproducibility

For development, a broad tag such as python:3.12 may be convenient. For a controlled release, however, teams should use an immutable version and preferably record the digest used in deployment. A practical deployment record might contain both:

registry.example.com/team/api:2.4.1
registry.example.com/team/api@sha256:<verified-digest>

The tag supports human communication, while the digest establishes the exact artifact. A deployment system can reject unexpected digest changes or update them only through an approved release process.

2. Image Layers and Build Structure

Docker images are generally assembled from multiple read-only filesystem layers. Each instruction in a Dockerfile can contribute a layer or metadata change, although the precise representation depends on the builder and instruction. The image configuration and an ordered set of filesystem layer references together describe the image.

Consider this simplified Dockerfile:

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY src/ ./src/
CMD ["python", "-m", "src.main"]

Conceptually, the base image supplies existing layers. The application build then adds layers for files and installed dependencies. The final COPY changes when application source changes, while the dependency installation may remain reusable. This ordering is deliberate: relatively stable inputs should appear before frequently changing inputs.

Layers provide several benefits:

1. Reuse: Multiple images can share identical base or dependency layers.
2. Distribution efficiency: A registry or host need not transfer a layer already present.
3. Build caching: An unchanged instruction and its relevant inputs may reuse a previous result.
4. Version history: Image construction can be understood as a sequence of filesystem changes.

Layers do not mean that every Dockerfile line always creates an independently useful permanent layer. Modern builders may optimize execution, combine operations, or use intermediate stages. Therefore, layer behavior should be understood conceptually and verified with tools rather than assumed from a simplistic one-line/one-layer rule.

The command below displays image history:

docker image history nginx:1.25

For more detailed layer information, tools such as docker image inspect and registry or build analysis utilities can be used.

Build cache

A build cache stores the result of prior build steps. When Docker can establish that an instruction and its relevant inputs are unchanged, it may reuse the cached result rather than executing the instruction again. Cache reuse reduces build time, network traffic, and registry load.

Cache effectiveness depends on build context, instruction order, dependency files, builder configuration, and invalidation rules. A common pattern is:

COPY package*.json ./
RUN npm ci
COPY . .

If only source files change, the package manifest and dependency installation may remain cached. By contrast, placing COPY . . before dependency installation can invalidate the dependency step whenever any source file changes.

A .dockerignore file prevents unnecessary files—such as .git, local build output, test caches, and secrets—from entering the build context. This improves performance and reduces the risk of accidentally copying sensitive material.

Multi-stage builds

Multi-stage builds separate compilation tools from the runtime image:

FROM golang:1.22 AS build
WORKDIR /src
COPY . .
RUN go build -o /out/service ./cmd/service

FROM gcr.io/distroless/static-debian12
COPY --from=build /out/service /service
USER 65532:65532
ENTRYPOINT ["/service"]

The first stage contains compilers and source material. The final stage contains only the executable and required runtime content. This can reduce image size and attack surface, although compatibility, debugging, certificates, and dynamic library requirements must be checked carefully.

3. Copy-on-Write and Writable Container Storage

When a container starts, Docker combines the image’s read-only layers with a thin writable container layer. This is commonly implemented through a storage driver using a union or overlay-style filesystem, although exact behavior varies by platform and storage driver.

If a process reads a file that exists in a lower, read-only image layer, the file can be accessed without being copied into the writable layer. If the process modifies that file, the storage system may copy the file into the writable layer first; subsequent changes occur there. This behavior is called copy-on-write.

Copy-on-write has important consequences:

• The original image remains unchanged.
• Containers created from the same image can have different writable changes.
• Initial container creation can be efficient because the entire image is not duplicated.
• Repeated modification of large files can create substantial writable-layer overhead.
• Writable-layer data is normally lost when the container is removed.

The writable layer is not a suitable database strategy. Databases and other stateful services should use volumes or an external storage system. A volume exists independently of the container lifecycle and can be mounted into a replacement container.

For example:

docker volume create app-data
docker run -d --name database \
  -v app-data:/var/lib/postgresql/data \
  postgres:16

A bind mount connects a host path to a container path, while a named volume is managed by Docker. Bind mounts are useful for development and deliberate host integration; named volumes are usually preferable for Docker-managed application data.

4. The Container Lifecycle

A container has an identity, configuration, filesystem layers, network attachments, and a process model. The container’s primary process—often called PID 1 inside the container—has special operational significance. When that process exits, the container normally reaches the exited state, even if other short-lived processes existed earlier.

The common lifecycle is:

1. Created: Configuration and filesystem are prepared, but the main process has not started.
2. Running: The main process is active.
3. Paused: Processes are suspended, usually by the runtime’s freezing mechanism.
4. Stopped or exited: The main process has terminated, either successfully or with an error.
5. Restarting: The runtime is attempting to start the container again under a restart policy.
6. Removed: The container metadata and writable layer are deleted; attached volumes may or may not be deleted depending on the command and options.

Typical commands are:

docker create --name web nginx:1.25
docker start web
docker pause web
docker unpause web
docker stop web
docker start web
docker restart web
docker rm web

In practice, docker run combines creation and starting:

docker run -d --name web -p 8080:80 nginx:1.25

The -d option runs in detached mode. The -p option publishes a host port to a container port. Publishing a port does not automatically make an application secure or publicly reachable; host firewall rules, cloud security groups, proxy configuration, and application binding behavior also matter.

The docker ps command lists running containers. To include stopped containers, use:

docker ps -a

A stopped container can often be started again with its existing configuration. If a new image version or changed environment setting is required, the usual operational pattern is to create a replacement container rather than mutate the existing one. This supports immutable deployment and makes rollback clearer.

Exit codes and restart policies

The exit code of the main process provides a useful diagnostic signal. Conventionally, zero indicates successful termination, while a nonzero value indicates an error; however, interpretation depends on the application. Inspect it with:

docker inspect --format '{{.State.Status}} {{.State.ExitCode}}' web

Restart policies include no, on-failure, always, and unless-stopped. For example:

docker run -d --restart unless-stopped --name worker worker:2.0

Restart policies improve resilience to process failure, but they do not replace health checks, orchestration, alerting, or root-cause analysis. A defective application can repeatedly crash and restart, hiding the underlying problem and consuming resources.

5. Logs, Inspection, and Diagnosis

Containerized applications should normally write operational logs to standard output and standard error. Docker can then collect and display them through its logging mechanism.

docker logs web
docker logs --follow --tail 100 web
docker logs --since 10m web

The exact retention, rotation, formatting, and forwarding behavior depends on the selected logging driver. In production, teams should define retention policies and avoid unlimited local log growth.

Inspection is a structured way to examine runtime metadata:

docker inspect web
docker inspect --format '{{.State.Status}}' web
docker inspect --format '{{json .NetworkSettings.Ports}}' web

Useful diagnostic commands include:

docker top web
docker stats web
docker port web
docker diff web
docker exec -it web sh

docker exec starts an additional process in a running container. It is useful for diagnosis, but manual changes made through an interactive shell are generally ephemeral and are not a substitute for rebuilding an image or changing deployment configuration. docker diff shows files changed in the container’s writable layer, which can reveal unexpected modifications.

A disciplined diagnosis sequence is:

1. Confirm whether the container exists and whether it is running.
2. Read recent logs and inspect the exit code.
3. Check configuration, environment, mounts, ports, and networks.
4. Review resource usage and host capacity.
5. Test connectivity from the correct network location.
6. Reproduce the issue with a controlled command.
7. Correct the image, configuration, or dependency rather than repeatedly restarting blindly.

6. Resource Limits and Operational Safety

Without limits, a container may consume excessive memory, CPU time, processes, or storage. Resource controls are especially important on shared hosts.

Example:

docker run -d --name api \
  --memory 512m \
  --cpus 1.5 \
  --pids-limit 200 \
  api:2.4.1

--memory limits memory available to the container. If the workload exceeds available memory, processes may be terminated by an out-of-memory mechanism. --cpus expresses a CPU quota relative to a CPU resource; 1.5 permits approximately one and a half CPUs worth of processing over the scheduling period, subject to host capacity and runtime behavior. --pids-limit limits the number of process IDs and helps reduce process-exhaustion risks.

These controls are not performance guarantees. Actual performance depends on competing workloads, host hardware, kernel scheduling, application behavior, and I/O. Limits should be based on measurements and tested under realistic load.

Other useful controls include:

• --read-only to make the container root filesystem read-only;
• --tmpfs to provide temporary writable space in memory;
• --user to avoid running as root inside the container;
• --cap-drop and selective capability additions to reduce privilege;
• health checks to express application readiness or liveness; and
• explicit CPU, memory, and storage monitoring.

A read-only root filesystem may require writable temporary directories or application-specific mounts. Security hardening must therefore be validated against application requirements rather than applied mechanically.

7. Cleanup and Storage Governance

Over time, hosts accumulate stopped containers, unused images, dangling layers, build cache, and unused volumes. Storage pressure can cause failed deployments and service disruption.

Inspect usage first:

docker system df
docker system df -v

Remove a specific stopped container:

docker rm web

Remove an image that is no longer referenced:

docker image rm registry.example.com/team/api:old

Prune unused resources cautiously:

docker container prune
docker image prune
docker builder prune
docker system prune

The broadest commands can remove resources needed by another project. Volumes require particular care because they may contain irreplaceable data. Never treat cleanup as a substitute for backup and retention policy. In shared or production environments, automated cleanup should use explicit age, label, ownership, and approval rules.

8. Applied Case: A Failing Web-Service Deployment

Background

This illustrative case concerns a small online retailer deploying an API container to a Linux host. Developers build the image locally and deploy it with a tag named api:latest.

Problem

After a release, the container repeatedly exits. The team observes slow builds, inconsistent deployments between hosts, and a host disk warning. The database also loses test data whenever its container is removed.

Analysis

The team performs the following investigation:

docker ps -a
docker logs --tail 200 retail-api
docker inspect retail-api
docker stats --no-stream retail-api
docker system df

Logs show that the application cannot find a required configuration value. Inspection reveals that one host received a different image because the mutable latest tag was pulled after it had been moved. The image history shows that the Dockerfile copies the entire source tree before installing dependencies, causing frequent cache invalidation. The database uses the container writable layer rather than a volume. Disk usage is increased by stopped containers and obsolete build cache.

Approach and solution

The team adopts the following controls:

1. Builds are labeled with a semantic version and recorded by digest.
2. Deployment references the verified digest rather than relying only on latest.
3. Configuration is supplied through an approved environment or secret-management mechanism.
4. The Dockerfile copies dependency manifests before source code to improve cache reuse.
5. A multi-stage build produces a smaller runtime image.
6. Database storage is moved to a named volume with tested backups.
7. Memory and CPU limits are established from load-test results.
8. Logs are rotated and forwarded to centralized storage.
9. Cleanup uses scheduled, reviewed policies rather than indiscriminate pruning.

Outcome

Deployments become reproducible, build times decrease, and the database survives replacement of its container. The service still requires monitoring and application-level health checks, but the team now distinguishes image defects, configuration errors, resource exhaustion, and storage problems instead of treating every failure as a restart problem.

Learning points

The case demonstrates that container reliability depends on the entire artifact and lifecycle process. Tags, layers, storage, logs, limits, and cleanup are interconnected. A technically valid image can still produce an unreliable service if its identity is ambiguous, its state is misplaced, or its resource behavior is unmanaged.

9. Best Practices and Limitations

Recommended practices include:

• Prefer small, maintained base images, but verify compatibility and provenance.
• Pin production artifacts by digest and maintain a controlled update process.
• Use .dockerignore, multi-stage builds, and sensible instruction ordering.
• Keep images immutable and externalize persistent state.
• Run applications as non-root users where feasible.
• Define health checks that reflect meaningful application readiness.
• Set resource limits and monitor both container and host behavior.
• Send logs to a managed destination with retention and access controls.
• Scan images and dependencies, verify signatures where required, and record build provenance.
• Remove unused artifacts through tested governance procedures.

Docker’s storage and lifecycle behavior also has limitations. Copy-on-write may impose overhead for workloads that modify large files. A container is not automatically a highly available service; availability requires redundancy, scheduling, health management, data protection, and operational processes. Container isolation reduces risk but is not an absolute security boundary equivalent to a separate physical system. Privileged containers, host mounts, excessive capabilities, vulnerable kernels, and exposed management sockets can create serious security weaknesses.

Chapter Summary

Docker images are immutable templates composed conceptually of metadata and ordered filesystem layers. Repositories organize images, tags provide convenient but mutable labels, and digests identify exact content. Layer reuse and build caching improve efficiency, particularly when stable dependency inputs are placed before frequently changing source files. Multi-stage builds can separate build tools from runtime content.

A container adds a writable layer over read-only image layers using copy-on-write behavior. This layer is temporary and should not be used as the primary location for important data; volumes or external storage should hold persistent state. Containers move through created, running, paused, exited, restarting, and removed states. Operators use logs, inspection, process views, statistics, and controlled execution to diagnose behavior.

Resource limits, restart policies, health checks, non-root execution, logging governance, image provenance, and careful cleanup are essential professional practices. Reliable container operations require treating images, configuration, storage, resources, and lifecycle management as one integrated system.

Key Terms

Image: An immutable package containing filesystem content and execution metadata.

Container: An isolated runtime instance created from an image.

Repository: A named collection of related image versions.

Tag: A human-readable, mutable label referring to an image.

Digest: A content-derived identifier for an exact image manifest or artifact.

Registry: A service that stores and distributes images.

Layer: A read-only filesystem change or reusable image component.

Build cache: Stored results of prior build steps that may be reused.

Build context: Files made available to a Docker build.

.dockerignore: A file specifying context content excluded from a build.

Copy-on-write: A storage behavior that copies data into a writable layer only when modification occurs.

Writable layer: Container-specific storage placed above image layers.

Volume: Docker-managed persistent storage independent of a container’s writable layer.

Bind mount: A mount connecting a host path to a container path.

Container lifecycle: The states and transitions experienced by a container.

Restart policy: A rule governing whether Docker restarts a terminated container.

Health check: A test used to report whether an application is functioning as expected.

Resource limit: A control restricting consumption such as memory, CPU, or process count.

Image provenance: Evidence describing how, from what sources, and by whom an image was built.

Pruning: Removing unused Docker resources to reclaim storage.

Review Questions

Short Answer Questions

1. What is the difference between an image tag and an image digest?
2. Why should persistent database data normally be stored in a volume rather than a container writable layer?
3. What happens when the main process of a container exits?
4. How does a .dockerignore file improve a build?
5. What information can docker logs and docker inspect provide?

Long Answer Questions

1. Explain Docker image layers, including their benefits for reuse, distribution, and caching.
2. Describe the container lifecycle from creation through removal and identify commands associated with major transitions.
3. Explain copy-on-write behavior and discuss its implications for performance and data persistence.
4. Compare mutable tags and immutable digests in development and production deployment.
5. Discuss a comprehensive strategy for controlling container CPU, memory, process, logging, and storage consumption.

Analytical / Application Questions

1. A Dockerfile installs dependencies after COPY . ., and builds are slow whenever a source file changes. Propose a revised instruction order and explain why it improves cache reuse.
2. A production container exits repeatedly with code 1, while its image was deployed using latest. Design a diagnostic and corrective procedure that addresses both the immediate failure and deployment reproducibility.
3. A host is nearly out of disk space and runs several applications, including a database. Explain how you would identify reclaimable resources, protect persistent data, and introduce a safe cleanup policy.

Multiple Choice Questions with Answers

Question 1

Which reference identifies an exact image artifact rather than a mutable label?

A. nginx:latest

B. nginx:stable

C. nginx@sha256:abc123...

D. nginx:1.25

Correct Answer: C. nginx@sha256:abc123...

Question 2

What is the main purpose of placing dependency files before source files in a Dockerfile?

A. To disable all image layers

B. To improve reuse of dependency build-cache results

C. To make the container run as root

D. To delete the build context

Correct Answer: B. To improve reuse of dependency build-cache results

Question 3

What normally happens to data stored only in a container’s writable layer when the container is removed?

A. It is automatically copied to every image

B. It remains guaranteed in the registry

C. It is normally lost

D. It becomes a named volume

Correct Answer: C. It is normally lost

Question 4

Which command lists stopped containers as well as running containers?

A. docker ps -a

B. docker image ls

C. docker volume ls

D. docker top

Correct Answer: A. docker ps -a

Question 5

Which Docker feature is most appropriate for preserving database data across container replacement?

A. A mutable image tag

B. A named volume

C. The container exit code

D. The build cache

Correct Answer: B. A named volume

Question 6

What does the --memory 512m option primarily control?

A. The maximum image download speed

B. The container’s memory limit

C. The number of image layers

D. The registry retention period

Correct Answer: B. The container’s memory limit

Question 7

Which command follows new log output from a container?

A. docker logs --follow container

B. docker image history container

C. docker rm --follow container

D. docker volume inspect --follow container

Correct Answer: A. docker logs --follow container

Question 8

Why are multi-stage builds commonly used?

A. To place compilers in every production container

B. To eliminate the need for a base image

C. To separate build dependencies from the runtime image

D. To make tags immutable

Correct Answer: C. To separate build dependencies from the runtime image

Question 9

What is a likely risk of relying exclusively on the latest tag in production?

A. The tag cannot be pushed to a registry

B. The tag may refer to different image content over time

C. The image cannot have layers

D. The container cannot produce logs

Correct Answer: B. The tag may refer to different image content over time

Question 10

Which command provides a summary of Docker disk usage?

A. docker system df

B. docker port

C. docker pause

D. docker top

Correct Answer: A. docker system df