3. Docker Fundamentals and Container Lifecycle

Introduction and Context

Containerization is a method of packaging an application together with its executable code, libraries, configuration, and selected runtime dependencies so that it can operate consistently across development, testing, and production environments. Docker popularized this approach by providing a practical interface to operating-system-level isolation, image distribution, networking, storage, and automation.

Docker is not a virtual machine platform in the traditional sense. A virtual machine includes a complete guest operating system and runs through a hypervisor. A Docker container normally shares the host operating system’s kernel while receiving an isolated process space, filesystem view, network stack, and resource limits. This makes containers comparatively lightweight and fast to start, although they do not provide an identical security boundary to a hardware-virtualized machine.

For DevOps professionals, Docker is important not merely because it starts processes in isolated environments, but because it establishes a repeatable lifecycle. Developers build immutable images; registries distribute them; the Docker Engine creates and supervises containers; operators inspect logs and resource usage; and automated pipelines promote tested images through environments. Kubernetes and other orchestrators later manage containers at larger scale, but effective orchestration depends on understanding the Docker objects and lifecycle examined in this chapter.

Learning Objectives

After studying this chapter, students should be able to:

1. Explain the architecture of Docker Engine and distinguish images, containers, registries, volumes, and networks.

2. Create, run, inspect, stop, restart, remove, and troubleshoot containers using the Docker CLI.

3. Explain image layers, tags, immutable digests, and the relationship between Dockerfiles and images.

4. Apply appropriate approaches to logging, resource control, persistent storage, networking, and image distribution.

5. Analyze common operational failures and select suitable diagnostic commands.

6. Evaluate security, governance, and professional practices in container-based delivery.

The Docker Object Model

Docker Engine consists principally of a client, a daemon, and an API. The Docker CLI is a client that sends commands to the Docker daemon, usually through a local Unix socket or a configured remote endpoint. The daemon manages images, containers, networks, and volumes. In contemporary installations, the engine uses lower-level components such as containerd and an OCI-compatible runtime to create and execute containers. Users generally interact with these implementation layers through the stable Docker interface.

An image is a read-only, layered package containing a filesystem snapshot and metadata such as the default command, environment variables, exposed ports, and working directory. An image is a template, not a running application. A container is a runtime instance of an image. It adds a writable container layer above the image layers and receives a process namespace, filesystem mount, networking configuration, and other runtime settings.

An image reference commonly has the form `registry/repository:tag`. For example, `registry.example.com/payments/api:2.4`. If the registry is omitted, Docker generally assumes Docker Hub; if the tag is omitted, the conventional default is `latest`. A tag is a mutable human-readable pointer. A digest, such as `sha256:...`, identifies exact image content and is therefore more suitable for reproducible deployments.

A registry stores and distributes images. Public registries support sharing, while private registries allow organizations to control access, scanning, retention, and geographical distribution. A volume is Docker-managed persistent storage whose lifecycle can be separated from that of a container. A network provides connectivity and name resolution among containers and, where configured, between containers and external systems.

From Dockerfile to Running Process

A Dockerfile is a declarative build recipe. Consider the following example:

```dockerfile

FROM python:3.12-slim

WORKDIR /srv/app

COPY requirements.txt .

RUN pip install --no-cache-dir -r requirements.txt

COPY . .

EXPOSE 8080

USER 10001

CMD ["python", "app.py"]

```

`FROM` selects a base image. `WORKDIR` establishes the process working directory. `COPY` transfers build-context files into the image. `RUN` executes a command during image construction, while `CMD` supplies the default runtime command. `EXPOSE` documents an intended listening port; it does not publish that port to the host. `USER` avoids running the application as root inside the container.

Building the image is commonly performed with:

```bash

docker build -t example/api:1.0 .

```

The final period identifies the build context. The context is sent to the daemon or builder, so a carefully designed `.dockerignore` file is important. It should exclude source-control metadata, credentials, local virtual environments, test artifacts, and unnecessary large files. Large contexts slow builds and may accidentally disclose sensitive material.

Docker images are assembled from layers. Instructions that change the filesystem often create layers, and unchanged earlier layers can be reused through the build cache. This improves performance, but it also means that secrets copied into an earlier layer may remain recoverable even if deleted in a later layer. Secrets should never be embedded in Dockerfile instructions or image layers. Multi-stage builds can compile software in one stage and copy only the required runtime artifacts into a smaller final stage.

A production-oriented image should be small enough to reduce transfer and attack surface, but minimality must not eliminate necessary diagnostic tools or certificates. Image selection should consider maintenance activity, vulnerability response, compatibility, licensing, and provenance rather than size alone.

Creating and Controlling Containers

The central command for creating and starting a container is `docker run`. For example:

```bash

docker run -d --name web -p 8080:8080 example/api:1.0

```

The `-d` option runs the container in detached mode, `--name` assigns a stable human-readable name, and `-p 8080:8080` maps host port 8080 to container port 8080. Port publishing is different from `EXPOSE`: publishing creates a host-level path to the container, whereas `EXPOSE` is metadata.

Environment-specific configuration should normally be injected at runtime:

```bash

docker run --rm --env APP_MODE=staging example/api:1.0

```

The `--rm` option asks Docker to remove the container after it exits. It is convenient for short-lived tasks, but unsuitable when post-failure inspection is required. A container’s main process is significant: when that process exits, the container normally enters the exited state, even if child processes were started. Applications should therefore run their intended foreground process rather than daemonizing themselves inside the container.

A practical lifecycle is:

```bash

docker ps

docker ps -a

docker start web

docker stop web

docker restart web

docker pause web

docker unpause web

docker rm web

```

`docker ps` lists running containers; `docker ps -a` includes stopped containers. `stop` requests graceful termination and, after a timeout, may send a forceful signal. `kill` terminates more abruptly and should be reserved for situations where graceful shutdown is ineffective. `rm` removes a stopped container, but normally does not remove named volumes unless explicitly requested.

A container can be created without being started:

```bash

docker create --name worker example/api:1.0

```

This distinction is useful in automation that separates resource creation from process execution. `docker exec` runs an additional process in an already running container:

```bash

docker exec -it web sh

```

This is valuable for diagnosis, but it should not become a substitute for correct image design or configuration management. Changes made interactively inside a container are generally ephemeral and are not reflected in the image.

Inspecting State and Diagnosing Failure

Operational diagnosis begins with observable state rather than assumptions. The following commands provide complementary evidence:

```bash

docker inspect web

docker logs web

docker stats web

docker top web

docker port web

docker events

```

`docker inspect` returns structured JSON containing configuration, mounts, network settings, state, exit code, restart count, and other metadata. It is often combined with formatting expressions:

```bash

docker inspect -f '{{.State.Status}} {{.State.ExitCode}}' web

```

`docker logs` retrieves the container’s standard output and standard error when the configured logging driver supports retrieval. Applications should generally log to these streams rather than write only to internal files. Useful options include `--follow`, `--tail`, and `--timestamps`. Log collection, retention, access control, and sensitive-data filtering remain operational responsibilities; Docker’s local logs are not a complete enterprise observability system.

`docker stats` provides live CPU, memory, network, and block-I/O information. A high CPU percentage may indicate an inefficient workload, a busy but healthy service, or a tight resource quota. Memory usage should be interpreted with awareness of caches and limits. `docker top` displays processes visible inside the container, while `docker events` helps correlate lifecycle events with external actions.

A disciplined troubleshooting sequence is as follows:

1. Confirm whether the container exists and determine its state with `docker ps -a`.

2. Read recent logs and identify the exit code.

3. Inspect command, arguments, environment, mounts, ports, and health status.

4. Test connectivity from the relevant network context.

5. Examine resource usage and host capacity.

6. Reproduce the failure with an interactive or temporary container when safe.

7. Correct the image, configuration, dependency, or resource policy rather than repeatedly restarting blindly.

A container that exits immediately may have an incorrect command, missing configuration, failed dependency connection, invalid file permissions, or an application that completed its intended one-off task. An application that cannot be reached may be listening on the wrong interface, using the wrong port, lacking a published port, or failing health checks. “It works on my machine” commonly reflects differences in image version, environment variables, mounts, architecture, or external services.

Storage: Ephemeral Layers and Persistent Data

The writable container layer is appropriate for temporary runtime files, not for durable application data. Removing or recreating a container can remove changes stored only in that layer. Docker offers several storage approaches.

A named volume is managed by Docker:

```bash

docker volume create dbdata

docker run -d --name db -v dbdata:/var/lib/postgresql/data postgres:16

```

A bind mount maps a host path into a container:

```bash

docker run --rm -v "$PWD/config:/etc/app:ro" example/api:1.0

```

Bind mounts are useful for local development and controlled host integration, but they create dependencies on host paths and permissions. The `:ro` suffix makes a mount read-only and reduces accidental modification. Volumes are usually preferable for database data managed by Docker, while bind mounts may be appropriate for source-code synchronization or explicitly managed host files.

Storage design must address backup, restoration, encryption, access permissions, capacity, and consistency. A volume is not automatically a backup. Database-aware backup procedures should be tested independently of container recreation.

Networking and Service Connectivity

Docker networks provide isolation and communication. A user-defined bridge network is generally preferable to relying on the default bridge because it supports clearer isolation and automatic container-name resolution:

```bash

docker network create appnet

docker run -d --name api --network appnet example/api:1.0

docker run --rm --network appnet curlimages/curl http://api:8080/health

```

Containers on the same user-defined network can usually reach one another by name. This allows an application to refer to `api` rather than a changing IP address. Publishing a port is unnecessary for container-to-container communication on the same network; it is needed when clients outside that network must connect.

Common network modes include bridge networking, host networking, and none. Host mode reduces network isolation and may improve specific performance characteristics, but it creates port conflicts and should be selected deliberately. Network design should define trust boundaries, exposed interfaces, DNS expectations, and the minimum connectivity required by each service.

Resource Governance and Health

Without constraints, a container can compete aggressively for host resources. CPU and memory controls make behavior more predictable:

```bash

docker run -d --name api --memory=512m --cpus=1.5 example/api:1.0

```

A memory limit can cause an out-of-memory termination if the process exceeds its allowance. CPU limits restrict scheduling capacity but do not guarantee a fixed amount of CPU time. Resource settings should be based on measurements and tested under realistic load, not selected arbitrarily.

A health check expresses whether an application is operational, not merely whether its process exists. It may be defined in an image or supplied at runtime:

```bash

docker run -d --name api \

--health-cmd='wget -qO- http://localhost:8080/health || exit 1' \

--health-interval=30s example/api:1.0

```

Health checks should be lightweight, meaningful, and resistant to transient conditions. They should not perform destructive operations or depend unnecessarily on unrelated services. A health status is an input to operational decisions; it does not by itself repair a failed application.

Registries, Tags, and Delivery Pipelines

To distribute an image, authenticate to a registry, assign a repository reference, and push it:

```bash

docker login registry.example.com

docker tag example/api:1.0 registry.example.com/team/api:1.0

docker push registry.example.com/team/api:1.0

```

A pull operation retrieves image metadata and layers:

```bash

docker pull registry.example.com/team/api:1.0

```

Teams should avoid treating `latest` as a reliable release identifier. Version tags can be useful for human communication, while immutable digests provide exact content selection. A mature pipeline builds once, tests the resulting image, scans it, records its provenance, and promotes that same artifact rather than rebuilding separately for each environment.

Image signing, software bills of materials, vulnerability scanning, admission policies, and registry access controls strengthen supply-chain governance. Scanning results require interpretation: a vulnerability may be unreachable or mitigated, while a seemingly low-severity issue may be important in a particular deployment. Automated gates should be risk-based and supported by remediation ownership.

Illustrative Case Study: Containerizing a Payment API

Background

A fictional financial-technology company, Northstar Payments, operates a Python API on manually configured virtual machines. Development and production use different Python patch levels, and deployment failures frequently arise from missing system libraries and inconsistent environment variables.

Situation and Problem

The team creates a Docker image but stores application logs inside the container filesystem. It also runs the process as root, embeds a database password in the Dockerfile, publishes the database port to the corporate network, and uses the `latest` tag in production. After a deployment, the API repeatedly restarts and the previous transaction-processing logs disappear.

Analysis

Inspection shows that the API exits because a required environment variable is absent. The image contains a secret in an earlier layer, although a later instruction deletes the visible file. The database is attached to the same network as an externally exposed API but does not require host publication. The restart loop obscures the original failure because operators inspect only the current container state rather than its logs and exit code.

Approach and Solution

The team adopts a multi-stage build, a maintained slim base image, a `.dockerignore` file, and a non-root runtime user. Configuration is injected through deployment tooling rather than embedded in the image. The database uses a named volume and is placed on a private user-defined network. Only the API port is published. Logs are written to standard output and collected by the platform. The pipeline builds an image tagged with the commit identifier, scans it, records a digest, and promotes that digest after tests. Resource limits and a meaningful health check are added.

Operators then create a temporary diagnostic container to verify DNS and API connectivity, inspect the failed container’s exit code, and validate environment configuration before rollout. Database backups are tested separately from container lifecycle operations.

Outcome

Deployments become reproducible, the database is no longer externally reachable, and failed containers retain useful logs until investigated. The organization also gains a clear release identity and a documented rollback procedure based on a previously approved image digest.

Learning Points

The case demonstrates that containerization is an operational discipline, not simply a packaging exercise. Reproducibility depends on immutable artifacts and explicit configuration; resilience depends on observable processes and resource policies; and security depends on reducing privileges, controlling networks, and protecting the software supply chain.

Risks, Governance, and Professional Practice

Containers can isolate processes, but isolation is not absolute. The Docker daemon commonly has powerful host access, and membership in a Docker administration group may effectively grant privileged control of the host. Remote daemon endpoints must therefore be strongly authenticated and protected. Images should run with the minimum required privileges, avoid unnecessary Linux capabilities, use read-only filesystems where practical, and avoid privileged mode unless its necessity is documented.

Organizations should establish policies for approved base images, patch timelines, registry permissions, secret handling, image retention, vulnerability exceptions, audit logging, and incident response. License obligations also apply to software distributed in images. Operational teams must know what data is collected in logs and who can access it; credentials, payment information, and personal data should not be written indiscriminately to standard output.

Current Developments and Best Practices

Docker remains widely used for local development, continuous integration, and packaging, while production environments frequently use containerd or Kubernetes directly. OCI image and runtime standards improve portability across tools. Rootless Docker reduces the privileges of the daemon and can lessen the impact of some host-level risks, although it introduces compatibility and operational considerations. BuildKit supports efficient caching, parallelism, secret mounts for build-time use, and improved provenance features.

Effective practice can be summarized as follows:

• Build deterministic, minimal, regularly updated images.
• Pin important dependencies and use digests for production promotion.
• Keep secrets out of Dockerfiles, images, source repositories, and ordinary logs.
• Use non-root users, narrow capabilities, read-only mounts, and private networks where appropriate.
• Treat containers as replaceable and store durable data outside their writable layer.
• Emit structured logs and expose meaningful health and metrics endpoints.
• Apply resource limits and test behavior under failure and load.
• Use automated scanning, signing, provenance records, and controlled registries.
• Diagnose from evidence—state, logs, inspection data, network tests, and resource metrics.

Chapter Summary

Docker provides a lifecycle for packaging, distributing, running, observing, and removing applications. Images are immutable, layered templates; containers are runtime instances with a writable ephemeral layer and explicit process, network, storage, and resource settings. The Docker CLI and Engine support the complete workflow from `build` and `push` to `run`, `inspect`, `logs`, `exec`, `stop`, and `rm`.

Reliable container operations require more than command familiarity. Engineers must design images carefully, separate configuration from artifacts, manage persistent data with volumes, control network exposure, impose resource limits, and use logs and inspection data for diagnosis. Registries, tags, digests, scanning, signing, and provenance connect local Docker practice with organizational software-supply-chain governance. These foundations are essential preparation for Kubernetes, where similar concerns are managed declaratively across clusters.

Key Terms

Image: A read-only, layered package containing application files and runtime metadata.

Container: A runtime instance of an image with isolated process and resource settings.

Docker Engine: The daemon, API, runtime components, and management functions that create and operate Docker objects.

Docker CLI: The command-line client used to communicate with Docker Engine.

Dockerfile: A declarative file containing instructions for building an image.

Layer: A read-only filesystem change contributing to an image or build cache.

Container writable layer: The temporary writeable filesystem layer added above image layers.

Registry: A service that stores, distributes, and controls access to container images.

Tag: A human-readable, potentially mutable label referring to an image version.

Digest: A content-addressable cryptographic identifier for exact image content.

Volume: Docker-managed persistent storage independent of a container’s writable layer.

Bind mount: A mapping of a host filesystem path into a container.

User-defined network: A Docker network created for controlled container connectivity and name resolution.

Port publishing: Mapping a host port to a port on a container.

Health check: A command or test indicating whether an application is operational.

Build context: Files made available to the image builder during a build.

Multi-stage build: A build using multiple stages to separate compilation from the final runtime image.

Rootless mode: Operation of Docker or containers without requiring a root-privileged daemon in the usual manner.

Container runtime: Software responsible for creating and executing container processes.

Software bill of materials: A structured inventory of components contained in a software artifact.

Review Questions

Short Answer Questions

1. Distinguish a Docker image from a Docker container.

2. Why is `EXPOSE` not equivalent to publishing a port with `-p`?

3. What information can `docker inspect` provide during troubleshooting?

4. Why should persistent database data not normally be stored only in the writable container layer?

5. Explain why an image digest is more reliable than the `latest` tag for production deployment.

Long Answer Questions

1. Describe the Docker container lifecycle from image construction through execution, inspection, stopping, and removal.

2. Explain Docker image layers, build caching, multi-stage builds, and the security implications of handling secrets.

3. Discuss a systematic troubleshooting method for a container that repeatedly exits or cannot be reached.

4. Compare named volumes and bind mounts, including their operational uses, risks, and backup considerations.

5. Explain how registries, vulnerability scanning, signing, provenance, and immutable promotion contribute to container supply-chain governance.

Analytical / Application Questions

1. A web container is running and its application listens on port 8080, but users outside the Docker host cannot connect. Identify at least three possible causes and specify commands or evidence you would use to distinguish them.

2. A team deploys a database using `docker run -d database:latest` without a volume. Analyze the operational risks and propose a revised command and release process.

3. An API container consumes all available host memory and is then terminated. Design a diagnostic and prevention plan using inspection, logs, resource limits, health checks, and application-level improvements.

22. Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes a Docker image from a container?

A. An image is a running process, while a container is a registry record.

B. An image is a read-only template, while a container is a runtime instance of that template.

C. An image is persistent storage, while a container is a network.

D. An image exists only on a host, while a container exists only in a registry.

Correct Answer: B. An image is a read-only template, while a container is a runtime instance of that template.

Question 2

A Dockerfile contains EXPOSE 8080, but users cannot reach the service from the host. What is the most likely missing action?

A. Adding a volume

B. Adding a second CMD instruction

C. Publishing the port with a host-to-container mapping

D. Adding a new image layer

Correct Answer: C. Publishing the port with a host-to-container mapping

Question 3

Which command is most appropriate for viewing a container’s exit code and mount configuration?

A. docker inspect

B. docker pause

C. docker tag

D. docker volume prune

Correct Answer: A. docker inspect

Question 4

Why should production deployments prefer an image digest over the latest tag?

A. A digest automatically reduces image size.

B. A digest identifies exact image content, whereas a tag may be changed.

C. A digest enables host networking.

D. A digest stores application logs persistently.

Correct Answer: B. A digest identifies exact image content, whereas a tag may be changed.

Question 5

A service must retain database data when its container is removed. Which design is most appropriate?

A. Store data only in the container writable layer.

B. Run the database with a named volume.

C. Use docker pause before removal.

D. Write data to standard output.

Correct Answer: B. Run the database with a named volume.

Question 6

Two containers on the same user-defined Docker network need to communicate. Which addressing approach is normally most appropriate?

A. Use the peer container’s name as the hostname.

B. Publish both containers on identical host ports.

C. Use the host’s public IP exclusively.

D. Disable network isolation.

Correct Answer: A. Use the peer container’s name as the hostname.

Question 7

A container exits immediately after startup. Which initial diagnostic sequence is most suitable?

A. Delete all images and rebuild the host operating system.

B. Check docker ps -a, inspect the exit code, and read the logs.

C. Publish every host port and restart repeatedly.

D. Create a second registry before examining the container.

Correct Answer: B. Check docker ps -a, inspect the exit code, and read the logs.

Question 8

Which practice best protects against accidentally retaining a secret in an image layer?

A. Copy the secret into one layer and delete it in the next.

B. Use a mutable latest tag.

C. Keep secrets out of image build layers and inject them at runtime or through secure build mechanisms.

D. Store the secret in the image label.

Correct Answer: C. Keep secrets out of image build layers and inject them at runtime or through secure build mechanisms.

Question 9

What is the primary purpose of applying --memory and --cpus options to a container?

A. To assign a registry repository.

B. To control the container’s consumption of host resources.

C. To convert a container into a virtual machine.

D. To make logs persistent.

Correct Answer: B. To control the container’s consumption of host resources.

Question 10

Which statement about docker exec is correct?

A. It builds a new image from a Dockerfile.

B. It runs an additional process in an already running container.

C. It permanently modifies the source image.

D. It removes all stopped containers.

Correct Answer: B. It runs an additional process in an already running container.