Docker Platform Architecture and Environment Setup

Introduction

Docker is a platform for developing, packaging, distributing, and running applications in containers. A container packages an application together with its user-space dependencies, configuration, and execution assumptions. Unlike a virtual machine, a container normally shares the host operating system kernel while processes remain isolated through Linux kernel mechanisms.

For DevOps practitioners, understanding Docker architecture is more important than memorising commands. A command such as docker run is the visible part of a larger system involving a client, an HTTP API, a daemon, image and container stores, containerd, and a low-level runtime. Correct environment setup also requires attention to operating-system support, permissions, networking, storage, security, and version compatibility.

Learning Objectives

By the end of this chapter, you should be able to:

• explain Docker Engine’s client-server architecture;
• distinguish the Docker CLI, daemon, API, containerd, and OCI runtimes;
• compare Docker Engine on Linux with Docker Desktop;
• identify practical Linux requirements and installation choices;
• configure Docker safely for development and administration;
• perform essential image, container, logging, inspection, and cleanup workflows;
• diagnose common environment and permission problems.

1. The Docker Platform Model

Docker uses a layered platform model. At the highest level, a developer interacts with a command-line client or another tool. The client sends requests to a Docker daemon through an API. The daemon manages images, containers, networks, and volumes. For container lifecycle operations, the daemon delegates work to containerd, which in turn uses an OCI-compatible runtime such as runc to create and execute containers.

Figure: The principal control and execution path of a Docker installation.

The arrows represent management relationships, not necessarily a single synchronous system call. The CLI communicates with the daemon; the daemon coordinates container operations through containerd; and the runtime asks the kernel to create isolated processes. Images and metadata are managed by higher layers, while the kernel provides the fundamental isolation and resource-control mechanisms.

2. Docker Engine Components

2.1 Docker client

The Docker client is the user-facing interface. The standard client is the docker command. It formats commands into API requests and displays daemon responses. The client does not normally create containers directly and does not need to run on the same machine as the daemon. A remote client can address a remote daemon, although exposing a daemon over an insecure network is dangerous.

Examples include:

docker version
docker info
docker context ls

docker version commonly reports both client and server information. If the server section cannot be retrieved, the CLI may be installed while the daemon is stopped, unreachable, or incorrectly configured.

2.2 Docker daemon

The Docker daemon, commonly called dockerd, is the long-running server process. It listens for API requests and manages:

• local image storage and image pulls;
• container creation, start, stop, restart, and deletion;
• networks and network endpoints;
• named volumes and bind mounts;
• logging configuration;
• communication with lower-level container services.

The daemon is privileged because it controls kernel resources and can mount filesystems, configure networks, and start processes. Consequently, access to the daemon is effectively highly privileged on many installations.

2.3 Docker API

The API is the contract between clients and the daemon. Local Linux communication commonly uses the Unix-domain socket /var/run/docker.sock or its equivalent. Remote communication can use TCP, preferably with mutual TLS and strict firewall controls. API version negotiation helps clients and servers of compatible versions communicate, but compatibility should not be assumed across arbitrary releases.

2.4 containerd

containerd is a container runtime manager responsible for image transfer and storage integration, container metadata, snapshots, and supervision of container lifecycle operations. Docker Engine uses it as an important intermediate layer. Its separation from the Docker daemon supports a clearer division between Docker’s user-facing platform functions and lower-level container lifecycle management.

2.5 OCI runtimes

The Open Container Initiative (OCI) defines specifications for image formats and runtime behaviour. An OCI runtime receives a bundle describing a container and asks the operating system to create the required process environment. runc is a widely used low-level runtime. Other runtimes may provide specialised isolation or virtualisation, such as stronger workload separation or hardware-specific execution.

The distinction is useful:

3. How a Container Starts

Consider docker run --name web -p 8080:80 nginx. The client sends the request to the daemon. The daemon checks whether the image is available and pulls it if necessary. It creates a writable container layer, configures networking and port publication, and asks containerd to create and start the container. containerd invokes the configured OCI runtime. The runtime uses kernel facilities to construct the process environment, after which the image’s configured entrypoint and command execute.

A container is therefore not a miniature virtual machine. It is a group of ordinary host-kernel processes with isolation and resource controls. Namespaces affect what a process can see, including process identifiers, network interfaces, mounts, and hostnames. Control groups, or cgroups, limit and account for resource use such as CPU and memory. Linux capabilities reduce the privileges of processes, and security mechanisms such as seccomp, AppArmor, or SELinux can restrict system calls and access.

Images are usually composed of immutable layers. A running container adds a writable layer above those image layers. This design supports reuse and efficient distribution, but it does not make a container’s writable layer a reliable data store. Persistent application data should normally use a volume or an intentionally managed bind mount.

4. Docker Engine, Docker Desktop, and Alternatives

Docker Engine on Linux

On a Linux server, Docker Engine commonly runs directly on the host operating system. The daemon uses the host kernel, and system services such as systemd can start and supervise it. This arrangement is efficient and common in cloud and continuous-integration environments.

Docker Desktop

Docker Desktop is an integrated application for Windows and macOS and is also available for Linux. On systems whose native environment does not provide the required Linux container kernel directly, Desktop runs Docker Engine inside a managed Linux virtual machine or equivalent subsystem. It integrates the daemon, CLI, credential handling, updates, networking, and developer-oriented configuration.

Desktop simplifies installation but introduces an additional virtualisation boundary and resource allocation settings. CPU, memory, disk image size, file sharing, and networking can affect performance. Desktop licensing terms may also apply to some organisations and larger enterprises; administrators should verify current vendor terms rather than assuming that every use case has identical licensing.

Rootless mode

Rootless Docker runs the daemon and containers without requiring a root daemon. It reduces the impact of some daemon or container compromises, but it has compatibility and feature limitations. For example, privileged networking, storage drivers, device access, and resource controls may differ from rootful operation. Rootless mode should be evaluated against workload requirements rather than adopted as a slogan.

5. Linux Requirements and Installation Planning

Before installation, verify:

1. a supported 64-bit Linux distribution and architecture;
2. a sufficiently recent kernel with namespaces, cgroups, and required filesystem support;
3. administrative access for package installation and service configuration;
4. network access to the configured package repository and image registry;
5. enough disk space for images, writable layers, volumes, and logs;
6. a plan for proxy servers, firewalls, DNS, and enterprise certificate authorities;
7. a policy for daemon access and membership of privileged groups.

Exact supported distribution versions change. Use the current official Docker documentation for the selected distribution and release. In production, prefer the vendor’s signed repository or an approved enterprise package source. Avoid unverified installation scripts because they make provenance, review, rollback, and auditing more difficult.

A typical repository-based Linux installation has this sequence:

# Update package metadata and install repository prerequisites
sudo apt-get update
sudo apt-get install ca-certificates curl

# Add the vendor repository according to the current official guide.
# Then install the Engine packages specified for the distribution.
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

# Enable and start the service
sudo systemctl enable --now docker

Package names and repository commands vary across Debian-based, RPM-based, and other distributions. The commands above are illustrative for a compatible Debian-family system; they should not be copied blindly into every environment.

Verify the result:

sudo systemctl status docker
docker version
sudo docker run --rm hello-world

The hello-world image is a diagnostic test, not an application deployment. It verifies that the client can reach the daemon, the daemon can obtain an image, and a basic container can start and exit.

6. Configuration and Security

Docker daemon configuration is commonly stored in /etc/docker/daemon.json on Linux, although the exact location and service arrangement may vary. A simple example is:

{
  "log-driver": "local",
  "data-root": "/var/lib/docker"
}

The data-root setting changes where Docker stores managed data. It must point to a suitable filesystem with correct ownership, capacity, backup policy, and performance. Changing it does not automatically migrate existing data; administrators must follow a documented migration procedure.

The local logging driver can help manage log-file growth compared with an unbounded configuration, but log retention, central collection, and rotation remain operational responsibilities. Configuration changes generally require validation and a daemon restart:

sudo dockerd --validate --config-file=/etc/docker/daemon.json
sudo systemctl restart docker

Validation support depends on the installed version; consult the applicable documentation. A restart can interrupt workloads, so perform it during a controlled maintenance period.

Socket and group permissions

By default, access to the Docker Unix socket is restricted. Adding a user to the docker group is convenient:

sudo usermod -aG docker "$USER"

The user must start a new login session before the group change is normally recognised. However, membership in the Docker group can confer root-equivalent control because a user can create a container with access to sensitive host paths or devices. Use it only under an approved privilege model. Alternatives include sudo, rootless mode, or a controlled remote API with authentication and encryption.

Never expose an unauthenticated Docker TCP socket to a network. If remote administration is required, use mutual TLS, network restrictions, certificate rotation, auditing, and least-privilege operational procedures.

7. Essential Command-Line Workflow

Discover the environment

docker version
docker info
docker system df

docker info reports server configuration, storage driver information, runtimes, images, containers, and resource details. docker system df estimates space used by images, containers, and local volumes.

Work with images

docker search nginx
docker pull nginx:stable
docker image ls
docker image inspect nginx:stable
docker image rm nginx:stable

Use explicit tags or, where reproducibility is critical, immutable digests. A tag such as latest is mutable and does not by itself identify an immutable artifact.

Run and inspect a container

docker run -d --name web -p 8080:80 nginx:stable
docker ps
docker ps -a
docker logs web
docker inspect web
docker exec -it web sh
docker stop web
docker rm web

The -d option detaches the container. -p 8080:80 publishes host port 8080 to container port 80; it does not mean that the application necessarily listens on the host interface inside the container. docker exec starts an additional process in a running container and is useful for diagnosis, not as a substitute for proper application configuration.

Use volumes and networks

docker volume create app-data
docker run -d --name db -v app-data:/var/lib/postgresql/data postgres

docker network create app-net
docker network connect app-net web
docker network ls

A named volume is managed by Docker and is usually preferable to placing important data in a temporary writable layer. In real deployments, database images also require credentials, supported environment variables, backups, health checks, and version-specific configuration.

Clean up cautiously

docker container prune
docker image prune
docker volume ls
docker system prune

Prune commands delete unused resources. They can remove artifacts needed by another project, so inspect the target resources and confirm retention requirements first. Do not treat cleanup as a replacement for capacity monitoring.

8. Troubleshooting Method

A disciplined troubleshooting sequence reduces guesswork:

1. Confirm the client and server versions with docker version.
2. Check daemon state using systemctl status docker and service logs.
3. Inspect storage, filesystem capacity, and inode availability.
4. Check image names, tags, registry credentials, DNS, proxy settings, and TLS certificates.
5. Inspect the container’s state, exit code, command, mounts, environment, and network settings.
6. Read application logs rather than assuming that a container failure is a Docker failure.
7. Reproduce with a minimal image or hello-world where appropriate.

For example, a container that exits immediately may be behaving correctly: its main process has ended. A container remains running only while its primary process remains active. Similarly, a published port does not fix an application configured to listen only on an incorrect address or port inside the container.

9. Applied Case: A Small Web Service Development Environment

Background

This illustrative case concerns a two-person development team building a web service. One developer uses Linux and another uses macOS. They need a repeatable environment and want to avoid installing different database versions directly on their laptops.

Problem

The team initially runs the web service directly on each host. Differences in language versions, database configuration, and environment variables produce inconsistent behaviour. A developer also stores database files inside a disposable container and loses local test data when recreating it.

Analysis

The team identifies three separate concerns: application execution, database persistence, and developer access. Containers can standardise the first two only if configuration is explicit and data is placed in a volume. Docker Desktop provides a consistent user experience on macOS, while Docker Engine runs directly on Linux.

Approach and solution

The team installs supported Docker releases, verifies docker version, and creates a project network. The web service and database are placed on the same user-defined network, allowing service-name-based communication. A named volume is mounted at the database’s documented data directory. Secrets are kept outside the image and are supplied through an approved development mechanism. The team records image tags, required ports, health checks, and startup instructions in version-controlled project configuration.

They test the workflow by stopping and recreating the database container, confirming that the named volume preserves data. They also inspect logs and resource usage rather than relying only on a browser test.

Outcome

The environment becomes reproducible across the two operating systems, and database data survives container replacement. The team still recognises that development containers do not automatically provide production security, backups, monitoring, or high availability.

Learning points

• Container portability depends on explicit configuration and compatible host capabilities.
• Persistent data belongs in a managed storage design, not an accidental writable layer.
• Desktop convenience does not remove the need to understand the underlying daemon and virtualisation boundary.
• Reproducibility requires controlled image versions, documented configuration, and repeatable tests.

10. Limitations, Governance, and Contemporary Practice

Docker improves packaging and operational consistency, but it is not a complete security boundary in every threat model. A container escape, unsafe bind mount, leaked secret, vulnerable image, or over-privileged capability can expose systems. Teams should scan images, minimise packages, run as a non-root user where feasible, drop unnecessary capabilities, restrict mounted paths, patch the host and daemon, and maintain software inventories.

Governance should define who may access the daemon, which registries are trusted, how images are signed or attested, how vulnerabilities are triaged, and how logs and audit records are retained. Supply-chain practices increasingly include signed images, provenance attestations, SBOMs, dependency scanning, and policy enforcement in CI/CD pipelines. These controls complement rather than replace sound runtime configuration.

Docker is also one part of a broader ecosystem. Compose-style tools help define multi-container development applications, while orchestrators such as Kubernetes introduce scheduling, service discovery, reconciliation, and cluster security concerns. A certificate-level practitioner should first master local Engine workflows before treating orchestration as a substitute for foundational understanding.

Best-Practice Checklist

• Install from a trusted, signed, supported source.
• Keep the CLI, daemon, kernel, and images patched.
• Use explicit image tags and immutable digests where reproducibility matters.
• Avoid unrestricted Docker socket access and unauthenticated TCP exposure.
• Prefer rootless operation or tightly governed privileged access where feasible.
• Store durable data in volumes or an intentional external storage system.
• Configure log rotation and central collection.
• Monitor disk space, image growth, container health, and daemon availability.
• Use least privilege, minimal images, non-root processes, and controlled capabilities.
• Document installation, configuration, rollback, backup, and recovery procedures.

Chapter Summary

Docker uses a client-server architecture. The Docker CLI or another client sends API requests to the Docker daemon, which manages images, containers, networks, and volumes. The daemon relies on containerd for important lifecycle functions and on an OCI runtime such as runc to create isolated processes using Linux kernel features.

Docker Engine commonly runs directly on Linux, whereas Docker Desktop provides an integrated environment that may run the Engine inside a managed Linux virtual machine. Installation requires supported hardware, kernel facilities, storage, networking, trusted package sources, and a deliberate privilege model. Essential workflows include verifying the daemon, pulling and inspecting images, running and diagnosing containers, managing networks and volumes, and pruning unused resources carefully.

Containers are not virtual machines and do not automatically solve persistence, security, observability, or supply-chain risk. Reliable practice combines explicit configuration, least privilege, controlled image provenance, persistent-storage design, logging, patching, and documented operational procedures.

Key Terms

Docker Engine: The Docker platform consisting principally of the daemon, client interfaces, and supporting runtime components.

Docker CLI: The command-line client used to send requests to a Docker daemon.

Docker daemon: The long-running server process that manages Docker resources.

Docker API: The programmatic interface through which clients communicate with the daemon.

containerd: A container runtime manager handling image and container lifecycle functions.

OCI: Open Container Initiative, which defines open standards for container images and runtimes.

OCI runtime: A low-level implementation that creates and executes a container process environment.

runc: A widely used OCI-compatible low-level runtime.

Image: An immutable, layered package containing a filesystem and metadata for creating containers.

Container: An isolated execution instance created from an image.

Namespace: A Linux isolation mechanism controlling what processes can see.

cgroup: A Linux control-group mechanism for resource limitation and accounting.

Volume: Docker-managed persistent storage independent of a container’s writable layer.

Bind mount: A host-path mapping made available inside a container.

Docker Desktop: An integrated Docker application that commonly provides a managed Linux environment on desktop operating systems.

Rootless mode: Operation in which the daemon and containers run without a root daemon.

Daemon socket: A local endpoint, commonly a Unix socket, used for client–daemon communication.

Image tag: A human-readable image reference that may change over time.

Image digest: A content-addressed identifier for a specific image artifact.

Container registry: A service that stores and distributes container images.

Review Questions

Short Answer Questions

1. What is the role of the Docker daemon?
2. Why does Docker use containerd and an OCI runtime as separate layers?
3. State two differences between a container and a virtual machine.
4. Why are named volumes useful?
5. What security concern is associated with unrestricted access to the Docker socket?

Long Answer Questions

1. Explain the complete processing path of a docker run command from client to Linux kernel.
2. Compare Docker Engine on Linux with Docker Desktop on a desktop operating system.
3. Describe a safe, repository-based Docker installation and verification procedure.
4. Explain how namespaces, cgroups, capabilities, and security profiles contribute to container isolation.
5. Develop a practical Docker configuration and maintenance policy for a small development team.

Analytical / Application Questions

1. A developer reports that docker version shows client details but no server details. Analyse possible causes and propose a troubleshooting sequence.
2. A database container loses all test data whenever it is recreated. Diagnose the design problem and specify a corrected storage approach.
3. An organisation wants remote Docker administration from a central automation server. Evaluate the security risks and design controls for a safer implementation.

Multiple Choice Questions with Answers

Question 1

Which component normally receives requests from the Docker CLI?

A. The Linux kernel directly

B. The Docker daemon

C. The OCI image registry

D. The container’s application process

Correct Answer: B. The Docker daemon

Question 2

What is the principal role of an OCI runtime such as runc?

A. Creating and executing the isolated container process environment

B. Managing user-facing Docker commands

C. Storing Git repositories

D. Providing image vulnerability reports

Correct Answer: A. Creating and executing the isolated container process environment

Question 3

Which Linux feature is primarily associated with limiting and accounting for container resources?

A. Namespaces

B. cgroups

C. DNS

D. TLS

Correct Answer: B. cgroups

Question 4

What does docker run -d --name web -p 8080:80 nginx do with the port option?

A. Maps container port 8080 to host port 80

B. Maps host port 8080 to container port 80

C. Exposes both ports only to the Docker registry

D. Changes the application’s internal listening port to 8080

Correct Answer: B. Maps host port 8080 to container port 80

Question 5

Why should important database data normally be stored in a Docker volume?

A. A volume makes the database image immutable

B. A volume preserves data independently of a container’s writable layer

C. A volume automatically encrypts every database record

D. A volume prevents all database failures

Correct Answer: B. A volume preserves data independently of a container’s writable layer

Question 6

Which statement about Docker group membership is most accurate?

A. It gives access only to read-only image metadata

B. It can provide highly privileged control over the host

C. It disables the Docker daemon

D. It forces all containers to run rootless

Correct Answer: B. It can provide highly privileged control over the host

Question 7

What is a key characteristic of an image tag such as latest?

A. It is always a permanent content identifier

B. It can refer to different image content over time

C. It can be used only for Windows containers

D. It identifies a running container rather than an image

Correct Answer: B. It can refer to different image content over time

Question 8

Which command is most directly useful for viewing a container’s configured mounts, network settings, and metadata?

A. docker inspect

B. docker search

C. docker login

D. docker pause

Correct Answer: A. docker inspect

Question 9

What is Docker Desktop primarily intended to provide?

A. An integrated Docker environment for desktop operating systems

B. A replacement for all Linux kernels in production servers

C. A container image registry only

D. A low-level OCI runtime only

Correct Answer: A. An integrated Docker environment for desktop operating systems

Question 10

Why is an unauthenticated Docker TCP socket dangerous?

A. It prevents images from being downloaded

B. It may allow remote users to obtain highly privileged daemon control

C. It automatically deletes all volumes

D. It limits containers to one CPU

Correct Answer: B. It may allow remote users to obtain highly privileged daemon control