Docker Architecture, Standards and Environment Setup

Introduction

Docker is a platform for packaging and running applications as isolated, portable processes. It is widely used in continuous integration, continuous delivery, microservices, testing, and cloud deployment. Docker does not constitute virtualization in exactly the same sense as a traditional virtual machine. A virtual machine emulates or virtualizes hardware and normally includes a complete guest operating system. A container shares the host kernel while isolating processes, filesystems, networks, and resource consumption.

Understanding Docker architecture is essential before building images or operating production platforms. A developer who understands only commands may be able to start a container, but may not understand privilege boundaries, image provenance, storage behaviour, runtime selection, or the consequences of exposing the Docker daemon. This chapter explains the major components of Docker Engine, the client–server model, container runtimes, registries, Open Container Initiative standards, Linux isolation mechanisms, layered filesystems, and safe installation practices.

Learning Objectives

By the end of this chapter, students should be able to:

• explain the components and workflow of Docker Engine;
• distinguish the Docker CLI, daemon, containerd, and low-level runtime;
• describe images, containers, registries, and layered filesystems;
• explain the role of OCI image and runtime standards;
• relate namespaces and control groups to container isolation;
• compare rootful and rootless Docker operation;
• install and verify Docker using a secure procedure; and
• identify operational, governance, and security risks in Docker environments.

1. The Docker Architecture Model

Docker uses a client–server architecture. The Docker client issues requests, while a Docker daemon performs most container-management operations. The client and daemon may run on the same computer or communicate with a remote daemon over a protected connection.

The principal components are:

A typical request travels through several layers. For example, when a user executes docker run nginx, the CLI sends a request to the daemon. The daemon resolves the image name, contacts a registry if the image is absent locally, prepares filesystem layers, asks containerd to create the container, and uses an OCI runtime to establish the process environment.

Figure: Simplified flow from Docker client through the daemon and runtime to the host kernel.

The diagram is a logical view rather than a promise that every Docker release implements every internal boundary identically. Docker is modular, and components may be upgraded independently.

2. Docker Client and Daemon

The Docker CLI is a client of the Docker Engine API. Commands such as docker image ls, docker pull, and docker inspect are translated into API requests. The CLI is not the container runtime: closing a terminal normally does not stop a detached container because the daemon continues managing it.

The daemon, dockerd, manages persistent Docker state. Its responsibilities include:

1. receiving and authorizing API requests;
2. pulling and storing image content;
3. creating container metadata and writable layers;
4. configuring networks and volumes;
5. delegating lifecycle work to containerd; and
6. returning status, logs, and error information.

The API may be accessed through a local Unix socket or, when configured, a TCP endpoint. A local socket such as /var/run/docker.sock is convenient but highly privileged. A user who can control a rootful Docker daemon can generally cause the host to mount sensitive paths, start privileged processes, or alter host resources. Therefore, access to the socket must be treated as administrative access, not as an ordinary application permission.

A remote daemon should use authenticated and encrypted communication, typically with mutually authenticated TLS. Exposing an unauthenticated Docker API to a network is an immediate and serious security vulnerability.

3. containerd and OCI Runtimes

Docker historically contained more implementation detail inside the daemon. Modern Docker separates responsibilities. containerd is a container runtime manager focused on image transfer, content storage, snapshots, container metadata, and task execution. It can manage containers without providing the complete end-user experience of the Docker CLI.

An OCI runtime is responsible for the final creation of the container process. runc is a widely used implementation. It consumes a configuration describing the process, root filesystem, namespaces, capabilities, mounts, environment, and resource constraints. It then asks the operating-system kernel to create the isolated process.

The distinction is useful:

• Docker provides a developer-oriented management experience.
• containerd coordinates image and container lifecycle operations.
• An OCI runtime creates the actual isolated process.

Other runtimes may be used for different security or performance requirements. Examples include virtual-machine-backed runtimes and sandboxed runtimes. These can provide a stronger boundary than ordinary Linux namespaces, but may introduce compatibility or performance trade-offs.

4. Images, Containers and Registries

A Docker image is an immutable, content-addressed package containing filesystem layers and metadata. It normally includes a base filesystem, application files, a default command, environment information, and configuration instructions. An image is not a running process.

A container is a runtime instance of an image. It adds a writable layer and execution state, including a process, namespaces, networking, mounts, and resource limits. Multiple containers can be created from one image; their writable state is separate unless they deliberately share volumes or other resources.

A registry stores and distributes image manifests and layers. Public registries are convenient but should not automatically be trusted. Organisations often use private registries to control access, scanning, retention, and geographic availability.

An image reference commonly has the form:

registry.example.com/team/service:2.4

The registry identifies the service endpoint, team/service identifies the repository, and 2.4 is a tag. Tags are mutable labels. For reproducible deployment, a digest is stronger because it identifies content by cryptographic hash, for example:

registry.example.com/team/service@sha256:<digest>

A digest does not prove that software is safe; it proves that the retrieved content matches a particular content identity. Trust also requires secure build processes, provenance, vulnerability management, and access control.

5. Open Container Initiative Standards

The Open Container Initiative (OCI) develops open specifications that improve portability among container tools. Three important areas are:

• Image Specification: defines how image configuration, manifests, and filesystem layers are represented.
• Runtime Specification: defines the configuration and behaviour expected of a low-level container runtime.
• Distribution Specification: defines protocols and interactions for pushing and pulling content from registries.

Docker images and tools commonly interoperate with OCI-compliant components, although Docker includes features and workflows beyond the OCI specifications. Standards reduce dependence on one vendor and allow a container image built by one tool to be used by another compatible tool.

Standards do not eliminate all portability problems. Applications still depend on CPU architecture, kernel features, filesystem behaviour, device access, time zones, external services, and security policies. An image may be format-compatible but operationally unsuitable for a different environment.

6. Linux Isolation Mechanisms

Containers are implemented through operating-system mechanisms rather than by copying a complete operating system into every instance. The principal mechanisms are namespaces, control groups, capabilities, and security policies.

6.1 Namespaces

Namespaces provide different views of selected system resources. Important Linux namespaces include:

• PID namespace: isolates process identifiers and process visibility;
• mount namespace: provides a separate view of mounted filesystems;
• network namespace: provides interfaces, routes, ports, and firewall context;
• UTS namespace: isolates hostname and domain name;
• IPC namespace: isolates several inter-process communication resources;
• user namespace: maps container identities to different host identities;
• cgroup namespace: provides an isolated view of control-group paths.

A process may be PID 1 inside a container while having a different process identifier on the host. This is isolation of visibility and naming, not a separate kernel. A kernel vulnerability may therefore affect all workloads sharing that kernel.

6.2 Control Groups

Linux control groups, commonly called cgroups, organise processes and control or measure resource usage. Depending on system configuration, cgroups can limit CPU time, memory, process counts, block-I/O activity, and other resources.

For example, a memory limit can prevent a container from consuming unlimited memory, but it does not guarantee that the application will behave correctly when memory is exhausted. The process may be terminated by the kernel's out-of-memory handling. Resource limits should therefore be combined with application-level monitoring and sensible capacity planning.

6.3 Capabilities and Security Policies

Traditional Unix privilege is divided into capabilities such as network administration, changing file ownership, and loading kernel modules. Containers should run with the minimum capabilities required. --privileged grants broad additional access and should not be used as a routine solution to permission errors.

Security mechanisms such as seccomp, Linux Security Modules, SELinux, and AppArmor can restrict system calls and resource access. Their exact behaviour depends on the host distribution and configuration. A container boundary is not an excuse to run untrusted code without additional defence.

7. Layered Filesystems and Storage

Images are constructed from layers. Each Dockerfile instruction commonly produces a filesystem change that can be represented as a layer. Shared lower layers allow many containers to reuse the same base content, reducing storage and transfer costs.

At runtime, the container normally receives a thin writable layer above read-only image layers. This is useful for temporary changes but is not a substitute for durable storage. When a container is removed, changes in its writable layer are normally removed as well. Persistent application data should be stored in named volumes, bind mounts, or an external storage service according to the application’s requirements.

A storage driver provides the implementation of layered storage. Copy-on-write behaviour can make reading shared files efficient, but repeated modification of large files may cause copying and performance overhead. Database workloads should be tested carefully and often benefit from a volume with appropriate filesystem and I/O characteristics.

A useful design separation is:

• image layers: immutable application and dependency content;
• container writable layer: ephemeral runtime changes; and
• volumes or external services: persistent data.

This separation supports replaceable containers and simplifies rollback.

8. Installing Docker Safely

Installation procedures vary by operating system and release. The following is a safe operational method rather than a command list tied to one distribution.

8.1 Prepare the Host

Use a supported 64-bit operating system and confirm that hardware virtualization, kernel features, and filesystem requirements are appropriate. On Linux, Docker normally uses the host kernel directly. On Windows and macOS, Docker Desktop commonly runs Linux containers inside a managed virtual machine or subsystem.

Before installation:

• remove conflicting or unofficial packages where the vendor documentation advises it;
• apply operating-system security updates;
• confirm available disk space and memory;
• identify whether the machine is for development, testing, or production; and
• establish who is authorised to administer Docker.

8.2 Obtain Packages from a Trusted Source

Use the official Docker documentation and the package repository appropriate to the operating system. Verify repository signing keys according to current vendor instructions. Avoid copying installation scripts from untrusted websites. A script executed with administrative privileges has the same authority as the administrator who runs it.

Pinning a package version may be appropriate for controlled environments, but pinned versions require a patch-management process. Unmaintained pins create security debt.

8.3 Configure Access Deliberately

Rootful Docker provides broad host authority through the daemon. On Linux, adding a user to a docker group may allow that user to control the daemon and therefore escalate privileges. This group must be governed as an administrative group.

Rootless Docker maps container-root identities to an unprivileged host account. It can reduce the impact of daemon compromise, but it has limitations involving networking, storage drivers, privileged ports, and some workloads. Rootless mode is a security option, not a universal replacement for host hardening.

Do not expose the daemon socket publicly. If remote management is necessary, use encrypted, authenticated communication and restrict network access with firewalls and policy controls.

8.4 Verify the Installation

After installation, verify both functionality and security:

docker version
docker info
docker run --rm hello-world

The test container confirms that the client can communicate with the daemon, an image can be obtained, and a container can start. It does not prove that the installation is production-secure. Also inspect daemon startup configuration, storage location, logging behaviour, enabled plugins, user permissions, and network exposure.

A basic operational check should include:

docker ps
docker image ls
docker system df

Use docker system df carefully. Cleanup commands can remove unused images, stopped containers, networks, or volumes. In particular, deleting volumes may destroy data. Production cleanup should be planned, reviewed, and backed up.

9. Applied Case: A Secure Development Workstation

This is an illustrative case.

Background

A software team wants a common development environment for a web API and a database. Developers use different operating systems, and local dependency installation has become inconsistent.

Problem

The team proposes installing Docker from an unofficial script and granting every developer unrestricted access to a remote Docker daemon. They also plan to store database files only in the container writable layer.

Analysis

The proposal creates three risks. First, the installation source may be untrusted. Second, unrestricted daemon access can become host-level administrative access. Third, the database will lose data when its container is replaced. The team also needs to distinguish development convenience from production security.

Approach and Solution

The team instead uses the official installation method for each supported operating system, records the selected Docker version, and applies host updates. Developers use a local daemon or an approved rootless configuration. A private registry stores approved images. The database uses a named volume, while source code is mounted for development. Images are built with a small trusted base, non-root application users, pinned dependency versions, and automated vulnerability and secret scanning. The remote daemon, where required, is restricted with encrypted authentication and network policy.

Outcome

The team obtains consistent application dependencies without treating Docker access as an ordinary user privilege. Database data survives container replacement, and image provenance is documented. The workflow remains convenient while reducing avoidable security and operational risks.

Learning Points

Containerisation improves consistency but does not automatically provide security, persistence, or reproducibility. Those properties require explicit design decisions involving identity, storage, supply-chain controls, and lifecycle management.

10. Limitations, Governance and Contemporary Practice

Docker containers share a kernel, so they are not equivalent to security boundaries provided by separate physical machines or virtual machines. Kernel vulnerabilities, excessive capabilities, unsafe mounts, exposed sockets, and malicious images can defeat assumptions about isolation.

Organisations should govern:

• approved base images and registries;
• image scanning and patch response;
• software bills of materials and provenance;
• secrets management;
• resource limits and logging;
• network segmentation;
• backup and recovery of persistent data; and
• access to daemon APIs and registries.

Secrets should not be embedded in image layers or Dockerfiles because layers may remain accessible even after a later instruction attempts to delete a file. Use a secret-management system or controlled runtime injection mechanism.

Modern container platforms increasingly use image signing, provenance attestations, SBOM generation, policy admission, rootless execution, sandboxed runtimes, and automated vulnerability assessment. These controls are complementary. A scanner may identify a vulnerable package, but an organisation still must decide whether the vulnerability is exploitable, whether a patch is available, and how quickly deployment should be changed.

Best-Practice Checklist

1. Use official or organisation-approved installation sources.
2. Keep Docker, the host kernel, and base images patched.
3. Treat rootful daemon access and Docker group membership as privileged.
4. Never expose an unauthenticated daemon endpoint.
5. Prefer immutable, versioned images and digest pinning for controlled releases.
6. Run applications as non-root users where practical.
7. Drop unnecessary Linux capabilities and avoid --privileged.
8. Apply memory, CPU, and process limits to shared environments.
9. Store persistent data in managed volumes or external services.
10. Scan images, protect secrets, record provenance, and test recovery procedures.

Chapter Summary

Docker implements a client–server architecture in which the Docker CLI communicates with the Docker daemon. The daemon manages images, containers, networks, and volumes, while containerd coordinates lifecycle operations and an OCI runtime such as runc creates the isolated process. Registries distribute image manifests and layers, and OCI standards support interoperability between tools.

Containers rely on kernel mechanisms including namespaces, control groups, capabilities, and security policies. Images use immutable filesystem layers, while containers add ephemeral writable state; durable data should therefore use volumes or external storage. Safe installation requires trusted packages, controlled privileges, secure daemon communication, host maintenance, and post-installation verification. Docker improves portability and repeatability, but it does not remove the need for security governance, supply-chain controls, monitoring, and operational discipline.

Key Terms

Docker Engine: The Docker platform components used to build, distribute, and run containers.

Docker daemon: The background service that manages Docker resources and processes API requests.

Docker CLI: The command-line client used to communicate with the Docker API.

Container: A running, isolated process created from an image.

Image: An immutable package of filesystem layers and execution metadata.

Registry: A service that stores and distributes container images.

Tag: A mutable human-readable label for an image reference.

Digest: A content identity, normally represented by a cryptographic hash.

containerd: A container lifecycle and image-management component.

OCI: Open Container Initiative, a project defining open container standards.

Runtime specification: An OCI description of how a runtime creates and executes a container.

Namespace: A kernel mechanism that provides an isolated view of a resource.

Control group: A kernel mechanism for organising, measuring, and limiting process resources.

Capability: A discrete unit of operating-system privilege.

Layer: An immutable filesystem change used to construct an image.

Writable layer: The ephemeral container-specific layer above image layers.

Volume: Persistent storage managed or attached independently of a container.

Rootless mode: Operation in which Docker and containers run without host-root privileges.

Daemon socket: An endpoint through which clients communicate with the Docker daemon.

Image provenance: Evidence describing where and how an image was produced.

Review Questions

5 Short Answer Questions

1. What is the relationship between the Docker CLI and Docker daemon?
2. Why is access to a rootful Docker socket considered privileged?
3. State two responsibilities of containerd.
4. What is the difference between an image and a container?
5. Why should persistent database data generally not be kept only in a container writable layer?

5 Long Answer Questions

1. Explain the complete request path when a user runs an image that is not present locally.
2. Compare namespaces, control groups, capabilities, and security policies as container-isolation mechanisms.
3. Explain the purpose of OCI image, runtime, and distribution specifications.
4. Describe a safe procedure for installing and verifying Docker on a development host.
5. Discuss the benefits and limitations of layered filesystems for application deployment.

3 Analytical / Application Questions

1. A team wants to expose the Docker daemon on an unauthenticated TCP port so that a CI server can start containers. Analyse the risks and propose a safer design.
2. A containerised database loses all recent records whenever the application is redeployed. Diagnose the storage design and recommend a corrected architecture.
3. An image works on one host but fails on another with a different CPU architecture and kernel configuration. Use Docker and OCI concepts to explain why format portability does not guarantee operational portability.

Multiple Choice Questions with Answers

Question 1

Which component normally creates the final isolated container process?

A. Docker CLI

B. OCI runtime such as runc

C. Container registry

D. Dockerfile parser

Correct Answer: B. OCI runtime such as runc

Question 2

What is the primary purpose of a container registry?

A. To limit CPU usage

B. To provide Linux namespaces

C. To store and distribute images

D. To replace the host kernel

Correct Answer: C. To store and distribute images

Question 3

Which statement best describes a Docker image?

A. A running process with a writable filesystem

B. An immutable package of layers and metadata

C. A physical virtual machine disk

D. A network firewall rule

Correct Answer: B. An immutable package of layers and metadata

Question 4

Which Linux mechanism limits and measures resources such as memory and CPU?

A. Namespaces

B. Control groups

C. Image manifests

D. Registry tags

Correct Answer: B. Control groups

Question 5

Why is a Docker image digest useful in deployment?

A. It changes automatically whenever a container starts

B. It identifies particular image content

C. It grants root privileges to the container

D. It stores application passwords securely

Correct Answer: B. It identifies particular image content

Question 6

What is a major risk of exposing an unauthenticated Docker daemon over TCP?

A. Images become read-only

B. The host may be remotely controlled through the daemon

C. Containers cannot use volumes

D. OCI standards stop applying

Correct Answer: B. The host may be remotely controlled through the daemon

Question 7

Which storage choice is generally most appropriate for data that must survive container replacement?

A. The container writable layer

B. A named volume or external storage service

C. A temporary PID namespace

D. An image tag

Correct Answer: B. A named volume or external storage service

Question 8

What does a PID namespace primarily isolate?

A. Process identifiers and process visibility

B. Registry authentication

C. Image compression

D. CPU architecture

Correct Answer: A. Process identifiers and process visibility

Question 9

Which practice best reduces unnecessary container privilege?

A. Use --privileged for every container

B. Run all applications as host root

C. Drop unneeded capabilities and use a non-root user

D. Expose the daemon without authentication

Correct Answer: C. Drop unneeded capabilities and use a non-root user

Question 10

What is rootless Docker intended to reduce?

A. The need for image layers

B. The impact of requiring host-root privileges

C. The need for a container runtime

D. The size of every registry manifest

Correct Answer: B. The impact of requiring host-root privileges