5. Docker Networking, Storage and Multi-Container Applications

Introduction and Context

A container is useful only when it can communicate with the services on which it depends and when its data can survive beyond the life of the container itself. Networking and storage therefore transform Docker from a tool for launching isolated processes into a platform for assembling complete application environments. A web application may require a reverse proxy, application server, database, cache, message broker, and monitoring agent. These components must discover one another, exchange traffic securely, and preserve selected data across upgrades and restarts.

Docker provides abstractions for these requirements through network drivers, virtual Ethernet interfaces, embedded DNS, port publishing, bind mounts, named volumes, and Docker Compose. The abstractions simplify development, but they do not eliminate architectural decisions. A developer must distinguish traffic intended for other containers from traffic intended for external clients, determine which data is disposable, select an appropriate storage mechanism, and ensure that a multi-container environment can be recreated consistently.

This chapter develops those decisions systematically. It connects the container concepts introduced earlier with practical application architecture and prepares the learner for later study of Kubernetes networking, persistent volumes, service discovery, and declarative orchestration.

Learning Objectives

By the end of this chapter, the learner should be able to:

1. Explain Docker’s network namespaces, virtual bridges, drivers, DNS, and port publishing.

2. Select and configure bridge, host, none, overlay, and macvlan networking appropriately.

3. Distinguish container-to-container communication from host-to-container access.

4. Compare bind mounts, named volumes, and temporary in-container storage.

5. Design backup, permission, and lifecycle strategies for container data.

6. Construct reproducible multi-container environments with Docker Compose.

7. Diagnose common networking and storage failures.

8. Evaluate security, governance, performance, and operational implications of design choices.

5.1 Docker Networking as an Application Boundary

Every container has its own network namespace by default. The namespace provides an isolated view of network interfaces, routing tables, and ports. A process listening on port 8080 inside a container is not automatically listening on port 8080 on the host. This separation is central to container isolation.

Docker commonly connects containers to a user-defined bridge network. Internally, Docker creates a virtual bridge on the host and connects containers to it through virtual Ethernet pairs. One end resides in the container namespace and the other is attached to the bridge. Containers on the same bridge can communicate through private IP addresses, while the host can route traffic through the bridge according to Docker’s rules.

A useful distinction is between three traffic paths:

• Container-to-container traffic: for example, an application connecting to `db:5432`.
• Container-to-host or container-to-internet traffic: for example, an application retrieving a package or calling an external API.
• External-client-to-container traffic: for example, a browser connecting to a web service through the host’s published port.

These paths have different configuration and security implications. A database generally needs to be reachable by the application but does not need a published host port. Publishing the database port unnecessarily increases its attack surface.

Port Publishing and Exposure

The `EXPOSE` instruction in a Dockerfile documents the port a process is expected to use; it does not publish that port. Port publishing is performed at container creation, commonly with `docker run -p HOST_PORT:CONTAINER_PORT image`. For example:

```bash

docker run --name web -p 8080:80 nginx

```

A request to port 8080 on the host is forwarded to port 80 in the container. The mapping can specify an address, such as `127.0.0.1:8080:80`, restricting access to the local host. Without that restriction, the service may be reachable through the host’s network interfaces, subject to firewall rules.

The port inside the container must match the port on which the application actually listens. A frequent error is publishing a port while the application binds only to `127.0.0.1` inside the container. In that situation, other interfaces cannot reach it. Services intended to receive container or external traffic normally bind to `0.0.0.0` within their container, while access control is enforced through networks, firewalls, proxies, and application authentication.

5.2 Network Drivers and Their Uses

Docker network drivers provide different connectivity models. The choice should follow the required isolation, performance, portability, and deployment scope.

Bridge Networks

The bridge driver is the normal choice for containers on a single Docker host. The default `bridge` network offers basic connectivity, but user-defined bridge networks are preferable for application systems. They provide automatic DNS-based service discovery, better isolation, and more predictable administration.

A network can be created with:

```bash

docker network create app-net

docker run -d --name api --network app-net my-api:1.0

docker run -d --name cache --network app-net redis:7

```

The API can address the cache by the name `cache`. Containers may be attached to multiple networks, allowing a reverse proxy to connect to both a public-facing network and an internal application network, while the database remains only on the internal network.

Host Networking

With the host driver, the container shares the host’s network namespace. Port translation is avoided, which may benefit specialized high-performance or network-monitoring workloads. However, isolation is substantially reduced: the container uses host interfaces and cannot independently bind ports already occupied on the host. Host networking is therefore a deliberate exception rather than a general default.

None Networking

The `none` driver creates a container with no externally useful network connectivity. It is appropriate for batch processing, cryptographic operations, or highly isolated workloads that do not communicate over a network. Applications requiring network access will fail unless interfaces are configured through other mechanisms.

Overlay Networking

Overlay networks connect containers across multiple Docker hosts, traditionally in Docker Swarm or related clustered environments. They encapsulate traffic over the underlying network and support service communication across nodes. Overlay design introduces concerns such as encryption, MTU, node membership, routing, and failure detection. In Kubernetes environments, networking is provided by the cluster’s container network interface rather than by ordinary single-host Docker bridge configuration.

Macvlan and IPvlan

Macvlan gives containers addresses that appear as distinct devices on the physical network. It can assist with legacy systems that require direct Layer 2 identity, but it complicates routing, host-to-container communication, address management, and security controls. IPvlan offers related capabilities with different handling of Layer 2 identity. These drivers should be selected only when integration requirements justify their operational complexity.

5.3 DNS and Service Discovery

A user-defined Docker network includes an embedded DNS service. Container names and network aliases can resolve to container IP addresses. In a Compose application, a service name such as `database` becomes a stable logical address even if the database container is recreated with a different IP address.

This leads to an important design principle: application configuration should normally use service names and container ports, not host-published ports. If the API connects to a database in the same Compose project, its connection string should resemble `postgresql://database:5432/app`, not `localhost:5433`. Inside the API container, `localhost` means the API container itself.

DNS records reflect the dynamic nature of containers. Applications should tolerate connection failures during startup and retry when appropriate. `depends_on` can control a limited startup order in Compose, but it does not automatically prove that a service is ready to accept requests. Health checks and application-level retry logic are more reliable than startup ordering alone.

Aliases, custom DNS servers, and extra host mappings can be useful, but hard-coded IP addresses are fragile. Service discovery should remain at the logical-name level wherever possible.

5.4 Storage: State Beyond the Container

A container’s writable layer is ephemeral from an application design perspective. Data written there disappears when the container is removed, although it may remain in Docker’s storage until garbage collection. Rebuilding an image or replacing a container must not destroy important database records, uploaded files, audit logs, or generated artifacts.

Storage decisions begin with classification:

• Ephemeral data: caches, temporary files, and process-local working data.
• Persistent application data: database files, user uploads, and durable queues.
• Configuration: source-controlled settings, secrets references, and environment-specific values.
• Shared artifacts: files intentionally exchanged between services or tools.

Bind Mounts

A bind mount maps a host path into a container:

```bash

docker run --rm -v "$PWD/src:/workspace/src" -w /workspace node:20 npm test

```

Bind mounts are valuable for local development because source-code changes on the host become visible immediately. They also support access to host-managed configuration or specialized files. Their weaknesses include host-path dependence, permission mismatches, accidental modification of host data, and reduced portability. A path that exists on one developer’s workstation may not exist on another.

Read-only mounts should be used when the container only needs to read data:

```bash

-v "$PWD/config:/app/config:ro"

```

Named Volumes

A named volume is managed by Docker rather than by a project-specific host path:

```bash

docker volume create pgdata

docker run -d --name postgres -v pgdata:/var/lib/postgresql/data postgres:16

```

Named volumes are well suited to databases and other persistent service data. They have a stable lifecycle independent of individual containers and are easier to reference in Compose. They are not automatically a backup system: administrators must still define backup, restore, retention, encryption, and migration procedures.

Temporary and In-Memory Storage

Some workloads benefit from temporary storage that is removed with the container. A `tmpfs` mount stores files in host memory and avoids writing sensitive temporary information to persistent disk. It is appropriate for short-lived secrets or high-speed scratch data, subject to memory constraints. It is not appropriate for data that must survive a restart.

Permissions and Ownership

A container process may run under a numeric user and group that differ from the host owner of a bind-mounted directory. The resulting permission errors are often incorrectly attributed to Docker networking or image defects. Professional solutions include using a consistent user ID, preparing directories with controlled ownership, using group permissions, avoiding privileged containers, and documenting platform-specific behavior. On SELinux-enabled systems, mount labeling options such as `:Z` or `:z` may be required, but they should be applied with an understanding of their sharing implications.

5.5 Docker Compose as a Declarative Application Model

Docker Compose describes an application made of services, networks, volumes, environment variables, health checks, and policies in a YAML file. Instead of remembering a sequence of commands, a team can version the desired environment and reproduce it on another machine.

Consider the following illustrative development configuration:

```yaml

services:

web:

build: ./web

ports:

• "8080:8080"

environment:

DATABASE_URL: postgres://app:secret@db:5432/app

volumes:

• ./web:/workspace

depends_on:

db:

condition: service_healthy

networks:

• front
• back

db:

image: postgres:16

environment:

POSTGRES_DB: app

POSTGRES_USER: app

POSTGRES_PASSWORD: secret

volumes:

• db-data:/var/lib/postgresql/data

healthcheck:

test: ["CMD-SHELL", "pg_isready -U app -d app"]

interval: 5s

timeout: 3s

retries: 10

networks:

• back

volumes:

db-data:

networks:

front:

back:

internal: true

```

The `web` service is reachable from the host through port 8080. The database is not published and is reachable only through the `back` network. The web service joins both networks, acting as the controlled connection between external traffic and the internal database. The named volume preserves database files. The health check measures readiness more meaningfully than process existence.

Compose service names supply DNS names within the project network. Project names also scope generated resources, helping separate environments. Profiles can enable optional services such as administration tools. Variable substitution and multiple Compose files can separate common configuration from local or production-specific values, but secrets should not be committed as plain text merely because they are stored in YAML.

A typical workflow is:

```bash

docker compose config

docker compose up --build -d

docker compose ps

docker compose logs -f web

docker compose down

```

`docker compose down` removes containers and networks but normally preserves named volumes unless `-v` is supplied. This distinction is operationally significant: deleting volumes can destroy development database state.

5.6 A Method for Designing Multi-Container Systems

A sound design begins with service boundaries rather than with commands. Identify independently deployable components, their protocols, their data ownership, and their failure behavior. Then apply the following reasoning sequence:

1. Map communication: distinguish public entry points, internal calls, asynchronous messaging, and administrative access.

2. Create network zones: place public proxies, application services, and data services on the smallest networks that satisfy communication needs.

3. Classify data: mark each path as ephemeral, persistent, shared, secret, or generated.

4. Choose storage: use bind mounts for source-oriented development, named volumes for Docker-managed persistent state, and external storage systems for production-scale durability where appropriate.

5. Define readiness: add health checks and retry logic; do not equate container start with application readiness.

6. Make configuration portable: use service names, environment substitution, documented defaults, and versioned Compose files.

7. Test failure: restart services, remove containers, fill disks, interrupt networks, and restore backups.

8. Secure the defaults: minimize published ports, drop privileges, use read-only mounts where possible, and separate secrets from ordinary configuration.

This method prevents a common anti-pattern in which every service is placed on one unrestricted network and every port is published for convenience.

Applied Case: Reproducible Development for a Retail API

Background

The fictional company Northstar Retail maintains a Python API, a PostgreSQL database, and a Redis cache. Developers previously installed these dependencies locally. Differences in operating systems, database versions, and abandoned local processes produced inconsistent test results.

Situation and Problem

The team wanted a one-command development environment. The database required persistence, source code needed live editing, and Redis had to be reachable by the API but not exposed to the office network. Developers also needed a reliable way to know when PostgreSQL was ready.

Analysis

The API is the only public application endpoint. PostgreSQL and Redis are internal services. Source code should use a bind mount because rapid editing is central to development. Database files should use a named volume because they must survive API container recreation. Service names should be used in connection strings. A health check is required because PostgreSQL may take several seconds to initialize.

Approach and Solution

The team created an `app-net` network for internal communication and published only the API port. The Compose file defined `api`, `db`, and `redis` services. The API joined the application network; the database and cache were not assigned host ports. `db-data` stored PostgreSQL data. The API used `db:5432` and `redis:6379`, while a health check tested database readiness. A separate production configuration removed the source bind mount and injected secrets through the organization’s secret-management process.

The team documented `docker compose up --build`, migration commands, log inspection, and a reset command that explicitly warned users before deleting the volume. CI used a clean project and ran integration tests against the same service topology.

Outcome

Developers obtained a consistent environment and could recreate containers without losing database state. The application’s internal addresses remained stable even when container IP addresses changed. Restricting publication reduced accidental exposure of internal services. CI failures became easier to reproduce because the dependency versions and network relationships were declared in version control.

Learning Points

The case demonstrates that Compose is not merely a convenience wrapper around `docker run`. It is an executable description of application relationships. Good results depend on correct network boundaries, explicit data lifecycle decisions, health validation, and environment-specific configuration.

Challenges, Risks, and Common Mistakes

The most common networking mistake is confusing `localhost` with a service name. Another is publishing every port, including database and cache ports, to simplify debugging. A third is relying on container IP addresses, which change when containers are recreated. Developers may also assume `depends_on` guarantees readiness, or place a health check on the wrong process.

Storage errors include writing important data into the writable layer, deleting volumes during cleanup, mounting an empty host directory over files supplied by an image, and assuming named volumes are backed up automatically. Bind mounts can expose sensitive host files if paths are overly broad. Database files should not normally be shared concurrently by unrelated containers through a common volume unless the database system explicitly supports that arrangement.

Performance differs by operating system and Docker implementation, particularly for intensive bind-mounted file workloads. Excessive file watching can make development slow. In production, databases may require dedicated storage, controlled I/O characteristics, replication, and provider-managed backup rather than a local Docker volume.

Security, Governance, and Professional Practice

Network configuration is an access-control decision. A service should be connected only to networks it needs, and administrative interfaces should be restricted. Published ports must be reviewed as part of threat modeling and firewall policy. Encryption in transit may be required even inside an overlay or development network when data sensitivity, compliance, or trust boundaries justify it.

Secrets such as passwords, tokens, and certificates should not be committed to a public repository or embedded permanently in an image layer. Development Compose files should use safe sample values, local secret files excluded from version control, or an approved secret manager. Production deployments require auditability, least privilege, image provenance, vulnerability scanning, and documented retention and deletion practices.

Data protection also has an ethical dimension. Logs, uploaded documents, and database volumes may contain personal or confidential information. Teams must define who can access them, how long they are retained, how backups are encrypted, and how data is securely deleted. Convenience must not override privacy or regulatory obligations.

Current Developments and Best Practices

Modern Compose implementations increasingly support profiles, health-aware workflows, build metadata, development watch modes, and integration with external secret and registry systems. Containerized development is also being integrated with Dev Containers and automated CI environments, allowing editor tooling and dependency services to be described consistently.

Best practice is not to make development and production identical in every implementation detail; rather, they should share meaningful service contracts, configuration names, health expectations, and testable infrastructure assumptions. Production may replace Compose-managed databases with managed services, use orchestration platforms, and apply centralized observability. The underlying principles—explicit connectivity, durable storage, readiness validation, least privilege, and reproducibility—remain applicable.

Chapter Summary

Docker networking isolates containers while providing controlled communication through drivers, virtual interfaces, bridges, DNS, and published ports. User-defined bridge networks are the usual single-host choice, whereas host, none, overlay, macvlan, and IPvlan address specialized requirements. Internal services should communicate through service names and container ports; only necessary endpoints should be published externally.

Container writable layers are not a dependable persistence mechanism. Bind mounts support host-oriented development, named volumes support Docker-managed durable state, and temporary mounts support disposable or memory-resident data. Every persistent design requires backup, restore, permission, security, and lifecycle decisions.

Docker Compose provides a declarative model for services, networks, volumes, health checks, and configuration. A robust multi-container environment separates public and internal networks, uses health checks and retries, avoids hard-coded IP addresses, protects secrets, and tests failure and recovery. These practices provide a foundation for more advanced orchestration systems.

Key Terms

Bridge network: A Docker network that connects containers on one host through a virtual bridge.

Bind mount: A mapping of a host filesystem path into a container.

Container port: A port on which a process listens inside a container namespace.

Docker Compose: A declarative tool for defining and running multi-container applications.

Embedded DNS: Docker’s internal name-resolution service for containers on user-defined networks.

Health check: A command or probe used to assess whether a service is ready or functioning.

Host network: A mode in which a container shares the host’s network namespace.

Macvlan: A network driver that gives containers distinct virtual network identities on a physical network.

Named volume: Docker-managed persistent storage identified by a logical name.

Network namespace: An isolated Linux networking environment containing interfaces, routes, and ports.

Overlay network: A virtual network spanning multiple container hosts.

Port publishing: Forwarding traffic from a host port to a container port.

Readiness: The condition in which a service can accept and correctly process requests.

Service discovery: The process by which one service locates another through a logical address.

Tmpfs mount: Temporary memory-backed storage mounted into a container.

User-defined network: A Docker-created network with explicit configuration and DNS-based discovery.

Writable layer: The container-specific filesystem layer that is normally removed with the container.

Network alias: An additional logical name by which a container can be addressed.

Review Questions

Short Answer Questions

1. Why does publishing a port differ from declaring `EXPOSE` in a Dockerfile?

2. What does `localhost` refer to inside a container?

3. State two reasons for preferring user-defined bridge networks to the default bridge.

4. When is a bind mount more appropriate than a named volume?

5. Why does `depends_on` alone not guarantee service readiness?

Long Answer Questions

1. Explain Docker’s principal network drivers and evaluate the situations in which each may be selected.

2. Compare bind mounts, named volumes, writable container layers, and tmpfs mounts.

3. Describe how Docker’s embedded DNS supports resilient service discovery.

4. Design a secure Compose topology for a reverse proxy, application server, database, and cache.

5. Discuss the governance and security issues involved in managing persistent container data.

Analytical / Application Questions

1. An API container cannot connect to PostgreSQL using `localhost:5432`, although PostgreSQL is healthy. Diagnose the problem and propose the correct configuration.

2. A development team publishes ports for its database, cache, and API. Analyze the security consequences and redesign the network exposure.

3. A developer removes and recreates a database container and loses all records. Identify the storage design failure and propose a recovery and prevention strategy.

22. Multiple Choice Questions with Answers

Question 1

A Compose service named `api` must connect to a PostgreSQL service named `db` on the same network. Which address is normally correct?

A. localhost:5432

B. db:5432

C. 127.0.0.1:5432

D. host.docker.internal:5432

Correct Answer: B. db:5432

Question 2

What is the primary purpose of publishing a Docker port?

A. To install a package inside the image

B. To make a container port reachable through a host port

C. To create a persistent volume

D. To assign a fixed container IP address

Correct Answer: B. To make a container port reachable through a host port

Question 3

Which storage option is most suitable for live source-code editing during local development?

A. Bind mount

B. Anonymous network

C. Host networking

D. Tmpfs only

Correct Answer: A. Bind mount

Question 4

Which statement about Docker’s `EXPOSE` instruction is correct?

A. It automatically opens the port on the host firewall

B. It publishes the port to every network interface

C. It documents an intended container port but does not itself publish it

D. It creates a named volume for the port

Correct Answer: C. It documents an intended container port but does not itself publish it

Question 5

A database should be reachable by an API container but not by external clients. Which design is most appropriate?

A. Publish the database port publicly

B. Place both services on an internal user-defined network and publish only the API

C. Use host networking for both services

D. Assign the database a public IP address

Correct Answer: B. Place both services on an internal user-defined network and publish only the API

Question 6

Why is a Compose health check useful even when `depends_on` is configured?

A. It changes the database schema automatically

B. It verifies service readiness rather than merely container start order

C. It encrypts all network traffic

D. It replaces the need for application retries in every case

Correct Answer: B. It verifies service readiness rather than merely container start order

Question 7

What is a major operational risk of storing database files only in a container’s writable layer?

A. The database cannot listen on TCP

B. The data may be lost when the container is removed

C. The image becomes automatically encrypted

D. DNS resolution stops working

Correct Answer: B. The data may be lost when the container is removed

Question 8

Which driver is designed to connect container networks across multiple Docker hosts?

A. None

B. Bridge

C. Overlay

D. Tmpfs

Correct Answer: C. Overlay

Question 9

A containerized application must read configuration from the host but must not modify it. Which mount option is appropriate?

A. :rw

B. :ro

C. :privileged

D. :publish

Correct Answer: B. :ro

Question 10

Which practice best improves reproducibility in a multi-container development environment?

A. Hard-code changing container IP addresses

B. Publish every internal service port

C. Version the Compose configuration and use service names

D. Store all application data in temporary memory

Correct Answer: C. Version the Compose configuration and use service names