Multi-Container Applications with Docker Compose

Introduction

Modern applications are rarely implemented as a single process. A typical web system may contain a user interface, an application programming interface (API), a database, a cache, a message broker, and background workers. Running each component in an independently managed container improves isolation and repeatability, but manually creating and connecting many containers quickly becomes inefficient.

Docker Compose addresses this problem by allowing developers and operators to describe a multi-container application in a declarative YAML file. The file specifies services, images or build instructions, networks, storage, environment variables, health checks, dependencies, secrets, and optional service profiles. Compose can then create, start, stop, inspect, and remove the application as a coherent unit.

Compose is particularly valuable during development, testing, demonstrations, and small-scale operational environments. It is not automatically a replacement for an orchestrator such as Kubernetes. Instead, it provides a practical model for defining application topology and teaches principles that also apply to larger orchestration systems.

Learning Objectives

By the end of this chapter, you should be able to:

• explain the purpose and architecture of Docker Compose;
• design a Compose file containing multiple services;
• distinguish service, network, volume, configuration, and secret objects;
• configure service discovery and inter-service communication;
• use dependency conditions and health checks appropriately;
• apply profiles and environment-specific override files;
• manage configuration without exposing sensitive values;
• diagnose common Compose failures; and
• evaluate the operational limitations and governance implications of Compose.

1. The Compose Mental Model

A Compose application is a project consisting of named resources. The central resource is a service: a definition of a container workload that may be created one or more times. A service definition normally specifies an image, or a build context from which an image is produced. It may also specify ports, environment variables, mounts, networks, commands, resource limits, and health checks.

Compose does not normally require users to assign fixed container IP addresses. Instead, services attached to the same user-defined network can reach one another through DNS using the service name. If a service is named db, another service can usually connect to it using hostname db and the database's internal container port. This is different from publishing a port to the host.

For example, an API might connect to db:5432, while a developer's laptop connects to the database through a published mapping such as 15432:5432. The first port is internal to the Compose network; the second exposes the container port through the host.

A Compose file is declarative. It states the desired configuration rather than providing a sequence of imperative commands. Compose calculates the resources required to approach that desired state. The exact implementation and supported attributes depend on the installed Docker Compose version, so teams should validate files with the Compose implementation used by their organization.

Important terminology

2. A Basic Compose File

The following illustrative file defines an API and a PostgreSQL database. The version top-level field is intentionally omitted; current Compose specifications use the Compose Specification rather than requiring a legacy version declaration.

services:
  api:
    build: ./api
    environment:
      DATABASE_URL: postgres://app:app_password@db:5432/appdb
    ports:
      - "8080:8080"
    depends_on:
      db:
        condition: service_healthy
    networks:
      - backend

  db:
    image: postgres:16
    environment:
      POSTGRES_DB: appdb
      POSTGRES_USER: app
      POSTGRES_PASSWORD: app_password
    volumes:
      - db-data:/var/lib/postgresql/data
    networks:
      - backend

networks:
  backend:

volumes:
  db-data:

The api service is built from a local directory and publishes host port 8080 to container port 8080. The db service uses a named volume so database files remain available when the database container is recreated. Both services share the backend network.

A typical lifecycle is:

compose.yaml → configuration validation → network/volume creation
             → image build or pull → container creation → startup
             → health and log observation → controlled shutdown

Useful commands include:

docker compose config
docker compose up -d
docker compose ps
docker compose logs -f api
docker compose exec api sh
docker compose down
docker compose down -v

docker compose down removes project containers and networks but normally preserves named volumes. Adding -v removes named volumes declared for the project and can destroy persistent data. It should therefore be used deliberately.

3. Service Design and Communication

Images and builds

A service may use image:, build:, or both. image: identifies an image to pull or use locally. build: identifies instructions for constructing an image. When both are present, the built image can be tagged with the specified name.

Production-oriented files should use explicit, preferably immutable image references. A tag such as api:1.4.2 is more reproducible than latest, although a digest reference provides stronger immutability. Images should be small, regularly patched, and built from trusted sources.

Ports

The syntax HOST:CONTAINER publishes a container port on the host. Publishing is needed for traffic entering from outside the Compose network, such as a browser accessing an API. It is not needed for service-to-service communication on a shared network.

Publishing every database, cache, and internal administration port increases attack surface. A sound default is to publish only the services that external users require.

Networks

Networks can express architectural boundaries. For example, a reverse proxy may join a public-facing network and an application network, while a database joins only the application network.

services:
  proxy:
    image: nginx:1.27
    networks: [edge, app]
  api:
    image: example/api:1.4.2
    networks: [app, data]
  db:
    image: postgres:16
    networks: [data]

networks:
  edge:
  app:
  data:

In this design, the proxy can reach the API, and the API can reach the database, but the database is not directly attached to the edge network. Network separation is useful, but it is not a complete security boundary; host security, image security, credentials, and application authorization remain necessary.

Service discovery

Compose provides DNS-based discovery for services on a common network. Applications should use stable service names rather than container IP addresses, because container IP addresses may change when containers are recreated. A service may be scaled to multiple replicas, but applications must then tolerate multiple address records and should not assume that one container is permanently associated with the service name.

4. Persistence with Volumes and Mounts

Containers are replaceable. Data that must survive replacement should be stored outside the writable container layer.

A named volume is managed by Docker and is appropriate for database files and other persistent application data. A bind mount maps a host path into a container and is useful for source-code sharing during development, but it introduces host-path, permission, and portability concerns.

services:
  api:
    build: .
    volumes:
      - ./src:/app/src:ro
      - api-cache:/app/cache

volumes:
  api-cache:

The :ro suffix makes the source mount read-only from the container's perspective. Read-only mounts reduce accidental modification but do not replace proper application permissions.

Database volumes require a recovery plan. Persistence is not the same as backup. A volume can be deleted, corrupted, encrypted by ransomware, or lost with its host. Teams should define backup frequency, retention, restoration testing, and ownership of the data.

5. Configuration, Environment Variables, and Secrets

Configuration should vary independently of application code. Compose supports environment variables, .env interpolation, env_file, configs, and secrets. These mechanisms have different purposes.

• Interpolation variables help construct a Compose configuration.
• Environment variables inside a container provide runtime settings.
• Configs provide non-sensitive files such as application configuration.
• Secrets provide sensitive values such as passwords, tokens, or certificates.

A common error is treating .env as a secure secret store. Values in .env may be visible to users, process tooling, logs, source-control systems, or diagnostic output. They should not be committed when they contain credentials.

A Compose secret can be declared as a file-backed secret:

services:
  db:
    image: postgres:16
    environment:
      POSTGRES_DB: appdb
      POSTGRES_USER: app
      POSTGRES_PASSWORD_FILE: /run/secrets/db_password
    secrets:
      - db_password

secrets:
  db_password:
    file: ./secrets/db_password.txt

The application image must support the relevant _FILE convention, or the application must read the mounted secret file itself. A secret is generally mounted under /run/secrets/secret_name. Secret handling in local Compose is not identical to a production secrets manager; file permissions, host access, backups, and CI logs still require governance.

Do not place credentials directly in a public Compose file. Also avoid printing environment variables in logs or embedding secrets in images, Dockerfiles, shell history, or diagnostic bundles.

6. Startup Order, Readiness, and Health Checks

Container creation order is not equivalent to application readiness. A database process may have started while still performing recovery or initialization. An API that immediately connects may fail even though its database container is running.

A health check tests a meaningful condition:

services:
  db:
    image: postgres:16
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app -d appdb"]
      interval: 5s
      timeout: 3s
      retries: 10
      start_period: 10s

interval specifies the time between checks, timeout limits each check, retries controls consecutive failures before an unhealthy status, and start_period allows initialization time. The command must test the service's actual usability, not merely whether a process exists.

A dependent service can use a health condition:

services:
  api:
    image: example/api:1.4.2
    depends_on:
      db:
        condition: service_healthy

depends_on expresses startup and shutdown relationships; it does not make an application resilient. A database can become unavailable after the API starts. The API should implement connection timeouts, retry with bounded backoff, reconnection, and graceful failure. It should also avoid retry storms.

A health check should be inexpensive, deterministic, and free of destructive side effects. It should distinguish liveness—whether a process is running—from readiness—whether it can serve useful requests. A service that is alive but disconnected from its database may be unsuitable for traffic.

7. Profiles and Environment-Specific Composition

Profiles allow optional services to be activated only when needed. They are useful for debugging tools, test runners, local administration interfaces, or observability components.

services:
  api:
    image: example/api:1.4.2

  mailpit:
    image: axllent/mailpit:latest
    profiles: [debug]
    ports:
      - "8025:8025"

The default application does not include mailpit; it can be enabled with a profile command such as docker compose --profile debug up.

Environment-specific configuration is often handled with a base file and an override file:

# compose.yaml
services:
  api:
    image: example/api:1.4.2
    environment:
      LOG_LEVEL: info

# compose.dev.yaml
services:
  api:
    build: ./api
    environment:
      LOG_LEVEL: debug
    volumes:
      - ./api:/app
    ports:
      - "8080:8080"

The files can be selected explicitly:

docker compose -f compose.yaml -f compose.dev.yaml up

Compose merges configuration according to attribute-specific rules. Lists, mappings, and scalar values do not all merge identically, and a later file may replace rather than append a value. Teams should inspect the effective result with docker compose config rather than relying on assumptions.

A good structure keeps common service identity in the base file and places development-only mounts, debugging tools, and local ports in the development file. Production deployment should use pinned images, external secret mechanisms, restricted ports, resource controls, and a separately reviewed configuration.

8. A Complete Illustrative Application

The following example models a small task application with an API, PostgreSQL database, and optional database administration tool.

services:
  api:
    build: ./api
    environment:
      DB_HOST: db
      DB_NAME: tasks
      DB_USER: tasks_app
      DB_PASSWORD_FILE: /run/secrets/db_password
    ports:
      - "8080:8080"
    depends_on:
      db:
        condition: service_healthy
    secrets:
      - db_password
    networks:
      - frontend
      - backend

  db:
    image: postgres:16
    environment:
      POSTGRES_DB: tasks
      POSTGRES_USER: tasks_app
      POSTGRES_PASSWORD_FILE: /run/secrets/db_password
    volumes:
      - postgres-data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U tasks_app -d tasks"]
      interval: 5s
      timeout: 3s
      retries: 12
    secrets:
      - db_password
    networks:
      - backend

  adminer:
    image: adminer:4
    profiles: [admin]
    ports:
      - "8081:8080"
    networks:
      - backend

volumes:
  postgres-data:

networks:
  frontend:
  backend:

secrets:
  db_password:
    file: ./secrets/db_password.txt

The API and database communicate through backend. The database is not exposed directly to the host. Adminer is optional and attached only to the backend network. The API waits for the database health condition during startup, but its code must still handle later database failures.

Before running this file, an operator should inspect it, create the secret file with restrictive permissions, verify that the API understands DB_PASSWORD_FILE, and confirm that the selected image versions are approved.

9. Applied Case: A Local Order-Processing Platform

Background

This illustrative case concerns a small retail team developing an order-processing platform. The system includes a web API, PostgreSQL, Redis for short-lived caching, and a worker that sends confirmation emails. Developers previously installed each dependency directly on their laptops, producing inconsistent versions and difficult onboarding.

Problem

The team needs one repeatable command to start the development system. It also needs an optional mail-capture tool, persistent local database data, and a reliable way to prevent the worker from starting before the database is usable.

Analysis

The API and worker require database access. Redis is internal and need not be published. The mail-capture tool is useful only during development and should not start by default. The database requires a named volume. A database process check is insufficient, so a readiness-oriented health check is appropriate.

Approach

The team creates services named api, worker, db, redis, and mail. The mail service receives the debug profile. The API and worker share an application network with the database and Redis. A separate edge network is used if a reverse proxy is later introduced. Database credentials are supplied through a secret file in local development and through a managed secret mechanism in the deployment environment.

The worker uses a restart-safe job design: processing a message twice does not create two shipments or two charges. This is important because Compose restart behavior cannot guarantee exactly-once execution.

Outcome

A new developer installs Docker, clones the repository, creates a local secret file, and runs the documented Compose command. The team gains consistent service names, repeatable dependency setup, isolated networks, and persistent database state. The mail tool can be enabled only when needed. CI can use a separate override file to run tests against the same service topology.

Learning points

1. Declarative topology reduces environmental drift.
2. Health checks improve startup coordination but do not replace application resilience.
3. Optional tools should not unnecessarily increase the default attack surface.
4. Volumes preserve data but do not provide backups.
5. Secrets require protection beyond merely moving values out of YAML.
6. Service names are more reliable than container IP addresses.

10. Troubleshooting and Operational Practice

A disciplined diagnosis begins with the effective configuration:

docker compose config
docker compose ps
docker compose logs --tail=200 api db
docker compose events

Common symptoms and causes include:

Use project names deliberately when running multiple environments. Accidental reuse of a project name can cause operators to inspect or stop the wrong containers. In CI, isolate projects and clean up resources after tests while preserving artifacts needed for diagnosis.

11. Limitations, Security, and Governance

Compose is powerful but has boundaries. It does not automatically provide multi-host scheduling, sophisticated rolling deployment, self-healing across machines, centralized policy enforcement, or a full production secrets-management system. It can be used in production for suitable single-host workloads, but that decision requires explicit operational justification.

Security practices include:

• pin image versions and scan images for vulnerabilities;
• use least-privilege users inside containers;
• avoid privileged mode unless specifically justified;
• publish only required ports;
• use read-only filesystems or mounts where practical;
• separate development and production credentials;
• protect the Docker socket, which provides highly privileged control;
• review bind mounts because they can expose host files; and
• document data ownership, retention, backup, and restoration responsibilities.

There are also governance concerns. Logs may contain personal data, tokens, or customer information. Database volumes may contain regulated records. Teams should apply appropriate access control, retention, encryption, and incident-response procedures. A convenient local Compose file must not become an uncontrolled path for copying production data to developer machines.

12. Contemporary Practice

Current Compose workflows increasingly integrate image signing, software bills of materials, vulnerability scanning, CI validation, and developer environments. Compose can also serve as a local representation of a service architecture that is later deployed through another platform. However, translation is not automatic: host networking, storage semantics, secret handling, resource constraints, and health behavior may differ between platforms.

AI-assisted tools can help generate a first draft of a Compose file or explain an error, but generated configuration must be reviewed. An AI system may invent unsupported attributes, expose credentials, choose insecure images, or misunderstand merge semantics. Professional responsibility remains with the engineer who approves and operates the configuration.

Best-Practice Checklist

Before sharing or deploying a Compose application, verify that:

• service names describe stable application roles;
• internal traffic uses service names and container ports;
• only necessary ports are published;
• persistent data uses deliberate volumes and has a backup plan;
• health checks test meaningful readiness;
• application code handles dependency failure after startup;
• secrets are not committed or printed;
• images are trusted, patched, and version-pinned;
• profiles keep optional tools disabled by default;
• override behavior has been rendered and reviewed;
• resource, logging, and restart policies are appropriate; and
• the operational boundary of Compose is understood.

Chapter Summary

Docker Compose provides a declarative way to model and operate multi-container applications. Services describe workloads; networks provide service discovery and communication boundaries; volumes preserve data beyond container lifetimes; configs and environment variables supply non-sensitive configuration; and secrets protect sensitive values when used with appropriate host and platform controls. Health checks and conditional dependencies improve startup coordination, but they do not replace retries, timeouts, and graceful application behavior. Profiles enable optional services, while base and override files support environment-specific configuration. Effective operation requires rendered-configuration inspection, careful troubleshooting, image and credential governance, backup planning, and a clear understanding that Compose is not automatically a multi-host orchestrator.

Key Terms

Service: A declarative definition of a container workload.

Compose project: The named collection of services and resources managed together.

Compose Specification: The current specification describing Compose application configuration.

Image: A packaged filesystem and metadata used to create containers.

Container: A running instance of an image.

Network: A logical communication boundary connecting containers.

Service discovery: Finding a service through a stable name rather than a changing IP address.

Published port: A host-to-container port mapping that exposes a service externally.

Named volume: Docker-managed persistent storage.

Bind mount: A host path mounted inside a container.

Health check: A test that reports whether a service is functioning or ready.

Readiness: The condition that a service can accept useful work.

Dependency condition: A rule controlling when a dependent service may start.

Profile: An optional group of services activated explicitly.

Override file: A Compose file used with another file to modify configuration.

Config: Non-sensitive data supplied to a service.

Secret: Sensitive data supplied with restricted exposure.

Service resilience: The ability to tolerate dependency failure, restart, or delay.

Image pinning: Selecting a specific image version or digest for reproducibility.

Declarative configuration: Describing desired state rather than issuing each operational step.

Review Questions

Short Answer Questions

1. What is the difference between a service name and a container IP address in Compose networking?
2. Why should a database normally use a named volume?
3. What problem does a health check solve?
4. What is the purpose of a Compose profile?
5. Why is a .env file not automatically a secure secrets store?

Long Answer Questions

1. Explain how services, networks, volumes, and secrets work together in a multi-container application.
2. Compare host port publishing with internal service-to-service communication.
3. Discuss why depends_on cannot by itself make an application resilient.
4. Describe a safe strategy for maintaining development and production Compose configurations.
5. Evaluate the operational and security limitations of using Compose for production workloads.

Analytical / Application Questions

1. An API starts successfully but fails whenever its database is restarted. Design changes at both the Compose and application levels to improve resilience.
2. A team publishes ports for its API, database, Redis instance, and administration tool. Analyze the security implications and propose a smaller exposure model.
3. A development override unexpectedly removes a production setting. Explain how you would investigate the merge behavior and validate the effective configuration.

Multiple Choice Questions with Answers

Question 1

Which hostname should an API normally use to reach a Compose service named db on the same network?

A. localhost

B. db

C. 127.0.0.1

D. the host machine's IP address

Correct Answer: B. db

Question 2

What is the primary purpose of a named volume for a database service?

A. To publish the database to the internet

B. To assign the database a fixed IP address

C. To preserve data beyond the life of a container

D. To replace database backups

Correct Answer: C. To preserve data beyond the life of a container

Question 3

What does the mapping 8080:80 mean in a Compose port declaration?

A. Container port 8080 is mapped to host port 80

B. Host port 8080 is mapped to container port 80

C. Two containers share port 8080

D. The service listens only on port 8080 internally

Correct Answer: B. Host port 8080 is mapped to container port 80

Question 4

Which condition delays a dependent service until the dependency reports a successful health check?

A. service_started

B. service_completed_successfully

C. service_healthy

D. container_ready

Correct Answer: C. service_healthy

Question 5

Which Compose feature is most appropriate for enabling an optional database administration tool only when requested?

A. A profile

B. A bind mount

C. A health check

D. A published volume

Correct Answer: A. A profile

Question 6

Why should an application still implement retries even when depends_on uses service_healthy?

A. Health checks permanently reserve network ports

B. Dependencies can fail after the initial startup sequence

C. Retries are required to create named volumes

D. Compose ignores all health checks

Correct Answer: B. Dependencies can fail after the initial startup sequence

Question 7

Which command is most useful for examining the effective result of merged Compose files?

A. docker compose config

B. docker compose pull --all

C. docker compose pause

D. docker image prune

Correct Answer: A. docker compose config

Question 8

Which practice best reduces unnecessary external exposure?

A. Publish every service port on the host

B. Use container IP addresses in application code

C. Publish only ports required by external clients

D. Place all services on separate networks with no shared connections

Correct Answer: C. Publish only ports required by external clients

Question 9

What is the main distinction between a config and a secret?

A. A config is always binary, while a secret is always text

B. A config is intended for non-sensitive data, while a secret is intended for sensitive data

C. A secret can be used only by databases

D. A config automatically encrypts all values

Correct Answer: B. A config is intended for non-sensitive data, while a secret is intended for sensitive data

Question 10

Which statement about volumes is correct?

A. A volume is automatically a tested backup system

B. A volume guarantees data replication across hosts

C. A volume can preserve data across container recreation but still requires backup planning

D. A volume can be mounted only by one container

Correct Answer: C. A volume can preserve data across container recreation but still requires backup planning