Multi-Container Applications with Docker Compose

Introduction

Modern applications are rarely delivered as a single process. A web system may contain a user interface, an API, a database, a cache, a message broker, and background workers. Running each component in a separate container improves isolation, portability, scalability, and operational control. However, manually creating and connecting several containers is inefficient and error-prone.

Docker Compose provides a declarative way to model and operate multi-container applications. A Compose file describes services, images, commands, networks, volumes, environment variables, health checks, and operational policies. The same model can be used by developers on laptops, automated tests in CI pipelines, and operations teams during deployment, although production environments may require additional orchestration platforms.

Compose is therefore best understood as an application model rather than merely a command shortcut. It expresses the desired relationships among services and allows Docker to create the required resources consistently.

Learning Objectives

By the end of this chapter, the learner should be able to:

• explain the purpose and structure of a Compose application;
• write and validate a Compose YAML file;
• configure service dependencies and distinguish startup order from readiness;
• use profiles to represent optional application components;
• manage configuration and secrets through environment mechanisms;
• attach persistent volumes and isolated networks;
• implement health checks and interpret their results;
• operate, troubleshoot, and secure a multi-container application; and
• evaluate the limitations of Compose in production environments.

1. The Compose Application Model

A Compose application consists primarily of services. A service is a definition from which one or more containers can be created. For example, an application might define frontend, api, database, and redis services.

A service definition normally specifies:

• the container image or a build context;
• environment variables;
• exposed or published ports;
• mounted volumes;
• attached networks;
• startup commands;
• resource and restart policies; and
• dependency and health-check conditions.

Compose creates a project containing the declared containers, networks, and volumes. The project name normally comes from the directory name, but it can be explicitly selected. Project naming is important because resource names are commonly prefixed with the project name, helping prevent collisions between separate applications.

A minimal file is shown below:

services:
  web:
    image: nginx:1.27-alpine
    ports:
      - "8080:80"

The web service uses an NGINX image and maps port 8080 on the host to port 80 inside the container. The notation HOST:CONTAINER applies to port mappings. Port publishing is needed when clients outside the Docker network must reach the service. It is not needed for service-to-service communication on a shared Compose network.

Compose Specification and YAML

Current Compose files generally use the Compose Specification. A top-level version field is no longer required by modern Docker Compose implementations and may produce a warning. YAML indentation is significant, and strings containing special characters should usually be quoted.

A useful complete skeleton is:

services:
  api:
    build:
      context: ./api
    environment:
      DATABASE_URL: postgresql://app:change-me@db:5432/appdb
    depends_on:
      db:
        condition: service_healthy
    networks:
      - backend
      - frontend

  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_DB: appdb
      POSTGRES_USER: app
      POSTGRES_PASSWORD: change-me
    volumes:
      - db-data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app -d appdb"]
      interval: 10s
      timeout: 5s
      retries: 5
    networks:
      - backend

  proxy:
    image: nginx:1.27-alpine
    ports:
      - "8080:80"
    depends_on:
      - api
    networks:
      - frontend

volumes:
  db-data:

networks:
  frontend:
  backend:

The example separates the public-facing network from the database network. The proxy and API share frontend; the API and database share backend. Consequently, the database is not directly attached to the public-facing network.

2. Service Discovery, Networks, and Ports

Compose creates a default network when no explicit network is supplied. Services on the same network can normally reach one another using the service name as a DNS hostname. In the example, the API should connect to the database host db, not localhost.

This distinction is fundamental:

• localhost inside a container means that same container;
• the service name, such as db, identifies another container on the Compose network;
• a published host port is mainly for access from the host or external clients.

For example, if PostgreSQL listens on port 5432 inside the database container, the API connection string uses db:5432. Publishing 5432:5432 is unnecessary unless a host-based database client must connect directly.

Network design principles

A network should represent a communication boundary, not merely a convenient list. Common practices include:

1. Put public gateways on a frontend network.
2. Put application services and databases on a backend network.
3. Attach a service to multiple networks only when it must bridge those boundaries.
4. Avoid publishing database and cache ports to the host unless required.
5. Use explicit network names only when integration with external systems demands them.

A service can join a pre-existing network using external: true, but the network must already exist. This is useful when multiple Compose projects must communicate, but it creates coupling and should be documented carefully.

3. Building Images and Configuring Commands

Compose can run a prebuilt image or build an image from a Dockerfile. The following service builds an application from a local directory:

services:
  api:
    build:
      context: ./api
      dockerfile: Dockerfile
    image: example-api:dev
    command: ["python", "-m", "app"]

context identifies the files sent to the Docker build process. A carefully designed .dockerignore prevents unnecessary source files, credentials, and large artifacts from entering the build context.

command overrides the image’s default command, while entrypoint overrides its entrypoint. These fields should be used deliberately because replacing an entrypoint may remove initialization behavior supplied by the image author.

Compose does not automatically make a container a good service. A process should normally run in the foreground, write useful logs to standard output and error, respond correctly to termination signals, and return a non-zero exit status when it fails.

4. Dependencies and Readiness

A common beginner design is:

depends_on:
  - db

This expresses a dependency and influences startup and shutdown order. It does not, by itself, prove that the database is ready to accept connections. A database process may have started while still performing initialization, applying migrations, or restoring data.

A stronger design uses a health check:

services:
  api:
    depends_on:
      db:
        condition: service_healthy

  db:
    image: postgres:16-alpine
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app -d appdb"]
      interval: 5s
      timeout: 3s
      retries: 10
      start_period: 10s

The health check runs inside the database container. Its command must exist in the image and should test the actual capability needed by the dependent service. A process check such as “is a PostgreSQL process present?” is weaker than a connection check.

Health checks and application resilience

Health checks improve orchestration decisions, but they do not eliminate the need for resilient applications. An API should still handle connection failures, retry transient operations with bounded backoff, and fail clearly when a dependency is unavailable. A successful health check is a point-in-time observation, not a permanent guarantee.

Health status commonly includes starting, healthy, and unhealthy. The interval controls the time between checks, timeout limits an individual check, retries determines how many consecutive failures are tolerated, and start_period provides an initialization grace period. These values are operational parameters rather than universal constants; they should reflect the service’s actual startup and response characteristics.

5. Profiles for Optional Services

Profiles allow one Compose model to describe optional components. For example, development may include a mail-testing tool and an administration interface, while the normal application excludes them.

services:
  api:
    image: example-api:dev

  mailpit:
    image: axllent/mailpit:latest
    profiles: ["tools"]
    ports:
      - "8025:8025"

Services without a profiles attribute are enabled by default. A service assigned to a profile is enabled when that profile is selected, for example with --profile tools, or through the COMPOSE_PROFILES environment variable.

Profiles are useful for:

• local debugging tools;
• performance-test dependencies;
• administrative interfaces;
• optional monitoring components; and
• integration-test services.

Profiles are not a substitute for security. A service hidden by a profile may still expose sensitive functionality when enabled, so its image, credentials, network access, and ports must be controlled.

6. Environment Management and Secrets

Configuration should be separated from the image. Compose supports environment variables through several mechanisms, including an environment mapping, an env_file, shell interpolation, and a project .env file.

Example:

services:
  api:
    image: example-api:dev
    env_file:
      - ./config/api.env
    environment:
      LOG_LEVEL: ${LOG_LEVEL:-info}
      DATABASE_HOST: db

The expression ${LOG_LEVEL:-info} means that info is used when LOG_LEVEL is unset or empty during interpolation. Interpolation occurs when Compose processes the file; it is distinct from variables that are later injected into the container.

A practical configuration classification is:

Do not commit passwords, private keys, cloud tokens, or production credentials to a repository. An .env file used locally should normally be excluded from version control, while a safe .env.example documents required variable names without real secrets.

Compose secrets can mount sensitive values as files, depending on the target environment and image behavior:

services:
  api:
    secrets:
      - db_password

secrets:
  db_password:
    file: ./secrets/db_password.txt

The application must be designed to read the mounted secret. Environment variables are convenient but can be exposed through diagnostics, process inspection, or accidental logging. Neither environment variables nor local secret files alone constitute a complete enterprise secret-management strategy.

7. Volumes and Data Persistence

A container’s writable layer is disposable. If a database stores data only there, removing the container can destroy the data. Named volumes provide Docker-managed persistent storage:

volumes:
  db-data:

services:
  db:
    image: postgres:16-alpine
    volumes:
      - db-data:/var/lib/postgresql/data

A named volume is different from a bind mount. A named volume is managed by Docker and is generally convenient for service data. A bind mount maps a host path and is useful for source-code sharing during development, but it introduces host-specific permissions and path behavior.

Volume persistence does not automatically provide backup, replication, encryption, or point-in-time recovery. A production data policy must specify backup frequency, retention, restoration testing, access controls, and recovery objectives.

The two principal recovery measures are often described as:

• Recovery Point Objective (RPO): the maximum acceptable amount of data loss measured in time;
• Recovery Time Objective (RTO): the maximum acceptable time to restore service.

A local named volume may be adequate for a development database but is rarely sufficient as the only production data-protection mechanism.

8. Operating a Compose Application

Common commands include:

docker compose config
docker compose up -d
docker compose ps
docker compose logs -f api
docker compose exec api sh
docker compose restart api
docker compose down
docker compose down -v

docker compose config renders and validates the effective configuration and is particularly useful for detecting interpolation errors. up -d starts the application in detached mode. ps displays service status, while logs supports diagnosis. exec opens a process inside a running container.

The command down removes containers and project networks. Adding -v also removes named volumes declared by the project, which can destroy persistent development data. It should never be used casually when a database volume matters.

Image rebuilding may be required after a Dockerfile or dependency change:

docker compose up -d --build

For repeatability, teams should pin image versions rather than relying on mutable tags such as latest. Digest pinning provides stronger immutability when the operational process supports it.

9. Worked Application: A Small Web Platform

Consider a training platform with an API, PostgreSQL database, and Redis cache. The API is reachable through host port 8000, while the database and cache remain internal.

services:
  api:
    build: ./api
    ports:
      - "8000:8000"
    environment:
      DATABASE_URL: postgresql://training:devpass@db:5432/training
      REDIS_URL: redis://cache:6379/0
    depends_on:
      db:
        condition: service_healthy
      cache:
        condition: service_healthy
    networks:
      - appnet

  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_DB: training
      POSTGRES_USER: training
      POSTGRES_PASSWORD: devpass
    volumes:
      - training-db:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U training -d training"]
      interval: 5s
      timeout: 3s
      retries: 8
    networks:
      - appnet

  cache:
    image: redis:7-alpine
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 3s
      retries: 8
    networks:
      - appnet

volumes:
  training-db:

networks:
  appnet:

The API uses service names, db and cache, for internal DNS resolution. Only the API port is published. The database has durable storage, while Redis is treated as a rebuildable cache. Health checks express readiness conditions, but the API should still handle temporary failures.

A developer can validate the model with docker compose config, start it with docker compose up -d --build, inspect readiness through docker compose ps, and review failures with docker compose logs api db cache.

10. Illustrative Case Study: Retail Order Service

Background

A small retailer operates an order application consisting of a web service, an API, PostgreSQL, and a worker that sends confirmation emails. Developers previously started containers manually, using inconsistent names and ports.

Problem

The team experienced three recurring failures: the API started before PostgreSQL was ready, local data disappeared when containers were recreated, and developers accidentally exposed the database port on shared networks.

Analysis

The application had three distinct concerns: dependency readiness, state persistence, and network exposure. Manual commands did not document these relationships. In addition, credentials were stored in shell history and committed configuration files.

Approach and solution

The team created a Compose project with:

• a health check on PostgreSQL;
• a conditional dependency from API and worker to the healthy database;
• a named database volume;
• a private backend network;
• a separate frontend network for the web gateway;
• an optional tools profile for a local mail-testing service; and
• an untracked environment file with a documented example template.

The worker was designed to retry database connections and to process messages idempotently, so that a repeated delivery would not create duplicate confirmations.

Outcome

New developers started the complete stack with one command and could inspect the effective model consistently. Database data survived ordinary container recreation, and the database was no longer published to the host. The team also added a backup script and documented that local volume persistence was not a production backup solution.

Learning points

Compose improved reproducibility because relationships were explicit. It did not, by itself, solve application retries, secret governance, backup, or high availability. Those requirements remained responsibilities of application and operational design.

11. Troubleshooting and Professional Practice

A systematic diagnostic sequence is more effective than repeatedly restarting containers.

1. Validate the rendered model with docker compose config.
2. Check service state and health with docker compose ps.
3. Inspect logs for the failing service and its dependencies.
4. Confirm that the application uses service names rather than inappropriate localhost addresses.
5. Check network membership and published ports.
6. Verify environment variables without printing secrets.
7. Inspect volume mounts and file permissions.
8. Test the dependency from inside the relevant container.
9. Confirm image architecture and version compatibility.
10. Recreate only the affected service when possible.

Typical failures include a misspelled service hostname, an incorrect container path, a health-check command absent from the image, a password mismatch caused by an old initialized volume, and a port collision on the host.

Security and governance

Compose files should be reviewed as infrastructure code. Important controls include least-privilege credentials, minimal published ports, non-root containers where supported, trusted and scanned images, pinned versions, restricted file permissions, and secret scanning in repositories and CI pipelines.

Do not assume that a private Docker network is an authorization system. Any compromised container attached to that network may attempt to contact other services. Application authentication, database authorization, TLS requirements, and network segmentation remain necessary.

Compose in relation to production orchestration

Compose is highly effective for development, demonstrations, automated integration tests, and small controlled deployments. It is not a complete replacement for platforms that provide multi-node scheduling, service autoscaling, rolling deployment strategies, advanced policy enforcement, and built-in high availability. Kubernetes, managed container services, and other orchestrators may be more appropriate for those requirements.

A Compose file can still serve as a valuable local development contract even when production uses another platform. Teams should avoid assuming that every Compose feature maps directly to a production orchestrator; translation and operational review are required.

12. Contemporary Developments

Current practice increasingly treats Compose files as versioned, testable infrastructure artifacts. Improvements include multi-stage image builds, supply-chain scanning, software bills of materials, automated configuration validation, and CI pipelines that build and test the complete service graph.

AI-assisted tools can help generate an initial Compose file, explain an error, or suggest health checks. However, generated configuration must be reviewed for image trust, secret exposure, incorrect dependency assumptions, excessive privileges, and destructive commands such as volume removal. Human technical judgment remains essential because the tool may not understand the application’s data, compliance, or recovery requirements.

Best-Practice Checklist

• Use a clear service-oriented structure.
• Prefer immutable, reviewed image versions.
• Use service names for internal communication.
• Publish only ports that external clients require.
• Separate frontend and backend networks where appropriate.
• Add health checks that test meaningful readiness.
• Treat depends_on as ordering plus optional health conditions, not complete resilience.
• Store persistent state in explicitly managed volumes.
• Back up important data independently of local volumes.
• Keep secrets out of source control and logs.
• Use profiles for optional tooling, not for access control.
• Validate the rendered configuration before deployment.
• Document destructive commands and recovery procedures.

Chapter Summary

Docker Compose provides a declarative model for multi-container applications. Services define images, builds, commands, configuration, networks, volumes, and operational behavior. Compose service names provide internal DNS, while published ports serve host or external clients. Networks can be designed to isolate public gateways from databases and caches.

Dependency declarations influence startup order, but readiness requires meaningful health checks and resilient application behavior. Profiles support optional services, while environment mechanisms separate configuration from images. Sensitive values require appropriate secret-management controls. Named volumes preserve data across ordinary container replacement, but they do not replace backup and recovery systems.

Compose is valuable for development, testing, and controlled deployments. Its effectiveness depends on disciplined image management, secure configuration, explicit networking, health-aware dependencies, and systematic operations. It should be viewed as an application model whose limitations must be considered when moving to highly available production orchestration.

Key Terms

Service: A declarative definition from which one or more containers are created.

Compose project: The grouped application resources managed by Compose.

Compose Specification: The current implementation-oriented specification for defining Compose applications.

Service discovery: The process by which services locate one another, commonly through Compose-provided DNS names.

Port publishing: Mapping a host port to a container port for external access.

Network: An isolated communication space connecting containers.

Dependency: A relationship indicating that one service relies on another.

Readiness: The condition that a service can perform the function required by dependents.

Health check: A command and policy used to assess a container’s operational condition.

Profile: A named mechanism for selectively enabling optional services.

Environment variable: A named configuration value supplied to a process.

Interpolation: Substitution of variable values while Compose processes a file.

Secret: Sensitive configuration such as a password, token, or private key.

Named volume: Docker-managed persistent storage identified by a name.

Bind mount: A mapping of a host filesystem path into a container.

Image: A packaged filesystem and metadata used to create containers.

Idempotence: The property that repeating an operation produces the same intended result without harmful duplication.

RPO: Recovery Point Objective, the maximum acceptable data-loss interval.

RTO: Recovery Time Objective, the maximum acceptable service-restoration interval.

Orchestrator: A platform that schedules and manages containers across infrastructure.

Review Questions

Short Answer Questions

1. What is the difference between a Compose service and a container?
2. Why should a containerized API usually use db rather than localhost to reach PostgreSQL?
3. What problem does a named volume solve?
4. Why is depends_on alone insufficient to prove database readiness?
5. What is the purpose of a Compose profile?

Long Answer Questions

1. Explain how Compose networks, published ports, and service-name DNS work together in a multi-container application.
2. Describe a secure and maintainable approach to environment variables, .env files, and secrets.
3. Compare named volumes and bind mounts, including appropriate development and production uses.
4. Explain how health checks should be designed and why application-level retries remain necessary.
5. Discuss the strengths and limitations of Docker Compose compared with a production orchestrator.

Analytical / Application Questions

1. An API starts successfully but repeatedly fails to connect to its database during the first 20 seconds. Diagnose the likely Compose-level causes and propose a robust configuration and application response.
2. Design network, volume, profile, and port settings for a web gateway, API, database, cache, and optional administration tool. Justify which services should be externally reachable.
3. A team uses docker compose down -v during troubleshooting and loses its local database. Analyze the operational failure and propose safeguards for development, backup, and production recovery.

Multiple Choice Questions with Answers

Question 1

Which hostname should an API normally use to connect to a PostgreSQL service named db in the same Compose network?

A. localhost

B. db

C. 127.0.0.1

D. host.docker.internal

Correct Answer: B. db

Question 2

What does publishing 8080:80 mean in a Compose service?

A. Port 80 on the host maps to port 8080 in the container

B. Port 8080 on the host maps to port 80 in the container

C. Both ports are opened only inside the container

D. The container is attached to network 8080

Correct Answer: B. Port 8080 on the host maps to port 80 in the container

Question 3

What is the main limitation of a basic depends_on declaration?

A. It removes the dependent service after startup

B. It prevents services from sharing networks

C. It does not by itself prove that the dependency is ready

D. It disables health checks

Correct Answer: C. It does not by itself prove that the dependency is ready

Question 4

Which Compose feature selectively enables optional services such as development tools?

A. Profiles

B. Volumes

C. Entrypoints

D. Port mappings

Correct Answer: A. Profiles

Question 5

What is the primary purpose of a named volume for a database?

A. To publish the database to the internet

B. To preserve database files beyond ordinary container replacement

C. To replace database authentication

D. To make the database stateless

Correct Answer: B. To preserve database files beyond ordinary container replacement

Question 6

Which service is generally least likely to require a published host port in a typical application?

A. A public web gateway

B. An API accessed by the gateway

C. A database used only by the API

D. A browser-facing frontend

Correct Answer: C. A database used only by the API

Question 7

What should a meaningful database health check primarily test?

A. Whether a database-related process name exists

B. Whether the database can accept the required connection

C. Whether the host has an internet connection

D. Whether the container has a writable temporary directory

Correct Answer: B. Whether the database can accept the required connection

Question 8

Which command is most useful for viewing the rendered and interpolated Compose configuration?

A. docker compose config

B. docker compose rm

C. docker compose pause

D. docker compose version

Correct Answer: A. docker compose config

Question 9

Why should production credentials generally not be committed to a Compose repository?

A. YAML cannot contain passwords

B. Credentials in source control can be exposed and are difficult to govern safely

C. Compose automatically deletes committed credentials

D. Environment variables cannot contain secrets

Correct Answer: B. Credentials in source control can be exposed and are difficult to govern safely

Question 10

What can happen when docker compose down -v is used on a project with a named database volume?

A. Only logs are deleted

B. The database image is upgraded automatically

C. The declared named volume and its data may be removed

D. The network becomes public

Correct Answer: C. The declared named volume and its data may be removed