Skip to content
Toolforge

12. Compose

Compose is the part of Docker most people use every day and think about least. It deserves a chapter because it is genuinely excellent at one job and genuinely bad at another, and the boundary is sharp.

What it is

Section titled “What it is”

Compose reads a YAML file describing several containers, then makes the same Docker API calls you would have made by hand. That is all. There is no scheduler, no controller, no reconciliation loop — it is a batch of docker run commands with the boilerplate removed.

The boilerplate it removes is not trivial, though. This:

   docker network create myapp_default
   docker volume create myapp_pgdata
   docker run -d --name myapp-db-1 --network myapp_default \
     -v myapp_pgdata:/var/lib/postgresql/data \
     -e POSTGRES_PASSWORD=secret --restart unless-stopped postgres:16
   docker run -d --name myapp-api-1 --network myapp_default \
     -p 127.0.0.1:3000:3000 -e DATABASE_URL=postgres://…@db:5432/app \
     --restart unless-stopped myapp:latest

becomes this:

services:
  db:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD: secret
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: unless-stopped


  api:
    build: .
    ports:
      - "127.0.0.1:3000:3000"
    environment:
      DATABASE_URL: postgres://postgres:secret@db:5432/app
    depends_on:
      - db
    restart: unless-stopped


volumes:
  pgdata:

docker compose up -d. And crucially, that file goes in the repository, so “how do I run this project” has an answer that does not decay.

Note two things Compose did for free, both of which chapters 9 and 10 argued you should always do manually: it created a user-defined network (so db resolves by name) and it created a named volume (so the database survives). Using Compose for two containers is worth it for those alone.

What Compose creates

Section titled “What Compose creates”
   a PROJECT NAME       from the directory name, or -p, or
                        COMPOSE_PROJECT_NAME. Everything is prefixed
                        with it, which is how two checkouts of the
                        same project stay separate.


   a NETWORK            <project>_default, a user-defined bridge.
                        Every service joins it and resolves by
                        SERVICE NAME (not container name).


   NAMED VOLUMES        <project>_<volume>, unless `external: true`.


   CONTAINERS           <project>-<service>-<index>

Because the network is per project, two projects cannot see each other by default — which is right, and occasionally the thing you are fighting when a service in one Compose file needs to reach another. The fix is an external network both files join:

   docker network create shared


   # in both files:
   networks:
     shared:
       external: true

depends_on does not mean what you want

Section titled “depends_on does not mean what you want”

This is the single most common Compose mistake:

   depends_on:
     - db

This means “start db first”. It does not mean “wait until Postgres is accepting connections”. The container starts in a few milliseconds; Postgres takes several seconds to initialize. Your API starts, connects, fails, and — if you did not set a restart policy — exits.

The fix is a healthcheck plus the long form:

services:
  db:
    image: postgres:16
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 3s
      retries: 10
      start_period: 30s


  api:
    depends_on:
      db:
        condition: service_healthy
   condition: service_started        the default. Almost useless.
   condition: service_healthy        waits for the healthcheck to pass.
   condition: service_completed_successfully
                                     waits for a one-shot container
                                     (migrations, seeding) to exit 0.

start_period matters: during it, failing checks do not count against retries, so a database that takes 25 seconds to initialize is not marked unhealthy at second 10.

Even with this, the application should still handle a lost database connection and retry. Healthchecks fix startup ordering; they do nothing about the database restarting at 3 a.m. Compose ordering is a convenience, not a correctness mechanism.

The file, feature by feature

Section titled “The file, feature by feature”
services:
  api:
    build:
      context: .
      dockerfile: Dockerfile
      target: dev              ← chapter 7's --target
      args:
        NODE_VERSION: "20"
      cache_from:
        - ghcr.io/org/api:buildcache


    image: ghcr.io/org/api:dev  ← names the built image, and is what
                                  gets pulled if there is no build


    command: ["npm", "run", "dev"]    ← overrides CMD
    entrypoint: ["/entrypoint.sh"]    ← overrides ENTRYPOINT
    user: "1000:1000"
    working_dir: /app
    init: true                        ← chapter 8's tini


    environment:
      - NODE_ENV=development
      - API_KEY=${API_KEY}            ← from your shell or .env
    env_file:
      - .env.local


    ports:
      - "127.0.0.1:3000:3000"         ← bind explicitly (chapter 10)
    expose:
      - "9229"                        ← reachable inside the network only


    volumes:
      - .:/app                        ← bind mount for live editing
      - /app/node_modules             ← anonymous volume mask (chapter 9)
      - pgdata:/var/lib/postgresql/data


    deploy:
      resources:
        limits:
          cpus: "2"
          memory: 1g


    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "3"


    restart: unless-stopped

A few notes:

  • ${VAR} interpolation reads your shell environment and a .env file in the project directory. That .env is read by Compose, for substitution into the YAML; it is unrelated to env_file, which is passed into the container. Two different files doing two different things with confusingly similar names.
  • deploy.resources.limits works in plain docker compose up now (it was Swarm-only historically). The rest of deploy — replicas, placement, update_config — is Swarm.
  • version: at the top of the file is obsolete. The Compose Specification dropped it, and current versions warn about it. Delete the line.
  • docker compose (plugin, Go) replaced docker-compose (standalone, Python). If you are still typing the hyphen, you are on v1, which is end-of-life and behaves differently in several places.

Overrides and profiles

Section titled “Overrides and profiles”

Compose merges files, which is how one project serves development and production without duplication.

   docker compose up
     → reads compose.yaml, then compose.override.yaml automatically


   docker compose -f compose.yaml -f compose.prod.yaml up
     → explicit; the override file is NOT read
   # compose.yaml — shared
   services:
     api:
       image: ghcr.io/org/api:${TAG:-latest}
       environment:
         DATABASE_URL: postgres://postgres@db:5432/app


   # compose.override.yaml — development, applied automatically
   services:
     api:
       build: .
       volumes:
         - .:/app
       environment:
         NODE_ENV: development
       command: ["npm", "run", "dev"]

Merge semantics are worth knowing because they are inconsistent by design: scalars are replaced, lists are appended, maps are merged. So two files each specifying ports gives you both port mappings, not the second one — a common surprise.

Profiles keep optional services out of the default up:

services:
  mailhog:
    image: mailhog/mailhog
    profiles: ["dev-tools"]


  # docker compose up                     → no mailhog
  # docker compose --profile dev-tools up → with mailhog

Good for debug UIs, seed jobs, load generators — anything a new contributor should not have to run.

watch — the underused one

Section titled “watch — the underused one”

develop.watch rebuilds or syncs on file changes, and it is better than a bind mount for languages where a bind mount is slow or wrong:

services:
  api:
    build: .
    develop:
      watch:
        - action: sync
          path: ./src
          target: /app/src
        - action: rebuild
          path: package.json
   sync      copy changed files into the running container.
             Fast. For interpreted code with a hot reloader.
   rebuild   rebuild the image and restart. For dependency changes.
   sync+restart  copy, then restart the container.

docker compose watch runs it. On macOS and Windows this is often dramatically faster than a bind mount, because bind mounts cross the VM boundary (see chapter 13).

The commands worth knowing

Section titled “The commands worth knowing”
   docker compose up -d --build      build and start in the background
   docker compose down               stop and REMOVE containers+network
   docker compose down -v            …and DELETE THE VOLUMES. Careful.
   docker compose logs -f api        follow one service
   docker compose ps                 status, ports, health
   docker compose exec api sh        shell in a RUNNING container
   docker compose run --rm api sh    a NEW throwaway container
   docker compose restart api
   docker compose up -d --scale worker=4
   docker compose config             the fully merged, interpolated file
                                     — the first thing to run when
                                       something is not as expected
   docker compose pull               refresh images

Two distinctions:

exec vs run. exec enters the container that is already running. run starts a new container from the same service definition — a fresh one, with its own network attachment, not sharing state. run --rm api npm test is the right way to run tests; exec is the right way to look at a live process.

down vs stop. stop leaves the containers in place. down removes them and the network. down -v also deletes the named volumes — which is how people lose their local database, and it happens often enough to be worth a habit: type down and stop.

Where Compose stops

Section titled “Where Compose stops”

Compose is the best tool available for:

  • local development environments — this is what it is for, and nothing else is as good
  • CI test environments — bring up a real database and a real message broker for integration tests, then tear them down
  • single-machine deployments — a side project, an internal tool, a demo box, a home server

It is the wrong tool the moment you need any of:

   ✗ more than one machine
   ✗ zero-downtime rolling deploys
   ✗ rescheduling containers when a machine dies
   ✗ health-based automatic restarts across a fleet
   ✗ autoscaling
   ✗ secrets management beyond environment variables
   ✗ any kind of declarative reconciliation

Because — and this is the honest core of it — Compose is imperative and one-shot. It makes API calls and exits. Nothing watches. If a container dies, only Docker’s own restart policy acts, and only on that one machine. If the machine dies, everything on it is gone until a human notices.

That gap is exactly what Kubernetes’ reconciliation loop exists to fill: a controller that continuously compares desired state to actual state and acts on the difference, forever. Compose has no equivalent and does not pretend to.

The intermediate options, since “Compose or Kubernetes” is a false binary:

   Docker Swarm       multi-node, rolling updates, secrets, an overlay
                      network — using nearly the same YAML. Still
                      maintained; a small ecosystem. Reasonable for a
                      handful of machines where Kubernetes is too much.


   Nomad              HashiCorp's scheduler. Much simpler than
                      Kubernetes, runs containers and plain binaries,
                      genuinely good at medium scale.


   A PaaS             Fly.io, Render, Railway, App Runner, Cloud Run.
                      You hand them an image; they run it. For most
                      small teams this is the right answer and it
                      is not close.


   Kubernetes         when you have several teams, many services, and
                      someone whose job includes the cluster.
                      Chapter 15 of the Kubernetes guide is blunt
                      about when it is not worth it.

A useful rule: do not move off Compose because of scale you are anticipating. Move when a specific thing it cannot do is costing you real time.

One more comparison worth making, since Compose is often someone’s first declarative tool. Compose and Terraform are the same shape: a file describing what should exist, a command that makes it so, and then nothing — no watcher, no correction. the Terraform guide is what that idea looks like taken seriously, with a dependency graph, a state file and a reviewable plan; Kubernetes is what it looks like made continuous.

What to take from this chapter

Section titled “What to take from this chapter”
  • Compose is imperative and one-shot — it makes the Docker API calls you would have made and exits. Nothing watches or reconciles.
  • It gives you, for free, the two things chapters 9 and 10 said to always do: a user-defined network with DNS by service name, and named volumes. Worth using for two containers.
  • depends_on only means “start first”, not “is ready”. Use a healthcheck plus condition: service_healthy, and start_period so slow startups are not marked unhealthy. The app should still retry regardless.
  • .env (interpolation into the YAML) and env_file (variables into the container) are different files doing different things.
  • Merge semantics: scalars replace, lists append, maps merge — so two files with ports gives you both. docker compose config shows the truth.
  • version: is obsolete, and docker-compose v1 is end-of-life — use docker compose.
  • compose watch with sync and rebuild beats bind mounts for iteration speed, especially on macOS and Windows.
  • exec enters the running container; run starts a new one. down -v deletes your volumes — build the habit of typing plain down.
  • Compose is the best tool for local development, CI environments and single-machine deployments, and the wrong one the moment you need multiple machines, rolling deploys, rescheduling or autoscaling. Swarm, Nomad and a PaaS all sit between it and Kubernetes — the choice is not binary.

Next: tuning and operations — image size, build speed, disk that fills up, log drivers, and the daemon settings worth setting on day one.

Written by Sepuri Sai Krishna

© 2026CC BY 4.0Source