6. A build, traced
Docker · 6. A build, tracedWe are going to follow one build all the way through. Every number is realistic for a mid-sized Node service on a developer laptop.
The Dockerfile:
# syntax=docker/dockerfile:1FROM node:20-slim AS buildWORKDIR /appCOPY package.json package-lock.json ./RUN npm ciCOPY . .RUN npm run build
FROM node:20-slimENV NODE_ENV=productionWORKDIR /appCOPY package.json package-lock.json ./RUN npm ci --omit=devCOPY --from=build /app/dist ./distUSER nodeEXPOSE 3000ENTRYPOINT ["node", "dist/server.js"]The command:
docker build -t myapp:v2 .Step 0 — the CLI resolves the frontend
Section titled “Step 0 — the CLI resolves the frontend” t = 0 msThat # syntax=docker/dockerfile:1 line on top is not a comment. It tells BuildKit which Dockerfile frontend to use — the parser itself is a container image, pulled from a registry.
This is why you can use newer Dockerfile features (RUN --mount, COPY --chmod, heredocs) on an older Docker installation: the syntax is versioned independently of the engine. It is one line, it costs one cached pull, and you should put it at the top of every Dockerfile you write.
Step 1 — the context is packaged and sent
Section titled “Step 1 — the context is packaged and sent” t = 0 → 400 ms "Sending build context to Docker daemon"Chapter 4 established that the build runs on the daemon, not on your filesystem. So the CLI walks the directory ., applies .dockerignore, tars what remains, and streams it over the socket.
the directory what is sent
src/ 2 MB src/ 2 MB package.json 4 KB package.json 4 KB node_modules/ 380 MB ✗ (excluded by .dockerignore) .git/ 210 MB ✗ (excluded) dist/ 8 MB ✗ (excluded) ───────────────────── ───────────── 600 MB 2 MBWithout .dockerignore this step alone is 600 MB over a unix socket and takes 20+ seconds. With it, 400 ms. BuildKit improves on this further by transferring incrementally and only what a COPY actually needs — but it can only skip what you told it to ignore.
Step 2 — the Dockerfile becomes a graph
Section titled “Step 2 — the Dockerfile becomes a graph” t = 400 msBuildKit does not execute the Dockerfile top to bottom. It compiles it into LLB (low-level build definition), a directed acyclic graph of operations. Each node is one of: fetch a source, run a command, copy files between results.
┌─ FROM node:20-slim ─┐ │ │ ┌──────────▼─────────┐ ┌───────▼──────────┐ │ STAGE "build" │ │ STAGE 2 (final) │ │ WORKDIR /app │ │ ENV NODE_ENV │ │ COPY package*.json│ │ WORKDIR /app │ │ RUN npm ci │ │ COPY package* │ │ COPY . . │ │ RUN npm ci --omit│ │ RUN npm run build │ │ │ └──────────┬─────────┘ └───────┬──────────┘ │ COPY --from=build │ └──────────────►───────────────┘ │ USER / EXPOSE / ENTRYPOINT │ imageTwo things fall out of the graph shape immediately.
Independent branches run in parallel. The two npm ci invocations have no dependency on each other, so BuildKit runs them simultaneously. The old builder ran everything serially.
Unused work is never done. If the final stage never referenced the build stage, that whole branch would simply not execute. With the old builder, every stage ran whether or not anything used it.
Step 3 — the cache is consulted, instruction by instruction
Section titled “Step 3 — the cache is consulted, instruction by instruction” t = 400 ms → 2 sThis is the part worth understanding precisely, because “why did my cache miss” is the most common Docker question after “why is my image so big”.
For each node, BuildKit computes a cache key and looks for a previous result with that key. The key includes the parent’s key, so a miss at any point poisons everything downstream.
What goes into the key depends on the instruction:
FROM node:20-slim the resolved DIGEST of the base image. (A tag that has been re-pushed upstream is a new digest → total rebuild.)
RUN npm ci THE COMMAND STRING ONLY. Docker does NOT look at what the command does, reads, or produces. `RUN apt-get upgrade` hits the cache forever.
COPY package*.json ./ a hash of the CONTENTS of the matched files, plus their mode and ownership. NOT the timestamp. NOT the path you typed.
ENV / WORKDIR / USER the instruction text.Let us walk our build with a realistic history: yesterday’s build is cached, and this morning someone changed one file in src/.
INSTRUCTION KEY MATCHES? TIME
FROM node:20-slim ✓ CACHED 0 ms WORKDIR /app ✓ CACHED 0 ms COPY package.json package-lock… ✓ CACHED 0 ms ← unchanged RUN npm ci ✓ CACHED 0 ms ← saved 3 min COPY . . ✗ MISS 900 ms ← src changed RUN npm run build ✗ forced 42 s ──────────────────────────────────────────────────────── FROM node:20-slim ✓ CACHED 0 ms ENV NODE_ENV=production ✓ CACHED 0 ms WORKDIR /app ✓ CACHED 0 ms COPY package.json package-lock… ✓ CACHED 0 ms RUN npm ci --omit=dev ✓ CACHED 0 ms ← saved 90 s COPY --from=build /app/dist ./ ✗ MISS 200 ms USER / EXPOSE / ENTRYPOINT ✗ forced ~0 ms ──────────────────────────────────────────────────────── TOTAL ~43 sNow reorder the first stage the wrong way — COPY . . before RUN npm ci — and the same source change gives:
COPY . . ✗ MISS 900 ms RUN npm ci ✗ forced 3 min 10 s RUN npm run build ✗ forced 42 s ──────────────────────────────────────────────────────── TOTAL ~4 minSame code, same result, five times slower, on every single build. That is chapter 5’s ordering rule, priced.
Two cache behaviours that bite
Section titled “Two cache behaviours that bite”RUN does not know what it does. The key is the literal command string. So:
RUN apt-get update && apt-get install -y curlcached in January still installs January’s curl in June, security patches and all, because the string did not change. Fixes: pin explicit versions (curl=7.88.1-10) so the string changes when the version does, or pass --no-cache on scheduled rebuilds, or --build-arg CACHEBUST=$(date +%s) before the line you want to force. The right answer for production is usually a scheduled --no-cache rebuild plus digest-pinned bases.
COPY hashes contents, not paths. So a file that is touched but not modified does not miss. But a file that gains a trailing newline does. And with COPY . ., every file in the context is in the hash — which is why a stray .log from a test run silently costs you three minutes.
Step 4 — the misses actually execute
Section titled “Step 4 — the misses actually execute” t = 2 s → 45 sFor RUN npm run build, BuildKit:
1. mounts the previous step's result as a snapshot (a real container, with namespaces and a root filesystem, exactly as chapter 2 described) 2. runs the command inside it 3. when it exits 0, takes the DIFF of the filesystem 4. stores that diff as a new layer, keyed by the cache key 5. if it exits non-zero, the build fails here, and the partial result is kept — you can `docker run` the last successful layer to debug (chapter 14)Note step 3: the layer is a diff of the whole filesystem after the command. This is why RUN npm run build that also writes a 300 MB cache into ~/.npm produces a 300 MB layer, even though only dist/ mattered. Chapter 7’s cache mounts are exactly the fix.
Step 5 — the image is assembled
Section titled “Step 5 — the image is assembled” t = 45 sOnly the final stage’s layers become the image. The build stage’s layers — including all its dev dependencies and its 400 MB of build tooling — are cache artifacts and are not part of the output.
BuildKit writes:
a CONFIG BLOB { Env, Cmd, Entrypoint, User, WorkingDir, ExposedPorts, architecture, os, rootfs.diff_ids: [sha256:…, sha256:…, …], history: [ … one entry per instruction … ] }
a MANIFEST { config: sha256 of the above, layers: [ {digest, size, mediaType}, … ] }
the LAYER BLOBS themselves, in /var/lib/docker/overlay2/The image ID you see is the hash of the config blob. The tag myapp:v2 is written into the local tag store as a pointer to it — and per chapter 3, a pointer is all it is.
Successfully built sha256:7c9e2f1a… Successfully tagged myapp:v2That history array is worth knowing about: it is what docker history myapp:v2 prints, and it contains the full text of every instruction, including every --build-arg value. It is the reason chapter 5 says ARG is not a secret mechanism.
Step 6 — what did not happen
Section titled “Step 6 — what did not happen”Worth stating explicitly, because people assume otherwise:
- Nothing was pushed. The image exists only on this machine.
docker pushis a separate act (chapter 3). - Nothing was run. A built image is inert.
ENTRYPOINTwas recorded, not executed. HEALTHCHECKdid not run. It is metadata until a container starts.- The base image was not re-checked. With a tag, BuildKit resolves the digest once and caches it.
docker build --pullforces a re-resolve — worth adding to CI so you actually pick up base image patches.
Reading the output
Section titled “Reading the output”BuildKit’s output is dense but every field is useful:
=> [internal] load build definition from Dockerfile 0.0s => [internal] load .dockerignore 0.0s => [internal] load metadata for docker.io/library/node:20 0.6s => [build 1/6] FROM docker.io/library/node:20-slim@sha256… 0.0s => [internal] load build context 0.4s => => transferring context: 2.11MB 0.3s => CACHED [build 3/6] COPY package.json package-lock.json 0.0s => CACHED [build 4/6] RUN npm ci 0.0s => [build 5/6] COPY . . 0.9s => [build 6/6] RUN npm run build 42.1s => [stage-1 6/7] COPY --from=build /app/dist ./dist 0.2s => exporting to image 1.8sCACHED— the key matched. The absence ofCACHEDon a step you expected to hit is the thing to investigate, and the first non-cached line is the cause; everything below it was forced.transferring context: 2.11MB— if this is hundreds of megabytes, fix.dockerignorebefore anything else.exporting to image— being slow here means large layers being compressed.- The
@sha256:…on theFROMline is the resolved digest. Copy it into your Dockerfile if you want reproducibility.
What to take from this chapter
Section titled “What to take from this chapter”# syntax=docker/dockerfile:1makes the Dockerfile parser a versioned, separately-pulled image — newer syntax on older engines. Put it on every Dockerfile.- The CLI tars the context and sends it to the daemon before anything runs.
.dockerignoredecides whether that is 2 MB or 600 MB. - BuildKit compiles the Dockerfile into a DAG, so independent stages run in parallel and unused stages never run at all.
- The cache key is: the base image digest, the literal command string for
RUN, and the content hash forCOPY. A miss poisons every step below it — the first uncached line in the output is always the cause. RUNhas no idea what it does.apt-get updatecached in January still serves January in June. Pin versions, or rebuild with--no-cacheon a schedule.- A
RUNlayer is the diff of the entire filesystem afterwards — including caches the command wrote into~/.npmor~/.cache/pip. That is what chapter 7’s cache mounts solve. - Only the final stage’s layers become the image; earlier stages are cache artifacts, which is why multi-stage builds shrink images so effectively.
docker historycontains every instruction’s full text, including build args — soARGis not a secret mechanism.- A build pushes nothing, runs nothing, and does not re-check the base image unless you pass
--pull.
Next: BuildKit properly — cache mounts, secret mounts, multi-stage patterns, and making the cache survive a CI runner that is destroyed after every job.