Every operation, traced
Not “INSERT adds a row.” Instead: which lock is taken, which buffer is pinned, which bytes are written where, in what order, and what happens if the power dies at each step.
Toolforge
How the tools that run the internet actually work — traced end to end, in plain English.
The idea
Section titled “The idea”Most documentation tells you which button to press. Most blog posts tell you what someone did once. Very little tells you what the machine is actually doing when you run a command.
That is the gap these guides fill.
For each tool we go all the way down: the processes it starts, the bytes it puts on disk, what happens microsecond by microsecond when a query arrives, how it survives a power cut, where it falls over, and — just as important — when you should not use it at all.
Plain English
Every piece of jargon gets a normal-words definition the first time it appears. If a sentence needs a dictionary, it gets rewritten.
When *not* to use it
Every guide has a chapter arguing against its own tool. Knowing the boundaries is most of the skill.
Built from the source
Behaviour is described as the code actually implements it, with the real parameter names, file names, and system views you will type.
Guides
Section titled “Guides”PostgreSQL
The relational workhorse. One node accepts writes, and it never gets an
answer wrong. Process model, MVCC, the WAL, the planner, replication,
and why VACUUM exists.
Cassandra
The opposite bet: no leader, no failover, every node writable. The ring, the LSM tree, tombstones, compaction, quorum arithmetic, and why the data model is permanent.
Redis
The third bet: keep everything in memory, and use one thread. The event loop, where the memory really goes, expiry and eviction, fork and copy-on-write, Sentinel, Cluster, and Streams.
Kafka
Not a database at all — a durable, replayable log. Partitions and offsets, segments and zero-copy, the ISR, rebalancing, exactly-once, and compaction.
Elasticsearch
The index turned inside out. Postings lists and immutable segments, the analysis chain, shards and relevance, aggregations, and why it should never be your only copy.
MongoDB
Store the thing you fetch, whole. BSON and WiredTiger, the ESR rule, the plan race, replica sets and the oplog, sharding, and the schema patterns that decide everything.
ClickHouse
The other direction entirely: columns, not rows. Sparse indexes over granules, compression that changes the economics, MergeTree, and aggregations over billions of rows in under a second.
MySQL
The database that ran the web. The clustered index that is the table, gap locks and next-key locks, two write-ahead logs kept in step, and schema changes on a table nobody can stop reading.
Kubernetes
Not a database at all — the thing the others run on. Declared state and reconciliation loops, pods and the scheduler, a Service IP that exists nowhere, and an honest chapter on not using it.
Nginx
One master, N never-blocking workers. The event loop, the location matching order everyone gets wrong, the phases a request walks, buffering, caching that survives an outage, and TLS.
Docker
The layer under everything above. Namespaces and cgroups, layers and copy-up, the Dockerfile line by line, a build traced through its cache, PID 1 and the ten-second stop, and volumes that survive.
Prometheus
The one that watches the other eleven. Labels instead of hierarchies, pull instead of push, two bytes a sample, PromQL’s one real idea, and alerts people do not learn to ignore.
Terraform
Declare the destination, not the route. The state file and why every
serious incident is a state incident, the dependency graph, a plan
traced end to end, and count versus for_each.
RabbitMQ
The broker that remembers what each consumer finished. Exchanges and the routing you would otherwise write yourself, quorum queues, prefetch, and building retry with backoff out of parts that have none.
Envoy
The proxy whose configuration arrives over a stream. Filter chains, the response-flag table that decodes every 503, retry budgets, and an honest account of what a service mesh costs.
Vault
Credentials that did not exist a second ago. Dynamic secrets and why they change what a leak means, policies and the path everyone gets wrong, leases at scale, and a recovery plan you have practised.
etcd
The database that would rather stop than be wrong. Raft explained properly, the global revision that Kubernetes is built on, and the two maintenance jobs whose neglect freezes a control plane.
Argo CD
Deployment with the arrow reversed. Why your CI system should not hold production credentials, sync status versus health, and the deletions that were not meant to happen.
Airflow
Scheduling a graph instead of guessing at times. Why ds is not
today, the top-level code that slows every installation, and the
database that quietly stops keeping up.
HAProxy
The specialist. Active health checks that cost one word, TCP mode done properly, and a log line whose two-character flag usually names the bug outright.
OpenTelemetry
A standard, not a product. Instrument once and decide later where the data goes — plus the temporality setting that silently makes every number on your dashboard wrong.
Consul
Turning a hand-edited list of addresses into a query. Gossip that scales where heartbeats do not, and the fencing token a distributed lock is not a lock without.
MinIO
Where the bytes actually live when six other guides tell you to “put it in object storage”. A filesystem with every hard-to-distribute feature deleted.
Ansible
Write down what the machine should look like. No agent, no server, nothing to bootstrap — and the honest answer to the charge that all this is convergent, not reproducible.
systemd
PID 1, and the layer under every other guide here. The cgroup that is the service, the two words that are not the same word, and the cheapest hardening available on a Linux server.
NATS
The only broker here that throws messages away on purpose — and the persistence layer bolted on beside it with the opposite guarantees, on the same connection.
Varnish
A cache that is only a cache, configured in a language that compiles to machine code — and the two mechanisms that keep a site up when its origin is not.
ZooKeeper
The coordination kernel half the data platforms you have heard of were built on — primitives rather than features, and the pause that deletes your lock.
Spark
Many machines answering one question as though they were one — and an honest account of the workloads that have since moved somewhere else.
Flink
The engine that made batch out of streaming rather than the other way round — and the watermark, which is a promise about how long you will wait.
dbt
The tool that decided a SQL transformation is software —
and that one function call, ref(), should build the
whole dependency graph.
Iceberg
The format that stopped a directory from pretending to be a table — one pointer, swapped atomically, and a git-shaped history underneath it.
Trino
The SQL engine that owns no data — one query across systems that were never meant to talk to each other, and the pushdown that decides whether it works.
Debezium
The tool that noticed your database already writes down every change it makes — and that reading that log solves a problem no amount of careful code can.
Temporal
A function that survives the machine it started on. The four lines that describe your business process, made unkillable — because every step it takes is written down.
Helm
The most-complained-about tool in Kubernetes and the most-used. Templating YAML is the worst part; remembering what it installed is the point.
Grafana
The layer that owns no data at all — and turned every monitoring system into the same interface. A dashboard is a saved question, not a saved answer.
DuckDB
The engine that noticed one machine is far bigger than anyone assumed — a whole analytical database inside your process, with no server at all.
Read them against each other
Section titled “Read them against each other”Each guide stands alone. Read a few together and something else appears: these are not thirty-eight separate subjects, they are a small number of questions answered differently. Five databases answering what is the state of the world? in five ways. Four proxies each doing one part of a job the others do not. A handful of systems that everything else quietly depends on.
Where to start
Section titled “Where to start”If you want the fundamentals
PostgreSQL first. MVCC, the write-ahead log, vacuum and query planning are the vocabulary the rest of the site assumes — and every other database is a different answer to the same questions.
If you run things in production
systemd, then Kubernetes. The layer underneath everything, and the layer most people meet first without the one below it.
If you work with data
If you write business logic
Temporal. The multi-step process that must not half-happen — a payment, an order, a provisioning job — is a different problem from a request, and it has a real answer.
If something is on fire
Every guide has a failure modes chapter (14) and a cheat sheet (16), written to be read at 3 a.m. Start there and work backwards.
How these are written
Section titled “How these are written”Every guide is sixteen chapters in the same shape, so the fifth one you read is easier than the first.
| Chapters | What they cover |
|---|---|
| 1 | What it is, why it exists, and what it is not |
| 2–12 | The mechanism — architecture, data model, and every major operation |
| ~6–8 | One operation traced end to end, hop by hop |
| 13 | Tuning and operations |
| 14 | Failure modes — symptom, cause, fix |
| 15 | When not to use it, and what to use instead |
| 16 | Cheat sheet — one page, everything |