Toolforge

DNS

2026-10-26T06:46:26.000Z

The oldest distributed database still running, and the one every request starts with. Delegation down a tree of zones, caches that only obey themselves, a protocol still shaped by a 512-byte packet — and why nothing ever "propagates".

1. What DNS is, and why it exists — A distributed database older than the web, answering one question billions of times a second — what delegation bought, what caching cost, and why it is still the first thing every request does.
2. The namespace — Labels, the trailing dot, case-insensitivity that is not quite case-insensitivity, what a zone actually is, where delegation cuts the tree, and the names that are special.
3. The architecture — Stub, recursive and authoritative servers — what each actually does, the root servers that are not thirteen machines, anycast, and the software that implements each role.
4. Records — The record types that matter — A, AAAA, CNAME, MX, TXT, SRV, CAA and the rest — what each is for, the CNAME rules everybody breaks, and the apex problem.
5. A lookup traced — One getaddrinfo call followed through every cache and server — the stub, the resolver, the root, the TLD, the authority, and what happens the second time.
6. Caching and TTL — The bargain that makes DNS scale — who caches what, negative caching and the SOA minimum, why nothing ever propagates, and how to actually perform a migration.
7. The wire protocol — The message format, the 512-byte limit that still shapes everything, EDNS0 and the 1232-byte compromise, TCP fallback, name compression, and the amplification problem.
8. Zones and delegation — The SOA fields and what each one actually controls, AXFR and IXFR, NOTIFY, TSIG, dynamic updates, and keeping the parent and child in step at a zone cut.
9. DNSSEC — What DNSSEC actually signs, the chain of trust from the root down, proving a name does not exist, the key ceremony, and the operational failures that take domains offline.
10. Encrypted DNS — DoT, DoH, DoQ and ODoH — what they encrypt, what still leaks, how they differ operationally, the centralisation argument, and what they mean for a corporate network.
11. DNS as service discovery — Round-robin, GeoDNS, latency routing and health-checked failover — what DNS can and cannot do as a load balancer, and what Kubernetes, Consul and CoreDNS actually build on top of it.
12. Operating a zone — Registrars and registries, the apex problem in practice, zone-as-code, split-horizon, subdomain takeover, and the changes that take a company off the internet.
13. Tuning and operations — Resolver sizing and cache hit rates, prefetching, stub timeouts, authoritative capacity and RRL, the metrics worth watching, and where the real latency actually is.
14. Failure modes — The incidents DNS actually produces — SERVFAIL that is really DNSSEC, works-for-me caching, exactly-5-second timeouts, dangling CNAMEs, and a diagnostic order that works.
15. Choosing a setup — Managed or self-hosted, one provider or two, which resolver to run, whether to sign, and an honest account of what each choice costs.
16. Cheat sheet — Record types, flags, response codes, dig recipes, SOA fields, TTL policy, special names, a diagnostic order — everything from this guide on one page.

Cassandra

2026-10-26T06:46:26.000Z

The opposite bet: no leader, no failover, every node writable. The ring, the LSM tree, tombstones, compaction, quorum arithmetic, and why the data model is permanent.

1. What Cassandra is, and why it exists — The problem that a single-writer database cannot solve, where Cassandra came from, and the trades it makes to solve it.
2. The ring — Consistent hashing, tokens, virtual nodes, replica placement, gossip, and how a cluster with no leader agrees on anything.
3. The storage engine on disk — The commit log, memtables, and SSTables — the LSM tree that makes writes fast and reads complicated.
4. The data model — Partition keys, clustering columns, and query-first design — the chapter that decides whether your cluster works.
5. A write, traced end to end — From the client through the coordinator to the replicas, into the commit log and memtable, and out to an SSTable.
6. A read, traced end to end — Bloom filters, caches, the partition index, merging SSTables by timestamp, and why reads are the hard part of an LSM tree.
7. Timestamps and tombstones — Last-write-wins conflict resolution, why a delete writes data, gc_grace_seconds, zombie rows, and the tombstone patterns that kill clusters.
8. Compaction — The four strategies, what each optimises for, how to choose, and how to tell when compaction is losing.
9. Consistency levels — The R + W > RF arithmetic, every level and what it costs, and why QUORUM across data centres is the classic expensive mistake.
10. Anti-entropy — hints, read repair, and repair — The three mechanisms that make replicas converge, why repair is a scheduled operational duty, and what happens if you skip it.
11. Lightweight transactions, batches, and counters — Paxos-backed compare-and-set, why BATCH is not a transaction, and the one data type that breaks all the rules.
12. Multi-datacenter, topology, and backups — Snitches, adding a data centre live, active-active deployment, snapshots, incremental backups, and restore.
13. Tuning and day-to-day operations — JVM and garbage collection, the settings that matter, what to monitor, rolling upgrades, and schema changes without breaking the cluster.
14. Failure modes and war stories — The twelve ways Cassandra actually goes wrong — what each looks like, why it happens, how to fix it now, and how to prevent it.
15. When to use Cassandra — and when not to — The honest boundaries. What Cassandra is genuinely best at, the workloads it suits, and the six cases where you should use something else.
16. Cheat sheet — Every command, setting, and rule of thumb from this guide, on one page.

ClickHouse

2026-10-26T06:46:26.000Z

The other direction entirely: columns, not rows. Sparse indexes over granules, compression that changes the economics, MergeTree, and aggregations over billions of rows in under a second.

1. What ClickHouse is, and why it exists — OLTP versus OLAP, why a row store cannot win at analytics, where ClickHouse came from, and the trades it makes.
2. Columns and compression — The files on disk, how a column becomes compressed blocks, the codecs that matter, and why sorting is what makes compression work.
3. MergeTree — parts, granules, and the sparse index — The engine everything is built on — immutable data parts, background merges, granules of 8192 rows, and a primary index that fits in memory.
4. The MergeTree family — Replacing, Summing, Aggregating, Collapsing and VersionedCollapsing — the engines that do work during merges, and the FINAL trap.
5. The ordering key, partitions, and TTL — The most consequential decision in ClickHouse — how to choose ORDER BY, how PARTITION BY differs from it, and how TTL moves and deletes data automatically.
6. An insert, traced — and why batching is mandatory — From INSERT to a part on disk, why one row at a time destroys a cluster, async inserts, and the truth about UPDATE and DELETE.
7. A query, traced end to end — Index analysis, PREWHERE, parallel reading, vectorised execution, and how to read EXPLAIN and the query log.
8. Types and the SQL dialect — Why Nullable costs you, integers sized properly, arrays and Map, Enum and LowCardinality, and the ClickHouse-specific SQL worth knowing.
9. Aggregation, combinators, and states — How GROUP BY actually runs, the combinator suffixes that multiply every function, approximate aggregates, and the memory limits.
10. Joins and dictionaries — Why the right table goes in memory, join order being your job, distributed join pitfalls, and why dictionaries are usually the better answer.
11. Materialised views and projections — The insert trigger that is not a view, the AggregatingMergeTree rollup pattern end to end, refreshable views, and projections.
12. Distributed ClickHouse — Shards and replicas, ReplicatedMergeTree and Keeper, the Distributed engine, sharding keys, and the shared-storage model.
13. Tuning and operations — The system tables that tell you everything, memory and quota limits, backups, upgrades, and the settings actually worth changing.
14. Failure modes — Ten things that actually go wrong in production ClickHouse — the symptom, the cause, the diagnosis, and the fix.
15. When to use ClickHouse — and when not to — What it is genuinely best at, the six cases where you want something else, and how it compares to DuckDB, Druid, Pinot, Snowflake and Elasticsearch.
16. Cheat sheet — The one-page reference — the numbers, the system tables, the DDL template, and the decisions that matter.

CockroachDB

2026-10-26T06:46:26.000Z

Postgres SQL and serializable transactions, spread across machines that are allowed to die. Ranges and per-range Raft, write intents, clocks that disagree on purpose, and the retry error you have to write code for.

1. What CockroachDB is, and why it exists — The gap between a database that is always right and a database that is always up, what it costs to stand in it, and the promises CockroachDB makes you.
2. The architecture — One binary, five layers, symmetric nodes — and why the machine you connect to is almost never the machine holding your row.
3. Ranges and the key space — How a SQL row becomes a key, how the key space is cut into ranges, and why your primary key decides the shape of your cluster's load.
4. Replication and Raft — One consensus group per range, why the leaseholder is separate from the Raft leader, and how the allocator decides where copies of your data live.
5. Transactions — Write intents, the transaction record, parallel commits — and the retry error your application has to be written around.
6. MVCC and clocks — Hybrid logical clocks, the uncertainty interval, and the garbage-collection window that decides how far back you can see.
7. A write traced — One INSERT followed from the client socket through the gateway, the leaseholder, Raft and Pebble — and back, with the milliseconds accounted for.
8. Reads — Why a read is usually one local lookup, and how follower reads, closed timestamps and bounded staleness keep a query inside its own region.
9. SQL and the optimizer — A cost-based planner whose costs include network hops, distributed execution, the join types that matter, and how to read a plan.
10. Schema and indexes — Primary keys as physical decisions, covering and partial indexes, hash sharding, foreign keys, and schema changes that never take the table offline.
11. Multi-region — Survival goals, table localities, and the latency arithmetic of a quorum that spans oceans — the feature people actually buy this database for.
12. Backups and changefeeds — Getting data out — incremental backups that depend on the GC window, restores you have practised, and change data capture that comes with the database.
13. Tuning and operations — What to watch, what to set, and how to find the one query, one range or one row that is hurting the whole cluster.
14. Failure modes — The specific ways a CockroachDB cluster goes wrong — what each looks like from the outside, why it happens, and what to do.
15. When to use it — The honest comparison — against PostgreSQL, Vitess, Spanner, Yugabyte and DynamoDB — and the cases where the answer is plainly no.
16. Cheat sheet — Commands, settings, SQL, metrics, thresholds and error messages — everything from this guide on one page.

DynamoDB

2026-10-26T06:46:26.000Z

Predictable latency bought by refusing to do anything unpredictable. Partition keys that decide everything, capacity units as the unit of thought, single-table design, and a bill that is a direct function of your access patterns.

1. What DynamoDB is, and why it exists — A database that refuses to do anything whose cost it cannot predict — where that principle came from, what it buys, and what it takes away.
2. Tables, keys and items — The data model in detail — primary keys, attribute types, item collections, and the hard limits that shape every design decision you will make.
3. Partitions and the storage layer — How a key becomes a machine — hashing, partition splits, replication across three availability zones, and the throughput ceiling behind every hot-key story.
4. Reads and writes — The whole API surface — GetItem, Query, Scan, the write operations, expressions, conditional writes, and the consistency choice you make on every read.
5. A request traced — One PutItem and one Query followed from the SDK through authentication, routing and replication — with the milliseconds and the capacity units accounted for.
6. Capacity and cost — RCUs and WCUs, on-demand versus provisioned, throttling and bursts — and how to predict the bill from your access patterns before you build.
7. Secondary indexes — Local and global indexes — what each one really is, what they cost on every write, and the backpressure that turns a throttled index into a table-wide outage.
8. Single-table design — Modelling from access patterns instead of entities — overloaded keys, adjacency lists, hierarchical sort keys, and an honest account of when not to bother.
9. Transactions and consistency — Conditional writes, TransactWriteItems, optimistic locking and idempotency — what is guaranteed, what it costs, and what you still have to build.
10. Streams and event-driven patterns — Change data capture built into the table — record formats, Lambda triggers, ordering guarantees, and the patterns that make a key-value store the centre of an architecture.
11. Global tables and backups — Active-active multi-region replication and its last-writer-wins arithmetic, point-in-time recovery, on-demand backups, and exporting to S3 for the queries DynamoDB will not run.
12. Operations and the ecosystem — IAM down to the item level, encryption, DAX, local development, infrastructure as code, and the tools that make the rest of this guide practical.
13. Tuning and operations — The levers that actually change latency and cost — item size, key sharding, pagination, batching, capacity mode — and the checklist to run before you ship.
14. Failure modes — Throttles, hot keys, silent truncation, index backpressure and cost explosions — what each looks like from the outside, and what to do about it.
15. When to use it — The honest comparison — against PostgreSQL, Cassandra, MongoDB, Redis and the managed alternatives — plus the lock-in question, stated plainly.
16. Cheat sheet — Limits, capacity arithmetic, expressions, CLI commands, metrics and error messages — everything from this guide on one page.

Debezium

2026-10-26T06:46:26.000Z

The tool that noticed your database already writes down every change it makes — and that reading that log solves a problem no amount of careful code can.

1. What Debezium is, and why it exists — The dual-write problem, why reading a database's log beats asking it questions, and an honest account of what change data capture costs.
2. The log, per database — WAL, binlog, oplog, capture tables and redo — what each database actually exposes, what it demands in return, and where the sharp edges are.
3. Deployment shapes — Kafka Connect, Debezium Server, the embedded engine and the Kubernetes operator — four ways to run it, and the constraint all four share.
4. The change event — The envelope, the operation codes, tombstones, and the type-mapping surprises that turn a decimal into a base64 string.
5. Snapshots — How the existing rows get out, why the initial snapshot is the worst part of CDC, and the incremental snapshot that fixed it.
6. A change traced — One UPDATE followed from COMMIT through the WAL, logical decoding, the connector, the converter and Kafka — and back again to advance the slot.
7. Offsets and delivery — What a restart actually replays, why at-least-once is the default, and what it takes to make consumers safe.
8. Topics and routing — Naming, partitioning, retention and the routing decisions that determine what every downstream consumer can and cannot rely on.
9. Transformations — Single message transforms — the unwrap everybody uses, masking and filtering, and the point at which you should stop and use a stream processor.
10. The outbox pattern — Publishing the events you designed instead of the table rows you happen to have — with the same transactional guarantee and none of the schema coupling.
11. Schema changes — What happens when the captured table is altered — the history topic that makes it survivable, and the migrations that break four teams at once.
12. Consumers and sinks — What actually reads a change stream — sink connectors, stream processors and services — and the upsert-by-key pattern every one of them converges on.
13. Tuning and operations — Keeping the source database safe, the metrics that matter, throughput tuning, and the runbook you need before the first incident.
14. Failure modes — The specific ways CDC pipelines go wrong — the two that take down the source database, and the many that are silently wrong for months.
15. When to use CDC, and when not — An honest comparison against polling, triggers, application events, Fivetran and the managed services — and the question that decides it.
16. Cheat sheet — Source setup, connector config, the event shape, signals, SMTs and diagnostics — the whole guide on one page.

DuckDB

2026-10-26T06:46:26.000Z

The engine that noticed one machine is far bigger than anyone assumed — a whole analytical database inside your process, with no server at all.

1. What DuckDB is, and why it exists — The analytical database that runs inside your process — where it came from, the gap it noticed, and why a laptop turned out to be bigger than the industry assumed.
2. The embedded model — What actually disappears when a database has no server — the process boundary, zero-copy data exchange, and the four things you give up in return.
3. The storage layer — What is inside a .duckdb file — blocks, row groups, column segments, the compression that runs per segment, and the write-ahead log that makes it durable.
4. The vectorised execution engine — Why 2048 rows is the right batch size, how push-based pipelines differ from pull, and morsel-driven parallelism — the three ideas that make one machine fast.
5. Reading files in place — Parquet, CSV, JSON, globs, Hive partitioning and object storage — the feature that removes the import step, and the details that decide whether it is fast.
6. The SQL dialect — Friendly SQL — the deliberate small kindnesses, the nested types, and the join and window features that make DuckDB's dialect a genuine reason to choose it.
7. A query traced end to end — One query over 41 million rows in Parquet, followed from the string you typed through parser, binder, optimiser, pipelines and morsels to the answer, with timings.
8. The optimizer — Cardinality estimation, join ordering, the rewrite passes that run before them, and how to read EXPLAIN ANALYZE when the plan is wrong.
9. Extensions and federation — The extension system, the ones worth knowing, and the honest account of how much work gets pushed into Postgres, Iceberg or S3 rather than pulled back to you.
10. Joins, aggregation and larger than memory — How hash joins, aggregations and sorts spill to disk, what it costs, and the specific cases where DuckDB still cannot finish a query that does not fit.
11. Concurrency and transactions — What "one writer" actually means, how many readers you can have, the optimistic conflict rules, and the four patterns for getting concurrent work done anyway.
12. The clients and the ecosystem — Python and the zero-copy Arrow path, the CLI, WebAssembly in the browser, dbt-duckdb, MotherDuck — and where the zero-copy claim stops being true.
13. Tuning and operations — The settings that actually matter, memory sizing in containers, backup, and running DuckDB in CI, Docker, Kubernetes and Lambda.
14. Failure modes — Fourteen ways DuckDB goes wrong in practice — the symptom, the mechanism, the immediate fix and the permanent one.
15. When to use DuckDB, and when not — The one question that decides it, honest comparisons against ClickHouse, Polars, SQLite, Postgres, Spark, Trino and a warehouse, and the four ways it is misused.
16. Cheat sheet — Settings, SQL, file reading, diagnostics, extensions and troubleshooting — the whole guide on one page.

Docker

2026-10-26T06:46:26.000Z

The layer under everything above. Namespaces and cgroups, layers and copy-up, the Dockerfile line by line, a build traced through its cache, PID 1 and the ten-second stop, and volumes that survive.

1. What Docker is and why it exists — The problem Docker actually solved — which was not isolation, because Linux already had that. Why VMs were the wrong shape, what Docker invented, and the four different things people call "Docker".
2. What a container actually is — Namespaces, cgroups, capabilities and the root filesystem pivot — built by hand, without Docker, so you can see there is no magic and no container object anywhere in the kernel.
3. Images, layers and registries — Overlayfs from the bottom up, why deleting a file makes the image bigger, content addressing, what a manifest really contains, multi-arch images, and how a registry deduplicates the world.
4. The architecture — CLI, dockerd, containerd, the shim and runc — what each piece is for, why the shim exists, what Kubernetes actually removed in 2020, and why the Docker socket is root on your machine.
5. The Dockerfile — The most important chapter in this guide — every instruction, which ones create layers, the ENTRYPOINT and CMD table everyone gets wrong, ARG versus ENV, COPY versus ADD, and the ordering rule that decides your build times.
6. A build, traced — What happens between pressing enter on docker build and having an image — the context tar, the LLB graph, the exact cache key rule, layer creation, and the manifest that lands on disk.
7. BuildKit, caching and multi-stage builds — Cache mounts that make dependency installs instant, secret mounts that do not leak, multi-stage patterns that cut images by 90 percent, and how to give a fresh CI runner a warm cache.
8. Running containers — The lifecycle from created to dead, PID 1 and the signal problem that makes every deploy ten seconds slower, zombie reaping, exit codes decoded, restart policies, and exec versus attach.
9. Storage and volumes — Why the writable layer is not storage, named volumes versus bind mounts versus tmpfs, the permission problem everyone hits once, the empty-directory surprise, and how to back any of it up.
10. Networking — veth pairs and the bridge, why publishing a port is just a NAT rule, the embedded DNS server that only exists on user-defined networks, host and none and macvlan, and reaching the host from inside.
11. Security — The socket is root, what --privileged really removes, rootless and user namespaces, the capability set worth dropping, read-only roots, and an honest view of image scanning and CVE noise.
12. Compose — The file, the networks and volumes it creates for you, healthchecks and dependency ordering, profiles and overrides, watch mode — and the exact point where Compose stops being the right tool.
13. Tuning and operations — Making images small and builds fast, the disk that always fills up, log rotation you must configure yourself, daemon settings worth setting on day one, and why Docker Desktop is slow on macOS.
14. Failure modes — The complete catalogue — what each failure looks like, what actually causes it, and the fix. Signal handling, cache misses, permission denied, exec format error, no space left, DNS, and the ones that only appear in production.
15. When to use Docker and when not — An honest assessment — where containers are clearly right, where they are a tax you should not pay, and the real alternatives from Podman and Buildpacks to Nix, Firecracker and just shipping a binary.
16. Cheat sheet — Every command, flag, Dockerfile pattern, exit code and diagnostic from this guide, on one page you can keep open.

Argo CD

2026-10-26T06:46:26.000Z

Deployment with the arrow reversed. Why your CI system should not hold production credentials, sync status versus health, and the deletions that were not meant to happen.

1. What Argo CD is, and why it exists — Push versus pull deployment, what GitOps actually claims, the drift problem nobody had a good answer to, and the four things Argo CD is genuinely for.
2. The architecture — Five components, what each one actually does, why Redis is not optional, where the state lives, and how a change gets from a repository to a cluster.
3. The Application resource — Every field that matters, source and destination, sync policy, the difference between managed and tracked resources, and the labels that decide what Argo CD believes it owns.
4. Manifest generation — What the repo server actually does with Helm, Kustomize and plain YAML, why Argo CD never runs helm install, the caching that makes it fast, and config management plugins.
5. The reconciliation loop traced — One commit followed from git push to a healthy pod — every step, what triggers it, how long each takes, and where the delay you are complaining about actually is.
6. Diffing — The three-way merge, normalisation, why an Application shows OutOfSync forever, the annotation size limit, and when server-side apply is the real fix.
7. Sync — Waves, hooks, prune, selective sync, the ordering rules that make complicated deployments work, and the database migration pattern everyone eventually needs.
8. Health — The five health states, how each resource kind is assessed, writing custom Lua checks for custom resources, and why an Application aggregates to the worst thing in it.
9. Repositories and credentials — Private repositories over HTTPS and SSH, GitHub Apps, credential templates, Helm and OCI registries, and the cluster credentials that decide your blast radius.
10. Multi-cluster and tenancy — AppProjects as the real security boundary, RBAC and SSO, sync windows, apps in any namespace, and the blast-radius question nobody asks until it is too late.
11. ApplicationSets and app-of-apps — Generating Applications instead of writing them, every generator that matters, the app-of-apps bootstrap pattern, and the deletion behaviour that has caught everyone once.
12. Secrets and image updates — The problem Argo CD deliberately does not solve, the four ways people solve it compared honestly, and the image-tag question that GitOps makes awkward.
13. Tuning and operations — Sharding the application controller, the settings that matter at scale, resource exclusions, webhooks, notifications, upgrades and what to monitor.
14. Failure modes — Stuck syncs, permanent OutOfSync, self-heal fighting an incident, the Argo CD that cannot fix itself, and the deletions nobody meant to happen.
15. When to use Argo CD, and when not — Argo CD versus Flux, versus pushing from CI, versus Terraform, the honest costs of GitOps, and when the friction is not worth it.
16. Cheat sheet — Commands, annotations, sync options, settings, metrics and procedures — everything from this guide on one page.

Consul

2026-10-26T06:46:26.000Z

Turning a hand-edited list of addresses into a query. Gossip that scales where heartbeats do not, and the fencing token a distributed lock is not a lock without.

1. What Consul is, and why it exists — Four products in one binary — service discovery, health checking, a key-value store and a service mesh. Why the hardcoded IP list stopped working, and what the licence change means.
2. Agents, servers and datacentres — What runs where — the client and server agent split, the catalog, anti-entropy, datacentres and WAN federation, and why every application talks to localhost.
3. Gossip and failure detection — The SWIM protocol in detail — why nodes probe each other instead of heartbeating a server, indirect probes, the suspicion mechanism, and what makes it scale.
4. Raft and consistency modes — What the servers agree on, the three read modes and what each actually costs, autopilot, and the leader election that decides whether writes work at all.
5. Service registration and the catalog — Service definitions, the four ways to register, tags and metadata, multiple instances per node, and the anti-entropy sync that puts back what you deleted.
6. Health checks — The seven kinds of check, hysteresis so one bad probe does not remove an instance, TTL checks and the pattern they enable, and the cascade that takes down a whole dependency graph.
7. Service discovery — The DNS interface and where its convenience betrays you, the HTTP API with blocking queries, prepared queries with failover, and choosing between them.
8. A lookup, traced end to end — One DNS query from the resolver call to the answer, the HTTP version with a blocking query, and what happens the moment an instance dies.
9. The key-value store — Check-and-set, sessions, distributed locks and leader election — plus the fencing token that a lock is not a lock without, and the limits that make it a coordination store rather than a database.
10. The service mesh — Sidecar Envoy proxies, mTLS from a built-in CA, intentions as deny-by-default authorisation, transparent proxy, and an honest account of what adopting it costs.
11. Traffic management — The three configuration entries that shape traffic — resolvers, splitters and routers — the order they apply in, and the canary deploy they add up to.
12. ACLs and security — Tokens, policies and roles, the default-allow to default-deny migration, gossip encryption and TLS, auth methods, and the bootstrap you can lock yourself out of.
13. Tuning and operations — Sizing servers, the settings that actually matter, upgrades without an outage, backups you have tested, and the metrics that warn you before anything breaks.
14. Failure modes and war stories — The gossip key rotation that split the cluster, the service that came back after you deleted it, the DNS that kept returning a dead address, and thirteen more.
15. When to use Consul, and when not to — An honest comparison with etcd, Kubernetes, Istio, Linkerd and the cloud registries — the five cases where it clearly wins, and what the BUSL licence means for the decision.
16. Cheat sheet — Every command, configuration block, API path, DNS form and metric from this guide, on one page.

Ansible

2026-10-26T06:46:26.000Z

Write down what the machine should look like. No agent, no server, nothing to bootstrap — and the honest answer to the charge that all this is convergent, not reproducible.

1. What Ansible is, and why it exists — The fleet problem, why agentless push over SSH won, the honest answer to the charge that configuration management is convergent rather than reproducible, and the four jobs Ansible is genuinely good at.
2. Inventory — Where the list of machines comes from, why groups are the real unit, how group_vars and host_vars work, and the dynamic sources that mean you never maintain a list at all.
3. The connection and execution model — What agentless actually costs — the payload copied per task, why the default is five hosts at a time, the Python interpreter problem, and the two settings that make everything several times faster.
4. Playbooks, plays and handlers — The anatomy of a play, the four task sections and the order they run in, and handlers — which do not run when you expect, do not run at all if something fails, and are worth the trouble anyway.
5. Modules and idempotence — What "changed" really means, why command and shell can never be idempotent, the keywords that let you tell Ansible the truth, and the module vocabulary worth memorising.
6. Variables and precedence — The twenty-two levels of variable precedence in the order that actually matters, why defaults and vars in a role are at opposite ends of it, facts, magic variables, and the scoping rules that generate most confusion in Ansible.
7. Templating with Jinja2 — The language doing the work inside every variable, every when clause and every template file — the filters worth memorising, the whitespace controls, and why everything comes out as a string.
8. Loops, conditionals and error handling — Loops and what they do to registered results, block/rescue/always, the retry pattern, and the include-versus-import distinction that quietly changes how tags and conditions behave.
9. Roles and collections — The directory layout that makes a role work with no configuration, the difference between a dependency and an import, and what changed when Ansible split itself into collections.
10. A play traced end to end — One playbook followed from the command line to the final recap — parsing, inventory, variable merging, fact gathering, the per-task loop, handler flushing, and where every second actually goes.
11. Orchestration and rolling updates — The thing Puppet cannot do — serial batches, draining a load balancer, delegation, running once, and the safe rolling deployment written out in full.
12. Secrets and Ansible Vault — Encrypting files and single values, vault IDs for multiple environments, the four ways a secret leaks anyway, and when to stop using Vault and fetch from a real secret store instead.
13. Testing, tuning and operations — The idempotence test that catches most bad playbooks, linting, Molecule, tags, and the settings that turn a twenty-minute run into a three-minute one.
14. Failure modes — The twelve things that actually go wrong — the variable that will not change, the handler that never runs, the playbook that succeeds and does nothing, and the second run that reports twenty changes.
15. When to use Ansible — and when not to — An honest comparison against images, Terraform, Puppet, Salt and shell scripts — the cases where each clearly wins, and the one question that decides most of it.
16. Cheat sheet — Every command, keyword, precedence rule and gotcha from the guide on one page — including the settings that make Ansible fast and the twenty rules worth following.

Airflow

2026-10-26T06:46:26.000Z

Scheduling a graph instead of guessing at times. Why ds is not today, the top-level code that slows every installation, and the database that quietly stops keeping up.

1. What Airflow is, and why it exists — Where cron stops working, what a DAG buys you, why Airflow is a scheduler and not a data processor, and the four jobs it is genuinely for.
2. The architecture — Scheduler, executor, workers, the DAG processor, the triggerer and the metadata database — what each does, and the Airflow 3 change that stopped tasks talking to the database.
3. Writing DAGs — The TaskFlow API, the top-level code mistake that slows every installation that makes it, dynamic DAG generation, dynamic task mapping, and how to test a DAG.
4. Scheduling and the data interval — Why ds is not today's date, the data interval explained properly, catchup and the backfill that surprised everyone, timetables, and asset-driven scheduling.
5. A DAG run traced end to end — One run followed from the scheduler loop through queueing, dispatch, execution and state — every state transition named, with where the time actually goes.
6. Operators, sensors and hooks — The building blocks, providers, the operators worth knowing, the sensor mode that quietly consumes your worker pool, and why KubernetesPodOperator solves the dependency problem.
7. The executors — Local, Celery, Kubernetes and Edge compared honestly — the startup-cost trade, queues, autoscaling, and how to choose without guessing.
8. State, dependencies and retries — Trigger rules and the join-after-branch surprise, retries and backoff, idempotency as a correctness requirement, clearing tasks, and what "success" actually means.
9. XComs and passing data — What XComs are for, the size limit that decides everything, custom backends, and the rule that Airflow should move control rather than data.
10. The metadata database — The bottleneck in almost every large Airflow installation — how it grows, why the scheduler is database-bound, db clean, and the sizing that keeps it healthy.
11. Deferrable operators and the triggerer — How a task waits for six hours without occupying anything, what a trigger actually is, writing one, and the operational properties of the triggerer.
12. Connections, variables and secrets — Where credentials live by default, why the Fernet key is not what people think, secrets backends, and the Airflow 2 problem that let any DAG read everything.
13. Tuning and operations — Every concurrency knob and how they interact, pools, DAG distribution, logging, monitoring, alerting on data rather than tasks, and upgrades.
14. Failure modes — The DAG that never appears, tasks stuck in scheduled or queued, the accidental catchup, zombies, OOM, and the database that quietly stopped keeping up.
15. When to use Airflow, and when not — Airflow versus Dagster, Prefect, Temporal, Argo Workflows, dbt and plain cron — an honest account of where each wins and where Airflow's age shows.
16. Cheat sheet — Commands, settings, states, template variables, metrics and procedures — everything from this guide on one page.

dbt

2026-10-26T06:46:26.000Z

The tool that decided a SQL transformation is software — and that one function call, ref(), should build the whole dependency graph.

1. What dbt is, and why it exists — The tool that turned SQL transformations into software — where it came from, the one idea it is built on, and an honest account of what it does not do.
2. The project — dbt_project.yml, profiles.yml, the directory layout, adapters and targets — the scaffolding every dbt project has, and what each piece is actually for.
3. Models and ref() — How a SELECT becomes a table, why one function call builds the whole dependency graph, and the difference between ref and source.
4. Materializations — View, table, incremental, ephemeral and materialized view — the DDL each one compiles to, what each costs, and how to choose.
5. Incremental models — The strategies, unique_key, late-arriving data and schema drift — how incremental models save hours and how they silently lose rows.
6. Sources, seeds and snapshots — Freshness checks on raw data, small CSVs in version control, and how snapshots capture history that the source overwrites.
7. A dbt run traced — One command followed end to end — config resolution, parsing, the manifest, Jinja's two passes, graph selection, the thread pool, materialization macros and the artifacts that come out.
8. Jinja and macros — The templating layer — the context dbt gives you, writing macros, running queries at compile time, cross-adapter dispatch, and knowing when to stop.
9. Tests and contracts — Generic tests, singular tests, unit tests and enforced contracts — what testing data actually means, and how much of it is enough.
10. Documentation and lineage — Descriptions, doc blocks, the catalog, exposures and the generated site — documentation that stays true because it lives next to the code.
11. Configuration and precedence — Where config can live, the order it resolves in, how schemas and names are actually computed, and the places dbt genuinely surprises people.
12. State, artifacts and slim CI — Using a production manifest to build only what changed — state selection, defer, retry, and the CI pipeline that makes large projects reviewable.
13. Tuning and operations — Where the time and money actually go, structuring a project that grows to thousands of models, orchestration, and the practices that keep it healthy.
14. Failure modes — The specific ways dbt projects go wrong — what each looks like from the outside, why it happens, and how to stop it happening again.
15. When to use dbt, and when not — An honest comparison against SQLMesh, Spark, orchestrators, warehouse-native tooling, GUI ETL and writing it yourself — and the one question that decides it.
16. Cheat sheet — Commands, selection syntax, configs, Jinja context, incremental strategies, tests and troubleshooting — the whole guide on one page.