Skip to content

1. What CockroachDB is, and why it exists

Two guides in this collection describe opposite instincts.

PostgreSQL never gets an answer wrong. One node accepts every write, holds every lock, decides every conflict — and when that node dies, somebody has to promote a replica and hope the application notices.

Cassandra never goes down. Every node takes writes, there is no leader to lose, a datacentre can burn while the cluster keeps serving — and in exchange you give up transactions, joins, and any expectation that two reads in a row agree.

Between them sits a question neither answers:

Can a database keep serializable SQL transactions while running on machines that are individually allowed to die?

CockroachDB is a sustained attempt to answer yes. This guide is about what that answer actually costs.

You are running PostgreSQL. It works. Then one of three things happens.

① THE MACHINE BECOMES THE LIMIT
Writes no longer fit on one box. You can buy a
bigger box about four more times, then you cannot.
② THE FAILOVER STOPS BEING ACCEPTABLE
Primary dies at 03:00. Promotion takes 90 seconds
if the tooling behaves. Someone must confirm the
old primary is really dead — two primaries is
worse than none.
③ THE LAW ARRIVES
German rows stay in Germany. American rows stay
in America. Still one application, one schema.

The standard answers all work, and all move the problem rather than solving it:

READ REPLICAS writes still land on one node,
reads are stale
SHARDING cross-shard transactions become
(Citus, Vitess, your job, so does resharding, and
app-level) the shard key is a decision you
cannot take back
FAILOVER TOOLING a shorter outage is still an
(Patroni, RDS) outage, and split brain never
fully goes away

Each takes something the database used to guarantee and hands it back to you. Cross-shard atomicity becomes an application concern. Consistency becomes a per-query argument. Failover becomes a runbook you practise and still dread.

CockroachDB’s proposition is that all three should remain the database’s problem.

ONE LOGICAL DATABASE, MANY MACHINES
┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐
│ node 1 │ │ node 2 │ │ node 3 │ │ node 4 │ │ node 5 │
└────────┘ └────────┘ └────────┘ └────────┘ └────────┘
▲ ▲ ▲ ▲ ▲
└──────────┴────┬─────┴──────────┴──────────┘
│
one SQL database, one schema,
one transaction scope, no shard key
Kill any node. Nothing is promoted. Nothing is
reconfigured. Affected queries retry and continue.

There is no primary in that picture, so there is nothing to fail over. That is the whole pitch. The rest of this guide is how it is paid for.

Cockroach Labs was founded in 2015 by Spencer Kimball, Peter Mattis and Ben Darnell — engineers who had worked at Google, two of them on GFS-era storage. They had seen Spanner, Google’s globally distributed SQL database, from the inside, then gone outside and found nothing like it.

The name is the thesis. A cockroach is not fast, elegant or clever. It is extremely hard to kill, and nobody has to tend to it.

The genealogy explains the design:

FROM SPANNER one ordered key space, cut into ranges
each range replicated by consensus
SQL layered over a transactional KV store
distributed transactions are the DB's job
NOT FROM SPANNER TrueTime. Google's atomic clocks and GPS
receivers let Spanner wait out clock
uncertainty in a few ms. Cockroach Labs
had to run on hardware you can rent, so
they use HYBRID LOGICAL CLOCKS and an
UNCERTAINTY INTERVAL instead. → ch 6
FROM etcd the Raft library, literally. → ch 4
FROM POSTGRESQL the wire protocol and much of the
dialect — compatibility, not code.

That last one is why your existing driver works unchanged:

Terminal window
# your psql, your ORM, your driver — nothing knows the difference
psql "postgresql://root@localhost:26257/defaultdb?sslmode=disable"
SELECT version();
-- CockroachDB CCL v25.x (x86_64-pc-linux-gnu, built …)

Port 26257, not 5432, is usually the only thing that changes in a connection string. Chapter 15 is careful about how far that compatibility really goes.

Everything follows from a single structural decision:

The cluster is one sorted key-value map. SQL is an encoding into it. The map is cut into contiguous chunks, and each chunk is replicated by its own independent Raft group.

Read that twice, because it is unlike the other distributed systems here.

etcd runs one Raft group for the whole database — one leader, one log, one machine’s worth of write throughput, forever. Fine when the database is 2 GB of cluster metadata.

CockroachDB runs one Raft group per range, where a range is about 512 MiB of your data.

THE KEY SPACE, CUT INTO RANGES
┌────────────────────────────────────────────────────┐
│ /users/1 … /users/900k … /orders/1 … /orders/4m … │
└────────────────────────────────────────────────────┘
range 7 range 8 range 9 range 10
│ │ │ │
replicated replicated replicated replicated
3× by Raft 3× by Raft 3× by Raft 3× by Raft
leader: n2 leader: n5 leader: n1 leader: n3
Four independent consensus groups. Four leaders.
Adding nodes adds groups, and therefore adds write
capacity — the thing etcd structurally cannot do.

The same algorithm that makes etcd deliberately unscalable makes CockroachDB horizontally scalable. The cost: a transaction touching two ranges is a transaction across two consensus groups, which is where chapters 5 and 6 spend their time.

The fastest way to make the rest of this guide concrete is a throwaway cluster on your laptop:

Terminal window
# a three-node in-memory cluster with a SQL shell attached
cockroach demo --nodes=3 --no-example-database
# or one node with real files
cockroach start-single-node --insecure --store=/tmp/crdb
-- every node, and what it is doing
SELECT node_id, address, locality, is_live FROM crdb_internal.gossip_nodes;
-- the ranges behind a table, and who holds the lease
SHOW RANGES FROM TABLE users WITH DETAILS;

In cockroach demo, \demo shutdown 3 kills a node mid-query. Nothing is promoted, and the query you were running finishes. That demonstration is the product.

Surviving failure without an operations event. The most common honest reason. Not “we have too much data” but “we cannot keep doing 03:00 promotions”.

Data that must live in a particular place. Row-level geographic homing, in one schema, is genuinely hard to build any other way. Chapter 11.

Write throughput past one machine, without choosing a shard key you can never change.

Postgres-shaped SQL on top of all three. Real transactions, real joins, real foreign keys, real secondary indexes — the things sharding usually costs you.

A single-row write that PostgreSQL commits in well under a millisecond takes several milliseconds here, because “committed” means a majority of replicas has it on disk.

PostgreSQL commit one fsync, one machine
CockroachDB commit fsync on a MAJORITY of replicas,
across the network, every time

You are buying failure tolerance with latency. Always.

The wire protocol matches and most of the dialect matches, and then you meet the gaps: extensions, some stored-procedure behaviour, planner differences, features that arrived years after the Postgres ones. Chapter 15 lists them without flinching.

It will run your GROUP BY over a hundred million rows, slowly and expensively. That query belongs in ClickHouse or DuckDB, fed by the changefeeds of chapter 12.

Sequential keys hurt far more here than in PostgreSQL, because one range means one leaseholder means one machine. Chapters 3, 10 and 14.

1. The cluster is ONE SORTED KEY SPACE cut into RANGES.
Every table and index is an encoding into it.
2. Each range is replicated 3× (or 5×) by its own RAFT
group. One replica holds the LEASE and serves reads
and coordinates writes for that range.
3. There is one isolation level that matters:
SERIALIZABLE. The database aborts your transaction
rather than let it see something impossible — so
your application MUST handle retry errors.
4. Time is a HYBRID LOGICAL CLOCK plus an UNCERTAINTY
INTERVAL. Clocks disagree, and this is accounted
for explicitly rather than wished away.
5. Latency is the DISTANCE TO A QUORUM. Every design
decision in chapter 11 is an attempt to shorten it.

Point 3 is the one that reaches your application code:

-- this is a normal, expected outcome, not a bug
ERROR: restart transaction: TransactionRetryWithProtoRefreshError:
WriteTooOldError … (SQLSTATE 40001)

Nothing else in this guide will change how you write software as much as accepting that a transaction may legitimately be told to run again.

Most people will never deploy CockroachDB. It is worth understanding anyway, for the same reason etcd is: it is the clearest worked example of a question the industry is still arguing about.

PostgreSQL grew multi-node ambitions. MySQL got Vitess. Spanner became rentable. Yugabyte, TiDB and Neon all exist. Every one of them answers “how do we keep SQL when one machine is not enough?” — and CockroachDB answers it most legibly, because it does not hide the trade. The retry error is visible. The uncertainty interval has a flag. The latency is the distance to your third replica, and you can measure it.

  • CockroachDB stands in the gap between PostgreSQL’s correctness with one writer and Cassandra’s availability without transactions: serializable SQL on machines that are allowed to die.
  • No primary, no failover. Losing a node is a retry, not an operations event.
  • Descended from Spanner in design, etcd in consensus code, PostgreSQL in wire protocol — with hybrid logical clocks replacing Spanner’s atomic clocks.
  • The structural idea: one sorted key space, cut into ranges, each range its own Raft group — precisely how it scales where etcd cannot.
  • It is not faster than PostgreSQL, not a drop-in replacement, not an analytics engine, and not a fix for a sequential primary key.
  • Carry five ideas: ranges, leaseholders, serializable-only with retry errors your code must handle, hybrid logical clocks, and latency as the distance to a quorum.

Next: the architecture — one binary, five layers, and why the node you connect to is almost never the node holding your row.

Written by Sepuri Sai Krishna

© 2026CC BY 4.0Source