Skip to content
Toolforge

10. The outbox pattern

Chapter 1 said CDC publishes your internal database schema as a public interface. This chapter is the escape from that, without giving up the guarantee that made CDC worth having.

The problem with raw CDC

Section titled “The problem with raw CDC”

Debezium works. It is also, by construction, publishing something you did not design.

   ✗ YOUR SCHEMA IS NOW A PUBLIC API
     rename a column and four teams break.
     a private refactoring is a breaking
     change. you cannot evolve the table
     without a migration plan for everyone.


   ✗ ROWS ARE NOT EVENTS
     consumers want "OrderShipped".
     they get "the status column of row 4471
     changed from 'pending' to 'shipped'".
     the meaning is inferred downstream, by
     everyone, separately, and inconsistently.


   ✗ NO AGGREGATE BOUNDARIES
     an order and its five line items are
     six events on two topics with no
     ordering between them (chapter 8).
     a consumer sees fragments.


   ✗ INTERNAL STATE LEAKS
     the retry_count column, the
     denormalised cache column, the
     temporary migration flag — all
     published, all now depended upon by
     somebody.


   ✗ ONE TABLE, ONE MEANING
     an UPDATE to `orders` might be a
     shipment, a cancellation, an address
     correction or a nightly batch job
     touching every row. downstream cannot
     tell.

That last point is the one that eventually forces the issue. A consumer wanting “orders that shipped” has to reverse-engineer intent from a column diff, and it gets it wrong the first time somebody writes a data migration.

The outbox

Section titled “The outbox”

Write the event you want, in the same transaction as the state change. Let CDC carry the event.

BEGIN;


UPDATE orders
   SET status = 'shipped', shipped_at = now()
 WHERE id = 4471;


INSERT INTO outbox (id, aggregatetype, aggregateid, type, payload)
VALUES (
  gen_random_uuid(),
  'Order',
  '4471',
  'OrderShipped',
  '{"orderId": 4471, "customerId": 812,
    "shippedAt": "2026-09-09T14:20:00Z",
    "carrier": "DHL", "trackingId": "JD0002..."}'
);


COMMIT;
   ONE TRANSACTION.


   both rows commit, or neither does.
   → the state change and the event are
     atomic
   → and Debezium reads the outbox table
     from the log, so delivery is still
     guaranteed

This is the dual-write problem solved twice over. The application still writes to one system. The event still cannot be lost. And now the event is one you designed.

Why it works

Section titled “Why it works”
   ┌──────────────────────────────────┐
   │  APPLICATION                     │
   │  ONE TRANSACTION:                │
   │    UPDATE orders                 │
   │    INSERT INTO outbox            │
   └──────────────┬───────────────────┘
                  ▼
   ┌──────────────────────────────────┐
   │  POSTGRESQL                      │
   │   orders ──┐                     │
   │   outbox ──┼──▶ WAL              │
   └────────────┼─────────┬───────────┘
                │         ▼
                │    DEBEZIUM
                │    capture ONLY the
                │    outbox table
                ▼         │
        (not published)   ▼
                     Kafka: OrderShipped
                            OrderCancelled
                            CustomerCreated

Debezium captures the outbox table and nothing else. The orders table is never published. Its schema is private again.

The EventRouter

Section titled “The EventRouter”

Debezium ships a transform that turns outbox rows into properly-shaped events.

CREATE TABLE outbox (
  id            UUID PRIMARY KEY,
  aggregatetype VARCHAR(255) NOT NULL,   -- "Order"
  aggregateid   VARCHAR(255) NOT NULL,   -- "4471"
  type          VARCHAR(255) NOT NULL,   -- "OrderShipped"
  payload       JSONB        NOT NULL,
  created_at    TIMESTAMPTZ  NOT NULL DEFAULT now()
);
transforms=outbox
transforms.outbox.type=io.debezium.transforms.outbox.EventRouter


transforms.outbox.route.by.field=aggregatetype
transforms.outbox.route.topic.replacement=outbox.event.${routedByValue}


transforms.outbox.table.field.event.key=aggregateid
transforms.outbox.table.field.event.payload=payload
transforms.outbox.table.fields.additional.placement=type:header:eventType,created_at:header:eventTime
transforms.outbox.table.expand.json.payload=true
   OUTBOX ROW
     aggregatetype = "Order"
     aggregateid   = "4471"
     type          = "OrderShipped"
     payload       = {...}


   BECOMES


     topic  outbox.event.Order
     key    "4471"
     value  the payload, unwrapped
     header eventType = "OrderShipped"
     header eventTime = "2026-09-09T..."

Three things fall out of that mapping.

The key is the aggregate id, so every event about order 4471 lands in one partition and is ordered — including events of different types. Chapter 8’s ordering guarantee now applies to the aggregate, not to a row.

The topic is per aggregate type, so Order events are together and Customer events are elsewhere. Consumers subscribe to a domain, not to a table.

The event type is a header, so a consumer can route on it without deserialising the payload.

table.expand.json.payload=true unwraps the JSON string into a real structured value, which matters if you are using Avro — otherwise the payload is a string containing JSON and every consumer parses it twice.

What you get

Section titled “What you get”
   ✓ YOUR SCHEMA IS PRIVATE AGAIN
     refactor `orders` freely. the event
     contract is the payload, and you own
     both sides of it.


   ✓ EVENTS HAVE MEANING
     "OrderShipped", not "a column changed".


   ✓ AGGREGATE BOUNDARIES
     an order and its items are ONE event,
     shaped as a coherent unit.


   ✓ EXPLICIT INTENT
     a data migration touching every row
     writes NO outbox rows. downstream sees
     nothing, correctly.


   ✓ ORDERING PER AGGREGATE
     from the key.


   ✓ YOU CHOOSE WHAT IS PUBLIC
     internal columns stay internal.


   ✓ VERSIONING IS YOURS
     add a v2 event type; publish both
     during a migration.

“A data migration writes no outbox rows” is worth pausing on. With raw CDC, a backfill that touches ten million rows emits ten million change events and floods every consumer. With an outbox, it emits nothing, because nothing meaningful happened.

The costs

Section titled “The costs”

Honestly, because they are real.

   ✗ THE APPLICATION MUST WRITE IT
     every place that changes state must
     also write the event. a code path that
     forgets is silently invisible
     downstream.
     ← the main risk, and it is a
       discipline problem, not a technical
       one


   ✗ MORE WRITES
     an extra insert per business event,
     and more WAL.


   ✗ THE OUTBOX TABLE NEEDS CLEANING
     see below.


   ✗ IT DOES NOT CAPTURE WHAT YOU DID NOT
     THINK OF
     raw CDC gives you everything, including
     changes you had not anticipated wanting.
     the outbox gives you exactly what was
     designed.


   ✗ MORE MOVING PARTS
     a table, a transform, a convention,
     and a review habit.

Mitigate the first with a repository or service layer that writes both, so an individual developer cannot forget. Domain events emitted by the model, persisted by the same unit of work, is the shape that holds up.

Cleaning the table

Section titled “Cleaning the table”

The operational detail people miss.

   the outbox grows forever unless something
   deletes from it.
   OPTION 1  DELETE IMMEDIATELY
     INSERT then DELETE in the same
     transaction.


     ✓ the table stays EMPTY
     ✓ the WAL still contains the insert,
       so Debezium still sees it
     ✓ no cleanup job at all
     ← surprising, and it works


   OPTION 2  PERIODIC DELETE
     DELETE FROM outbox
      WHERE created_at < now() - INTERVAL '7 days';


     ✓ the rows are inspectable while
       debugging
     ✗ a job to run, and vacuum pressure

Option 1 is elegant and genuinely used. CDC reads the write-ahead log, and the insert is in the log whether or not the row still exists. The table is a queue that is always empty.

Two caveats: delete events must be filtered out (transforms.outbox handles this, or delete.handling.mode=drop), and you lose the ability to look at the table when debugging. Start with option 2 and move to option 1 once you trust it.

Either way, on PostgreSQL, watch vacuum — a high-churn table generates dead tuples fast.

Designing the events

Section titled “Designing the events”

The part that determines whether this pays off.

   ✓ PAST TENSE, DOMAIN LANGUAGE
     OrderShipped, PaymentFailed,
     CustomerEmailChanged
     ✗ OrderUpdate, CustomerChange


   ✓ SELF-CONTAINED
     include what consumers need. an event
     that forces every consumer to call back
     into your database has given up most of
     the benefit.


   ✓ INCLUDE IDENTIFIERS AND A TIMESTAMP
     aggregate id, event id, occurred-at.


   ✓ ADDITIVE EVOLUTION
     add optional fields. to change meaning,
     publish a new event type alongside the
     old one.


   ✗ DO NOT DUMP THE ROW
     an outbox whose payload is
     `SELECT * FROM orders` is raw CDC with
     extra steps.

That last one is the failure mode of outbox adoption. Teams adopt the pattern and then serialise the whole row into the payload, which reproduces every coupling problem the pattern exists to solve.

Outbox or raw CDC

Section titled “Outbox or raw CDC”
   RAW CDC                    OUTBOX
   ─────────────────────      ────────────────────
   no application change      application must
                                write events
   everything, including      only what you
     what you did not           designed
     anticipate
   schema is public           schema is private
   row-level                  domain-level
   ready today                a design exercise


   ✓ replicating to a         ✓ integrating with
     warehouse / search /       other SERVICES
     analytics                ✓ a public event
   ✓ you own all the            contract
     consumers                ✓ many teams
   ✓ getting started          ✓ long-lived
   ✓ you cannot change the      integrations
     application

The honest split: raw CDC for data movement, outbox for service integration.

Sending your orders table to ClickHouse or an Iceberg lake is data movement — you want the whole table, the analytics team can cope with schema changes, and designing events for it is wasted effort. Telling the fulfilment service that an order shipped is integration, and it deserves an event you designed.

Many organisations run both, from the same database, with two connectors. That is a good outcome rather than indecision.

The alternative worth knowing

Section titled “The alternative worth knowing”

Listen-to-yourself: the application publishes an event, and its own consumer applies the state change.

   app → Kafka: OrderShipRequested
         → the app's own consumer → DB

It solves the dual-write problem differently and introduces eventual consistency into your own write path — the caller cannot read its own write immediately. Occasionally right, usually more disruptive than an outbox table.

The outbox has the advantage of being entirely conventional from the application’s point of view: a transaction, two inserts, done.

What to take from this chapter

Section titled “What to take from this chapter”
  • Raw CDC publishes your internal schema as a public API, emits row diffs rather than domain events, has no aggregate boundaries, and cannot express intent.
  • The outbox writes the event you designed in the same transaction as the state change, so it is still atomic and still cannot be lost.
  • Debezium captures only the outbox table. Your real tables stay private.
  • The EventRouter maps aggregate type to topic, aggregate id to key, and event type to a header — which gives ordering per aggregate and domain-shaped topics.
  • A data migration writes no outbox rows, so it floods nobody — unlike raw CDC.
  • The main risk is a code path that forgets to write the event. Solve it in a service or repository layer, not by discipline alone.
  • Deleting the outbox row in the same transaction works, because CDC reads the log rather than the table — the queue is always empty.
  • Design events as past-tense domain facts, self-contained, evolving additively. An outbox payload that is the whole row is raw CDC with extra steps.
  • Raw CDC for data movement; outbox for service integration. Running both from one database is a reasonable answer.
  • The outbox gets the event out atomically; it does not run what happens next. If the consumer is a multi-step process with compensations, Temporal is the usual other half of this design.

Next: schema changes — what happens when the table you are capturing is altered, and the history topic that makes it survivable.

Written by Sepuri Sai Krishna

© 2026CC BY 4.0Source