Skip to content
Toolforge

11. Schema changes

Chapter 1 said CDC publishes your schema as a public interface. This is the chapter about living with that.

Two different problems

Section titled “Two different problems”
   1  CAN DEBEZIUM STILL READ THE LOG?
      a binlog event from three days ago
      must be interpreted against the
      schema AS IT WAS THREE DAYS AGO.
      → the schema history topic


   2  CAN CONSUMERS STILL READ THE EVENTS?
      a new field, a removed field, a
      changed type.
      → schema registry compatibility, and
        coordination with humans

The first is a Debezium mechanism; the second is an organisational problem. Both need attention, and the second is harder.

The schema history topic

Section titled “The schema history topic”

For MySQL, SQL Server, Oracle and Db2 — connectors that read a log containing raw values without column names.

   A MySQL BINLOG ROW EVENT SAYS:


     table_id 4471
     values: [4471, 812, "shipped", 149.99]


   IT DOES NOT SAY WHICH COLUMNS THOSE ARE.


   → Debezium must know the table's schema
     at that point in the log
   → and that is not necessarily the schema
     the table has now
   day 1   orders: (id, customer_id, status, amount)
   day 2   ALTER TABLE orders ADD COLUMN region
   day 3   orders: (id, customer_id, status, amount, region)


   Debezium restarts, resumes from day 1
     → day-1 events have FOUR values
     → the current schema has FIVE columns
     → without history, it misinterprets
       every one

The schema history topic stores every DDL statement the connector has seen, with its log position. On startup Debezium replays it and rebuilds the schema timeline.

schema.history.internal.kafka.topic=schema-history.orders-cdc
schema.history.internal.kafka.bootstrap.servers=kafka:9092
   ⚠ SINGLE PARTITION — ordering is the
     whole point
   ⚠ replication factor 3
   ⚠ cleanup.policy = delete, retention = -1
     NEVER COMPACT IT. NEVER EXPIRE IT.
   ⚠ nothing else may write to it

This topic is the most commonly damaged piece of Debezium state, and it is usually damaged by a cluster-wide retention policy applied by someone who did not know it existed.

   LOSING IT


   → the connector cannot start
   → error: "Could not find existing
     database schema history"
   → recovery:
       schema.history.internal.store.only.captured.tables.ddl
       + snapshot.mode=recovery
     or
       a full re-snapshot

PostgreSQL and MongoDB do not need it. pgoutput sends relation metadata inline (chapter 6), and MongoDB has no fixed schema. This is a real operational advantage of PostgreSQL CDC and it is rarely mentioned.

The schema change topic

Section titled “The schema change topic”

Different thing, same word. This one is for consumers.

include.schema.changes=true      # default for some connectors

A topic named after topic.prefix alone, carrying a record per DDL statement:

{
  "source": {"db": "app", "table": "orders", "ts_ms": 1757413200123},
  "databaseName": "app",
  "ddl": "ALTER TABLE orders ADD COLUMN region VARCHAR(8)",
  "tableChanges": [{
    "type": "ALTER",
    "id": "\"app\".\"orders\"",
    "table": {
      "primaryKeyColumnNames": ["id"],
      "columns": [ /* the full new column list */ ]
    }
  }]
}

Useful for automation: a sink that adds a column to its target table when the source gains one, or an alert to the data platform team that a captured table changed.

Turn it off if nothing consumes it. It is one more topic and one more thing to reason about.

Downstream compatibility

Section titled “Downstream compatibility”

The harder half.

When the table changes, the change event’s schema changes, and the registry checks it.

   BACKWARD    new schema can read OLD data
               → consumers upgrade FIRST
               → safe: ADD an optional field,
                 REMOVE a field
               ← the usual default


   FORWARD     old schema can read NEW data
               → producers upgrade first
               → safe: ADD a field, remove an
                 optional one


   FULL        both. the strictest useful
               setting.
               → safe: add or remove OPTIONAL
                 fields only


   NONE        no checking. anything goes.
               ← and everything breaks
                 downstream, at read time
Terminal window
curl -X PUT http://registry:8081/config/cdc.public.orders-value \
  -H 'Content-Type: application/json' \
  -d '{"compatibility": "BACKWARD"}'

Set compatibility deliberately per subject. The registry rejecting a schema is a good outcome — it stops the connector, loudly, before a topic contains records nobody can read. NONE moves that failure to every consumer, at 3 a.m.

What each DDL change does

Section titled “What each DDL change does”
   ADD COLUMN (nullable)               ✓ SAFE
     new field appears, null in old
     records. backward and forward
     compatible.


   ADD COLUMN NOT NULL, with a default ✓ SAFE
     ⚠ but on MySQL/Postgres a rewriting
       ALTER can generate an UPDATE event
       PER ROW. a 400M-row table becomes
       400M events. see below.


   DROP COLUMN                         ⚠ BREAKS FORWARD
     consumers reading the old schema
     lose a field they may require.


   RENAME COLUMN                       ✗ WORST CASE
     = drop + add. every consumer breaks.
     the old field vanishes, a new one
     appears, and nothing connects them.


   CHANGE TYPE, widening               ~ USUALLY OK
     int → bigint. Avro may still reject
     it depending on compatibility mode.


   CHANGE TYPE, narrowing              ✗ BREAKS
     and may lose data.


   ADD / DROP INDEX                    ✓ INVISIBLE
     not a schema change for CDC.


   CHANGE PRIMARY KEY                  ✗ SEVERE
     the event KEY changes → partitioning
     changes → ordering and compaction
     break for every existing row.
     → treat as a new table.


   TRUNCATE                            ⚠ op = "t"
     one event, no per-row deletes.
     a sink that only handles c/u/d
     silently keeps everything.


   DROP TABLE                          ⚠ capture
     stops. no event says the table is gone.

Two deserve emphasis.

A rewriting ALTER TABLE can produce an event per row. Adding a NOT NULL column with a default, or changing a type, may rewrite every row — and every rewritten row is a change in the log. A schema migration becomes a 400-million-event flood that arrives with no warning at whatever rate the connector manages. Chapter 14 has this; the mitigation is to know which of your database’s ALTER forms rewrite and which do not.

Changing the primary key is not a schema change, it is a new table. The key determines the partition (chapter 8), so every row moves, ordering breaks, and a compacted topic ends up with two entries per logical row under different keys.

The migration playbook

Section titled “The migration playbook”
   ADDING A COLUMN
     1  add it, nullable
     2  events gain the field, null for
        old records
     3  backfill with an INCREMENTAL SNAPSHOT
        (chapter 5) if consumers need history
     4  consumers pick it up when ready
     → no coordination needed


   REMOVING A COLUMN
     1  ANNOUNCE IT. find consumers first.
     2  wait for them to stop reading it
     3  drop it
     → the announcement is the hard part


   RENAMING A COLUMN
     1  ADD the new column
     2  write BOTH from the application
     3  backfill the new one
     4  migrate consumers
     5  drop the old one
     → an expand-and-contract migration,
       exactly as for any public API


   CHANGING A TYPE
     as for rename. a new column, not an
     ALTER.

“Find the consumers first” is the recurring difficulty, and it is worth building the capability before you need it: consumer-group listings, an owner registered per topic, or a simple registry of who reads what.

Expand-and-contract is the answer for anything breaking, and noticing that it is the same migration you would do for a public HTTP API is the point. CDC made your schema a public API; migrate it like one.

The structural fix

Section titled “The structural fix”
   THE OUTBOX (chapter 10) makes all of this
   go away.


   → the event payload is the contract
   → the table schema is private
   → rename a column freely; the outbox
     payload is unchanged
   → version events explicitly when the
     CONTRACT changes

If schema-change coordination is consuming real time, that is the signal to adopt the outbox for the integrations that need stability, and keep raw CDC for the analytics sinks that can absorb change.

Sinks and schema changes

Section titled “Sinks and schema changes”
   ICEBERG        add a column: metadata-only,
                  free. field ids make renames
                  safe.
                  → the Iceberg guide's
                    chapter 6


   CLICKHOUSE     ALTER TABLE ADD COLUMN,
                  usually cheap.


   ELASTICSEARCH  new fields map dynamically;
                  TYPE CHANGES REQUIRE A
                  REINDEX.


   POSTGRES SINK  a migration on the target.


   A WAREHOUSE    depends. some auto-evolve.

Iceberg is the best-behaved sink here by a distance — its field ids mean a rename upstream does not corrupt anything, and adding a column is a metadata edit. That is a genuine argument for it as a CDC destination.

Elasticsearch is the worst-behaved, because a type change means reindexing.

Monitoring

Section titled “Monitoring”
   ✓ ALERT ON DDL against captured tables
     the schema change topic, or the
     database's own audit
   ✓ ALERT ON REGISTRY REJECTIONS
     the connector stopping on an
     incompatible schema is the system
     working — but somebody must know
   ✓ TRACK SCHEMA VERSIONS PER SUBJECT
     a subject gaining versions quickly
     means an unstable contract
   ✓ REVIEW MIGRATIONS ON CAPTURED TABLES
     a code-review rule: any migration
     touching a captured table needs the
     data platform team's sign-off

That last one is the highest-value control in this chapter, and it costs nothing but a convention.

What to take from this chapter

Section titled “What to take from this chapter”
  • Two problems: whether Debezium can still read the log, and whether consumers can still read the events. The second is organisational and harder.
  • The schema history topic lets MySQL, SQL Server and Oracle connectors interpret old log entries against the schema as it was then. Single partition, replication factor 3, never compacted, never expired.
  • Losing it stops the connector and means recovery mode or a full re-snapshot. It is usually destroyed by a cluster-wide retention policy.
  • PostgreSQL and MongoDB need no history topic — a real and under-appreciated advantage.
  • The schema change topic is a different thing, for consumers, and useful for automating sink migrations.
  • Set registry compatibility deliberately. A rejected schema stopping the connector is the system working; NONE moves the failure to every consumer at read time.
  • Adding a nullable column is safe. Dropping breaks forward compatibility. Renaming is a drop plus an add and breaks everyone.
  • A rewriting ALTER TABLE can emit an event per row — a schema migration becomes a hundred-million-event flood.
  • Changing the primary key is a new table, because the key is the partition.
  • Use expand-and-contract, exactly as for a public HTTP API — because that is what your schema now is.
  • The outbox pattern makes this whole chapter go away for the integrations that need stability.
  • Require sign-off on migrations to captured tables. It costs nothing and prevents most of this.

Next: consumers and sinks — what actually reads a CDC stream, and the upsert pattern they all converge on.

Written by Sepuri Sai Krishna

© 2026CC BY 4.0Source