The dual-write problem and the transactional outbox
Category: messaging and events Difficulty: 4 Asked: 2026-10-31 · messaging-events-01 · Messaging and events · Edit
Tip: answer aloud for ~5 minutes before reading the solution.
Question
The Order service must save an order to Postgres and publish OrderCreated to Kafka. A team writes db.commit(); kafka.send(). What breaks, and how do you redesign it? How do consumers stay correct?
Step-by-step solution
Identify the dual write: crash between commit and send → lost event; send then rollback → phantom event. No 2PC across DB and Kafka in practice.
Transactional outbox: in the same local transaction, insert the order and an outbox row (aggregate id, type, payload).
Relay the outbox: either a poller (simple, adds latency/load) or CDC log-tailing (e.g. Debezium reading the WAL) to publish to Kafka.
Delivery is at-least-once → consumers must be idempotent (dedupe by event id / processed-messages table).
Ordering: partition by aggregate id so events for one order stay ordered.
For multi-service business transactions, chain this with a saga (compensating actions), not distributed transactions.
Final answer
Write the business row and the event into an outbox table in one ACID transaction, publish via CDC (Debezium) or a poller, key by aggregate id for ordering, and make consumers idempotent because delivery is at-least-once.
Trade-offs & common mistakes
Poller vs CDC: simplicity vs latency/operational complexity of connectors.
Proposing XA/2PC — fragile, poorly supported by brokers.
Forgetting consumer dedupe; assuming global ordering.