Transactional Outbox: Database and Message Consistency

What problem does this solve?

An order commits but the process crashes before publishing its event. Publishing first creates the opposite risk: consumers act on an order that later rolls back. Two independent writes create a failure window in either order.

What is it?

The transactional outbox writes a business change and an event record in one database transaction. A separate relay publishes committed pending records. Delivery is usually at least once, so consumer idempotency remains necessary.

Mental model

Write the invoice and a dispatch instruction in the same ledger update. A courier later reads the instruction. The instruction cannot disappear just because the application stopped after committing the invoice.

How it works internally

An outbox row carries an event ID, payload, state and retry information. A relay claims work, publishes, then marks success. A crash after publish but before marking success causes a duplicate. Concurrent relays need locking or leases. Per-aggregate ordering is a separate design requirement; an outbox alone does not guarantee it.

Production example

The GCP commerce project uses a relay and Pub/Sub workers; CloudOps uses an outbox worker for DynamoDB activity. These are project implementation examples, not claims about an employer's production system.

Node.js / TypeScript implementation

An illustrative pg adapter transaction; tables and a configured Pool are prerequisites. The relay is deliberately outside this transaction. Parameterization and rollback preserve the local atomic boundary.

import { randomUUID } from "node:crypto";
import type { Pool } from "pg";
async function createOrder(pool: Pool, orderId: string) {
  const client = await pool.connect();
  try {
    await client.query("BEGIN");
    await client.query("INSERT INTO orders(id) VALUES ($1)", [orderId]);
    await client.query(
      "INSERT INTO outbox(id, payload) VALUES ($1, $2::jsonb)",
      [randomUUID(), JSON.stringify({ type: "OrderCreated", orderId })]
    );
    await client.query("COMMIT");
  } catch (error) {
    await client.query("ROLLBACK");
    throw error;
  } finally { client.release(); }
}

Common mistakes

Publishing within the SQL transaction still does not make the broker and database atomic. Deleting a row before acknowledgement can lose events. Multiple workers without ownership rules can duplicate work continuously. An endless retry loop hides poison messages.

Performance / reliability considerations

Bound retries with backoff and alert on oldest pending age. Index pending work, plan retention and expose replay tooling. Keep event schemas versioned. Consumers should deduplicate event IDs atomically with their business mutation where possible, including after a relay lease expires.

Interview questions

Which dual-write failure does the pattern fix? Why can duplicates still happen? How do multiple relays claim work? Is ordering guaranteed? What distinguishes publication success from consumer completion?

Quick Revision

Business row plus outbox row commit together. Relay publication happens later. Crash after publish can cause duplicates. Use stable event IDs, ownership/leases, idempotent consumers, bounded retries and retention. Monitor backlog age. Outbox provides a durable publication obligation, not universal exactly-once processing.

Further reading

AWS: transactional outbox pattern ↗