Transactions: ACID & Distributed (2PC, Saga, Outbox)
A multi-step job must either complete fully or not happen at all, whether it is one DB or many microservices.
17.1 Why is it used?
- While placing an order the payment was deducted but the order was not created: such a half-done state loses both money and trust.
- Within one DB an ACID transaction gives this guarantee, but in microservices every service has its own DB.
- Distributed transactions need patterns such as 2PC, Saga and Outbox.
- When doing both the DB update and sending the Kafka event, one of them can fail (the dual-write problem); Outbox solves this.
17.2 How it works
- A, Atomicity: all steps or none (rollback). C, Consistency: rules/constraints are never broken.
- I, Isolation: parallel transactions do not disturb each other (Read Committed, Repeatable Read, Serializable). D, Durability: once committed, it stays even after a crash (WAL).
- 2PC (Two-Phase Commit): the coordinator first asks everyone to "prepare" and, if all say yes, sends "commit". Strong, but blocking and slow.
- Saga: break a big transaction into small local transactions; if one fails, run compensating actions for the earlier steps (refund, release seat).
- Choreography saga: services listen to events and do the next step themselves. Orchestration saga: a central orchestrator runs the steps.
- Transactional Outbox: write the business data and the event into the outbox table in one local transaction; a separate process (CDC/poller) sends the event to Kafka.
- Idempotency: a retry must not do the same work twice; use an idempotency key / unique request ID.
- Locking: handle concurrent updates with pessimistic locking (SELECT ... FOR UPDATE) or optimistic locking (version column check).
- 1.A, Atomicity: all steps or none
- 2.C, Consistency: rules / constraints are not broken
- 3.I, Isolation: parallel transactions do not interfere
- 4.D, Durability: commit = survives a crash (WAL)
- 1.1. Order service: orders = PENDING + outbox = OrderCreated, in one local transaction
- 2.2. Debezium reads the event from the outbox and 3. sends OrderCreated to Kafka
- 3.4. Payment service deducts the money (key = order_id, no double charge) and 5. sends PaymentDone
- 4.6. PaymentDone reaches the Restaurant service; 7. the restaurant is closed!
- 5.8. A RestaurantRejected event arrives
- 6.9. Compensation: Payment issues a refund; 10. Order status = CANCELLED
- 7.The user sees: "Order cancelled, refund initiated"
The system stays consistent without 2PC
- 1.2PC (Two-Phase Commit): coordinator asks "prepare?" -> all say yes -> "commit". Strong, but blocking + slow
- 2.Saga: small local transactions; on failure run compensating actions (refund, seat release)
- 3.Transactional Outbox: data + event in one local transaction; CDC/poller sends to Kafka (fixes dual-write)
17.3 Tools
| Tool | Type | How it is used |
|---|---|---|
| PostgreSQL / MySQL InnoDB | ACID relational DB | BEGIN/COMMIT/ROLLBACK, isolation levels, row locks; 2PC as well via XA/PREPARE TRANSACTION. |
| Temporal / Cadence | Workflow orchestrator | To write an orchestration saga as durable workflow code; Uber built Cadence. |
| AWS Step Functions / Azure Durable Functions | Managed workflow | Define saga steps and compensation (catch/rollback) in the cloud. |
| Debezium + Kafka | CDC + event streaming | Reads changes of the outbox table and publishes events to Kafka (Outbox Event Router). |
| Axon / Eventuate / Seata | Saga frameworks | Saga and distributed transaction support in Java microservices (Seata is from Alibaba). |
| Google Spanner / CockroachDB | Distributed SQL | Provide multi-shard ACID transactions internally using 2PC + consensus. |
17.4 Comparison: 2PC vs Saga vs Outbox Pattern
| Point | 2PC | Saga | Transactional Outbox |
|---|---|---|---|
| What it solves | Atomic commit across several DBs | A long business flow across several services | Both the DB write and the event publish, reliably |
| Consistency | Strong (atomic) | Eventual, via compensation | Eventual, at-least-once events |
| Performance | Slow, holds locks | Fast, local transactions | Fast, one local transaction |
| Failure handling | Participants can block if the coordinator crashes | Compensating actions (refund, cancel) | The relay retries, the consumer must be idempotent |
| Isolation | Full | No, intermediate state can be visible | Only up to the local transaction |
| Complexity | Simple protocol, hard to operate | Compensation logic has to be written | Outbox table + CDC/poller |
| Best for | Few participants within one organisation, short transactions | Microservices order/booking flows | Every event-driven microservice |
17.5 When to choose which
- Local ACID transaction: when all the data is in one DB; always the first choice.
- 2PC: few participants, same data center, strong atomicity is a must and latency is acceptable.
- Saga: a multi-step business flow in microservices (order, payment, inventory, delivery).
- Outbox: whenever you must send an event along with a DB update; it is also used in every step of a Saga.
- Use idempotency keys in every case, because retries will certainly happen.
17.6 Real-world example
A Swiggy/Zomato order placement saga (an illustrative scenario)
- The user places an order: the Order service writes the order (status PENDING) and the OrderCreated event into the outbox in a local transaction.
- Debezium sends the event from the outbox to Kafka.
- The Payment service deducts the money and sends a PaymentDone event (idempotency key = order_id, so no double charge).
- The Restaurant service is to accept the order, but the restaurant is closed: a RestaurantRejected event arrives.
- Compensation: the Payment service issues a refund and the Order service sets the status to CANCELLED.
- The user sees "Order cancelled, refund initiated": the system is consistent, without 2PC.
- If everything had been fine, the status would be CONFIRMED and the Delivery service would assign a rider.
17.7 Interview tips
- Using 2PC across microservices is a mistake: if one service is slow everything blocks; Saga is better.
- Publishing to Kafka after the DB commit without an outbox is a mistake: if a crash happens in between, the event is lost.
- Not making compensating actions idempotent is a mistake: a retry can refund twice.
- Interview tip: explain all four ACID letters with an example, then say "in microservices, Saga + Outbox".