CAP Theorem, PACELC & Consistency Models
In a distributed system, when the network breaks you need either consistency or availability: you cannot get both in full at the same time.
16.1 Why is it used?
- When data lives on many servers/regions, a network partition will happen sooner or later; then you must decide how the system behaves.
- A wrong bank balance is not acceptable, but a slightly old likes count is fine; every feature has a different need.
- This understanding is essential when choosing a database (Cassandra vs Spanner vs MongoDB).
- It is the most common theory question in system design interviews.
16.2 How it works
- C, Consistency: every read sees the latest write (linearizability).
- A, Availability: every request gets a response (no error), even if some nodes are down.
- P, Partition tolerance: the system keeps running even when the network breaks (nodes cannot talk to each other).
- CAP: during a partition choose one of C or A. CP = give an error rather than wrong data; AP = give old data but do answer.
- PACELC: if Partition then A or C, Else (normal time) Latency or Consistency; even in normal time the price of strong consistency is latency.
- Strong consistency: after a write all readers see the new data immediately. Eventual: after a short while everything becomes the same.
- Read-your-writes: at least the user who wrote sees their own new data. Monotonic reads: the user never sees data go backwards.
- Causal consistency: events that are related (a post and then its comment) are shown in the right order.
- 1.Node Mumbai (balance = 500) X partition X Node Delhi (balance = 500)
- 2.CP (consistency first): Paytm / UPI wallet. Send Rs 400 -> the minority side returns an error "Try again later"
- 3.CP: a wrong balance is never shown, no double spend; cost: some users get an error for a while. Spanner, CockroachDB (Raft/Paxos)
- 4.AP (availability first): YouTube views / Instagram likes. A like is pressed -> the Delhi node says OK, accepts it (count 1.2M, old)
- 5.AP: as soon as the network is fine the nodes sync and everything becomes the same (eventual); cost: a slightly old count. Cassandra, DynamoDB (tunable)
- 1.Is there a partition? Yes -> P: choose A or C
- 2.No -> Else: choose L or C (latency or consistency)
- 3.Wallet: PC / EC: consistency during a partition, and even in normal time a quorum write = a few ms slower (the price paid in latency)
- 1.Strong: everyone sees the new data right after a write (e.g. bank balance)
- 2.Causal: related events in the right order (e.g. post -> comment)
- 3.Read-your-writes: the writer sees their own new data (e.g. profile photo)
- 4.Eventual: everything becomes the same after a while (e.g. likes, views)
Strong side: more consistent, more latency. Eventual side: more available, less latency
16.3 Tools
| Tool | Type | How it is used |
|---|---|---|
| Google Spanner / CockroachDB | CP distributed SQL | Strong (external) consistency, Paxos/Raft; during a partition the minority side rejects writes. |
| Apache Cassandra | AP wide-column DB | Eventual consistency by default, but tunable per query with the consistency level (ONE, QUORUM, ALL). |
| Amazon DynamoDB | Managed NoSQL | Eventually consistent reads by default, a strongly consistent read with ConsistentRead=true. |
| MongoDB | Document DB | Reads from the primary are strong; tuned with readConcern/writeConcern majority and causal sessions. |
| Azure Cosmos DB | Multi-model DB | 5 consistency levels: Strong, Bounded staleness, Session, Consistent prefix, Eventual. |
| etcd / ZooKeeper | CP coordination store | Strong consistency via Raft/ZAB: for config, leader election and locks. |
16.4 Comparison: Strong vs Eventual vs Read-your-writes (Session) consistency
| Point | Strong | Eventual | Read-your-writes / Session |
|---|---|---|---|
| What it guarantees | Every read shows the latest write | All replicas become the same at some point | The writer sees their own write immediately |
| Latency | Higher (quorum/consensus) | Lowest | Low-medium |
| Availability (during partition) | Lower: CP | Higher: AP | Higher, sticky for that user |
| Implementation | Raft/Paxos, sync replication, quorum | Async replication | Session token, primary/sticky reads |
| Examples | Spanner, etcd, DynamoDB ConsistentRead | Cassandra ONE, DNS, CDN | Cosmos DB Session, MongoDB causal session |
| Best for | Payment, inventory, bank balance, locks | Likes, views, feeds, recommendations | Profile update, comment post, cart |
16.5 When to choose which
- CP / strong: when wrong data causes harm: money, seat booking, stock count, unique username.
- AP / eventual: when staying available matters and slightly old data is fine: likes, views, feeds.
- Read-your-writes: the user must see their own update immediately, while other users can tolerate a little lag.
- Use different levels per feature within one app: one rule for the whole app is not needed.
16.6 Real-world example
Instagram/YouTube likes vs a Paytm wallet (an illustrative scenario)
- A YouTube video's view count is not shown exactly at once, it updates after a while: eventual consistency (AP).
- Even if a region's network is cut, the video keeps playing and likes are accepted: availability first.
- In a Paytm/UPI wallet the balance must never be double-spent: here we need strong consistency (CP).
- During a partition the wallet service returns a "Try again later" error, but never shows a wrong balance.
- A user changed their profile photo: for read-your-writes, that user's reads come from the primary for a few seconds.
- PACELC: even in normal time the wallet is a few ms slower because of the quorum write: latency is the price paid for consistency.
16.7 Interview tips
- Calling a distributed DB a "CA system" is a mistake: you cannot avoid network partitions, so P has to be accepted.
- Confusing CAP's Consistency with ACID's Consistency is a mistake: they are different things.
- Giving the whole system a single label (CP/AP) is a mistake: most DBs are tunable and the choice is made per operation.
- Interview tip: mention PACELC along with CAP; it shows you also understand the latency trade-off. Number: Cosmos DB has 5 consistency levels.