Beyond CAP: The Consistency Tradeoff You Actually Make
Every distributed-systems discussion eventually invokes the CAP theorem, usually as "pick two of three: consistency, availability, partition tolerance." That framing is misleading, and it hides the tradeoff you actually make in production.
What CAP really says
Partitions are not optional. On a real network, packets drop and links fail — P
is a fact of life, not a menu item. So CAP isn't "choose two"; it's a conditional:
When a partition happens, you must choose between consistency (C) and availability (A). You cannot have both.
During a partition, a node cut off from its peers faces a dilemma: answer with possibly-stale data (choose A), or refuse to answer until it can guarantee correctness (choose C). "CA" isn't a real operating point — it just means you haven't thought about partitions yet.
PACELC: the missing half
CAP only describes the failure case. But partitions are rare; most of the time the network is healthy — and you're still making a tradeoff. PACELC spells it out:
If there is a Partition, choose A or C; Else, in normal operation, choose Latency or Consistency.
Strong consistency isn't free even when nothing is broken. To guarantee a read sees the latest write, you have to coordinate across replicas — a quorum, a leader round-trip — and coordination costs latency. Relax consistency and you can serve from the nearest replica immediately.
| System | On partition | Normal operation |
|---|---|---|
| Cassandra, Dynamo | A — stay available | L — low latency |
| HBase, Bigtable | C — stay consistent | C — consistent |
| Spanner | C — stay consistent | C — pays latency for it |
| MongoDB (default) | C | C |
Dynamo-style systems (PA/EL) and Bigtable-style systems (PC/EC) sit at
opposite ends. Most real designs are a deliberate point on that line.
The consistency spectrum
"Consistent or not" is also too coarse. Between the extremes lies a spectrum:
- Strong / linearizable — every read sees the most recent write, as if there were a single copy. Easiest to reason about, most expensive to provide.
- Causal — operations that are causally related are seen in the same order by everyone; unrelated ones may diverge. A sensible default for collaborative apps.
- Eventual — replicas converge eventually if writes stop. Cheapest and most available; you absorb the anomalies in application logic.
How to actually choose
Ask what a stale read costs this particular feature:
- A "like" count off by one for a second? Serve eventual, cache hard, stay fast.
- An account balance right before a withdrawal? Pay for strong consistency — a wrong answer here is a real loss.
The mistake is applying one answer everywhere. The same product usually wants strong consistency for its ledger and eventual consistency for its activity feed — and the craft is drawing that line per feature, not per company.