Tunable consistency

Apache Cassandra · Practical · 13 min

Consistency level decides whether reads see the latest writes and whether requests succeed when nodes are down.

Each read and write specifies a consistency level (CL): how many replicas must acknowledge before the coordinator answers.

  • ONE, TWO, THREE: that many replicas,
  • QUORUM: a majority of all replicas across data centers (2 of 3 with RF 3),
  • LOCAL_QUORUM: a majority in the coordinator's data center, the usual choice for multi-DC clusters,
  • EACH_QUORUM (writes): a quorum in every data center; ALL: every replica,
  • LOCAL_ONE: one replica in the local data center; ANY (writes only) accepts even a stored hint.

Writes are always sent to all replicas; the CL only decides how many acknowledgements to wait for. Reads contact enough replicas to satisfy the CL and return the newest value by timestamp.

The rule of thumb: if R + W > RF, every read overlaps at least one replica that has the latest write, so you read your writes (strong consistency, ignoring clock skew). With RF 3, QUORUM + QUORUM (2 + 2 > 3) is strongly consistent and tolerates one replica down. ONE + ONE is fastest and most available but may return stale data.

Replicas that missed writes catch up through hinted handoff (the coordinator stores hints for a down node, for 3 hours by default), read repair and anti-entropy repair.

A failed write at a given CL isn't necessarily rolled back: some replicas may have applied it. Make writes idempotent so retrying is safe.

Example

-- cqlsh: the session consistency level applies to following statements
CONSISTENCY LOCAL_QUORUM;
INSERT INTO shop.users_by_email (email, user_id, name)
VALUES ('ana@example.com', 5b6962dd-3f90-4c93-8f61-eabfa4a803e2, 'Ana');

CONSISTENCY ONE;      -- fast, possibly stale read
SELECT name FROM shop.users_by_email WHERE email = 'ana@example.com';

-- With RF = 3:
--   W=QUORUM(2) + R=QUORUM(2) = 4 > 3 -> reads see the latest acknowledged write
--   W=ONE(1)    + R=ONE(1)    = 2     -> may read stale data
--   W=ALL(3)    + R=ONE(1)    = 4     -> consistent, but any down node fails writes

Remember

  • CL = how many replicas must acknowledge, chosen per request
  • R + W > RF gives read-your-writes consistency
  • LOCAL_QUORUM is the usual multi-DC default
  • Hints, read repair and repair bring replicas back in sync

Watch out for

  • Using ALL for safety: one unavailable replica makes requests fail.

Check yourself

  1. With RF 3, which combination guarantees reads see the latest successful write?
  2. With RF 3 and one replica down, can QUORUM reads and writes still succeed?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: cassandra.apache.org/doc/latest/cassandra/architecture/dynamo.html