TopolearnEvery production database runs with replicas, and replica lag causes real, confusing bugs.
Replication keeps copies of the data on several machines. Reasons: survive a machine failure (high availability), spread read traffic across read replicas, and keep copies in other regions.
The common setup is leader-follower (primary-replica): all writes go to one leader, which ships its change log (PostgreSQL's WAL, MySQL's binlog) to followers that replay it. Alternatives are multi-leader (several nodes accept writes and must resolve conflicts) and leaderless quorum systems like Cassandra and DynamoDB.
The key choice is synchronous vs asynchronous:
- Asynchronous: the leader commits without waiting for followers. Fast, but if the leader dies, the last few committed transactions may never have reached any follower and are lost on failover.
- Synchronous: commit waits until at least one follower confirms. No data loss on failover, but higher latency, and writes stall if the synchronous follower is down. A common compromise is one synchronous follower plus asynchronous others.
With async replicas, reads can be stale (replication lag). Classic bug: a user saves their profile, the page reloads from a replica, and the change "disappears". Fixes include reading your own recent writes from the leader, or waiting until the replica has caught up to the write's log position.
Replication is not a backup: a DROP TABLE replicates too. You still need backups with point-in-time recovery.
Remember
- Leader-follower: writes to one node, changes streamed to replicas
- Async replication can lose recent commits on failover; sync adds latency
- Replica lag breaks read-your-writes; replication is not a backup
Watch out for
- Sending a read to a replica right after a write and assuming it reflects that write.