TopolearnReplication is what makes Kafka fault tolerant, and its basic terms appear in every configuration and incident.
A Kafka cluster consists of brokers: servers that store partitions and serve clients. Each partition is copied to several brokers according to the topic's replication factor (3 is the usual production choice).
For each partition one replica is the leader; the others are followers that continuously copy the leader's log. Producers write to the leader and consumers normally read from it. If the leader's broker fails, one of the up-to-date followers, the in-sync replicas (ISR), is elected as the new leader and clients switch automatically. Leaders are spread across brokers so the load is balanced.
Cluster metadata (which topics exist, where each partition's replicas and leader are, configurations, ACLs) is managed by KRaft controllers, a small quorum (typically 3) using Kafka's Raft-based consensus. Since Kafka 4.0, KRaft is the only mode; the older ZooKeeper dependency has been removed. In development a single process can act as both broker and controller.
Clients connect using a list of bootstrap servers; any broker can tell them the full cluster layout, after which they talk directly to the right leaders.
With replication factor 3, a partition survives the loss of two brokers without losing data, provided producers wait for acknowledgement from the replicas (covered under producer reliability).
Remember
- Brokers store partition replicas; replication factor 3 is typical
- Each partition has one leader and follower replicas (the ISR when caught up)
- KRaft controllers manage metadata; ZooKeeper is gone since 4.0
- Clients bootstrap from any broker and then talk to leaders
Watch out for
- Running production topics with replication factor 1, so one disk failure loses data.