TopolearnEvery Kafka 4 cluster runs on KRaft; operating, scaling and troubleshooting it requires knowing how metadata flows.
In KRaft mode, cluster metadata (topics, partitions, replica assignments, ISR changes, configs, ACLs, producer ids) is stored as events in an internal, single-partition topic, __cluster_metadata, replicated across a quorum of controllers using a Raft-based protocol. One controller is the active controller (the Raft leader); it makes decisions such as leader elections and writes them to the metadata log. Brokers replicate the metadata log and apply it to their in-memory state; snapshots bound replay time.
Compared with the ZooKeeper era, this means faster controller failover, metadata propagation by offset (every broker knows exactly how up to date it is), and support for far more partitions per cluster.
Operational points:
process.rolesisbroker,controlleror both (combined mode, for development only)- run 3 controllers (tolerates 1 failure) or 5 (tolerates 2); a majority must be available for metadata changes
- storage is formatted with
kafka-storage.sh formatand a cluster id before first start; dynamic controller quorums (--standalone/--initial-controllers) allow adding and removing controllers later kafka-metadata-quorum.sh describe --statusshows the leader, epoch and follower lag;kafka-metadata-shell.shbrowses metadata snapshots
Kafka 4.0 removed ZooKeeper entirely; older clusters must first migrate to KRaft on a 3.x bridge release (3.9 recommended) before upgrading to 4.x.
Example
# server.properties for a dedicated controller node
process.roles=controller
node.id=1
controller.quorum.bootstrap.servers=ctl1:9093,ctl2:9093,ctl3:9093
listeners=CONTROLLER://:9093
controller.listener.names=CONTROLLER
log.dirs=/var/lib/kafka/metadata
# format storage once, then start
# kafka-storage.sh format -t $CLUSTER_ID -c server.properties --initial-controllers ...
# check quorum health
# kafka-metadata-quorum.sh --bootstrap-controller ctl1:9093 describe --status
# LeaderId: 2 LeaderEpoch: 7 HighWatermark: 48211 CurrentVoters: [...]Remember
- Metadata is an event log (__cluster_metadata) replicated by a Raft controller quorum
- Brokers replay the metadata log; failover is fast
- Use 3 or 5 dedicated controllers in production
- ZooKeeper is gone in 4.x; migrate via a 3.x bridge release
Watch out for
- Running combined broker+controller nodes in production, so broker load affects metadata availability.