TopolearnA local broker and the command-line tools are how you experiment, debug and check what is really in a topic.
The quickest way to run Kafka is the official Docker image, which starts a single-node cluster in KRaft combined mode (broker and controller in one process), listening on port 9092:
docker run -d --name kafka -p 9092:9092 apache/kafka:4.3.1The image includes the command-line tools under /opt/kafka/bin. Every tool takes --bootstrap-server:
kafka-topics.sh --create / --list / --describe / --deletemanages topics;--describeshows partitions, leaders, replicas and the ISRkafka-console-producer.shsends each line you type as a recordkafka-console-consumer.shprints records; add--from-beginningto read history,--groupto join a consumer groupkafka-consumer-groups.sh --describe --group gshows each partition's committed offset, end offset and lagkafka-configs.shreads and changes topic and broker configs
Topics can be auto-created on first use when the broker allows it (the default), but they then get default settings. Create topics explicitly with the partition count and replication factor you want.
For a multi-broker setup locally, use Docker Compose. Managed services such as Confluent Cloud, Amazon MSK and Aiven run Kafka for you with the same client APIs.
Example
docker run -d --name kafka -p 9092:9092 apache/kafka:4.3.1
alias kt='docker exec -it kafka /opt/kafka/bin/kafka-topics.sh --bootstrap-server localhost:9092'
kt --create --topic orders --partitions 3 --replication-factor 1
kt --describe --topic orders
# Topic: orders Partition: 0 Leader: 1 Replicas: 1 Isr: 1 (and 1, 2)
# type a few lines, then Ctrl-D
docker exec -it kafka /opt/kafka/bin/kafka-console-producer.sh \
--bootstrap-server localhost:9092 --topic orders
docker exec -it kafka /opt/kafka/bin/kafka-console-consumer.sh \
--bootstrap-server localhost:9092 --topic orders --from-beginning --group cli
docker exec -it kafka /opt/kafka/bin/kafka-consumer-groups.sh \
--bootstrap-server localhost:9092 --describe --group cli
# GROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG ...Remember
docker run apache/kafkagives a single-node KRaft cluster on 9092- kafka-topics, console producer/consumer and consumer-groups are the daily tools
- Create topics explicitly with the partitions and replication you need
- Consumer lag = log end offset minus committed offset
Watch out for
- Relying on auto-created topics in production and getting one partition with default settings.