Kafka & Event Streaming
A log you can split, replay, and trust, once you know what each guarantee costs.
5 lessons · ~52 min
By the end you can
- Pick a partition key that keeps per-key order, spreads load, and survives a hot key
- Size a consumer group to its partitions and commit offsets in the right order
- Say exactly what an acked write survives: acks, in-sync replicas, and leader failover
- Get effectively-once results end to end, including the database write Kafka can't cover
- Choose Kafka or a queue on purpose, and size partitions and retention with numbers
Lessons
- 1The log, split into partitions10 minKeys pick the partition, order holds only inside one, and the hot key that breaks the math.
- 2Consumer groups and offsets11 minOne reader per partition, the ceiling that sets, and the commit that decides what a crash costs.
- 3Acked means replicated11 minLeaders, in-sync replicas, acks=1 against acks=all, and what a broker crash takes with it.
- 4Exactly once, honestly11 minWhere duplicates come from, what Kafka's exactly-once covers, and the database write it can't.
- 5Kafka or a queue9 minRetention, compaction, sizing a cluster out loud, and when a plain queue is the better answer.
Design with it after
The design library has full system design prompts on a whiteboard canvas with AI feedback. Every one of them gives you a place to apply this.
Open the design library