← back to stream

Kafka partitions

#backend#kafka

A partition is an ordered, append-only sequence of messages — a slice of a topic. A topic with N partitions gives you N-way parallelism: producers can write to all of them in parallel, and up to N consumers in a consumer group can read independently. The crucial property: order is guaranteed within a partition, not across a topic. This is why the partition key matters — messages with the same key always land in the same partition (hash of key mod partition count), so all events for user:123 are processed in order. If you want strict global ordering, you need a single partition (and give up parallelism). Partition count is the main capacity knob — you can add partitions but it's ugly (breaks key-based ordering for existing keys), so over-provision slightly (~2-3x expected consumer parallelism) at topic creation time.