Kafka offsets

Every message in a partition has an increasing integer offset — 0, 1, 2, ... — that uniquely identifies its position. Consumers track offsets to know where they left off; committed offsets are stored in an internal Kafka topic (__consumer_offsets). Two delivery semantics fall out of when you commit: at-least-once (commit after processing — if you crash mid-process, you'll re-read the message and reprocess, so consumers must be idempotent — this is the default); at-most-once (commit before processing — never duplicates but may lose messages on crash, rarely what you want). Auto-commit every 5s is the easy mode; manual commit after processing is the right mode for anything important. Kafka also supports exactly-once semantics via transactional producers + idempotent consumers, but the setup is non-trivial; most systems use at-least-once + application-level deduplication and are fine.