Kafka Streams

Kafka Streams is a JVM library for building stream-processing applications on top of Kafka topics — no separate cluster, just a library you embed in your service. It gives you a high-level DSL: filter, map, flatMap, groupBy, time-windowed aggregate, joins between streams, joins between streams and "tables" (stateful views built from compacted topics). It handles the hard parts — local state with RocksDB, checkpointing into an internal replicated topic, exactly-once processing via transactions, elastic scaling across instances. When to reach for it: you need stateful streaming (sliding aggregations, joins, enrichment against a reference table) and you're already on Kafka. The alternatives — Flink for more complex / non-JVM pipelines, simple consumer code for stateless transforms — each have their place. The mental model: SQL for unbounded streams, executed one event at a time, with fault tolerance built in.