Kafka vs SQS

WHAT IS BEING DECIDED How services hand work to each other asynchronously. Kafka is a durable, ordered, replayable log with multiple independent consumer groups. SQS is a managed queue where a message is consumed once and then gone. Teams often frame this as "which messaging system", but they are not the same category of thing. The clarifying question: does more than one consumer need the same message, and will you ever need to reprocess history? HOW TO WEIGH IT Weight replay and fan-out heavily if events represent facts other teams will want later — a durable log lets a new consumer start from the beginning, which a queue cannot offer at any price. Weight operational burden heavily if nobody will own a broker: self-managed Kafka is a substantial commitment, and even managed Kafka carries partition and consumer-group concepts a queue does not. Weight ordering only if you genuinely need it per key. Many systems assume they need global ordering and do not. WHEN EACH WINS SQS wins for task distribution: send work, one worker does it, done. It is nearly free to operate, scales without thought, and its dead-letter handling is built in. For a small team decoupling a slow job from a request, it is almost always the right answer. Kafka wins when events are a shared substrate — multiple consumers, replay after a bug, per-key ordering, or stream processing. It also wins when retention itself is the feature, because the log becomes the integration point rather than a transport. HOW TO DECIDE Answer the fan-out and replay questions first; they usually decide it outright. If neither applies, choose the queue and revisit when a second consumer appears. Adopting Kafka for a workload a queue handles is one of the most common over-engineering patterns in this space — and the cost lands on whoever is on call. See /dataops for the pipeline discipline.