Kafka

Kafka handles high-throughput, fault-tolerant event streaming for systems that process millions of messages per second. It's a distributed event streaming platform that handles millions of messages per second with durability guarantees. Picture a highly reliable conveyor belt for data: producers put messages on, consumers take them off, and nothing gets lost. Apache 2.0. Kafka stores streams of events durably and in order. Consumers can replay from any point in history. Built-in partitioning handles horizontal scaling. Kafka Connect integrates with hundreds of data sources and sinks.

Fully free from Apache. Confluent (founded by Kafka creators) offers Confluent Cloud starting at $0.015/GB ingested, with a free tier of $400 in credits. AWS MSK starts at ~$0.21/hr per broker.

The catch: running Kafka yourself is a serious commitment. A production cluster needs ZooKeeper (or the newer KRaft mode), at least 3 brokers, proper disk provisioning, and someone who understands topic partitioning, consumer groups, and rebalancing. This is not a weekend project. Most teams under 10 engineers should use a managed service or consider simpler alternatives.