Having worked with both Kafka and Redis, this certainly rings true for me:
"Redis streams aren’t exciting for their innovativeness, but rather than they bring building a unified log architecture within reach of a small and/or inexpensive app. Kafka is infamously difficult to configure and get running, and is expensive to operate once you do. [...] Redis on the other hand is probably already in your stack."
My current job is maintaining Kafka infrastructure, dozens of clusters. I literally don’t understand how it got so popular. It is by far the least reliable software I have ever experienced. We hit so many bugs and weird edge cases that essentially cripple the entire cluster and stop all producing. Zookeeper in addition has been a constant source of headaches.
"Redis streams aren’t exciting for their innovativeness, but rather than they bring building a unified log architecture within reach of a small and/or inexpensive app. Kafka is infamously difficult to configure and get running, and is expensive to operate once you do. [...] Redis on the other hand is probably already in your stack."