These benchmark posts are good but they are a bit unsatisfying, I wish they’d spend more time a laying out the fundamental differences of each technology and the trade offs that are the result of how the different systems are built.
At the end of the day an engineer shouldn’t pick database because it’s the fastest but because it fits their use case the best.
I get that reddis, memcached, and keydb are all roughly in memory hashmaps but beyond that knowing what makes them different, how hard are they to operate, what they are good at and bad at is going to be more helpful in selecting which technology to use than a simple throughout benchmark.
> At the end of the day an engineer shouldn’t pick database because it’s the fastest but because it fits their use case the best.
Every comparison article doesn't need to (and shouldn't) include a comparison about every kind of detail about two technologies.
> I get that reddis, memcached, and keydb are all roughly in memory hashmaps but beyond that knowing what makes them different, how hard are they to operate, what they are good at and bad at is going to be more helpful in selecting which technology to use than a simple throughout benchmark.
If a post is about speed/performance, and you want to know implementation details (or accessibility or how well it plays with your stack) then look for something else. A quick google search quickly gives me whatever details I want to know about whatever two widely used technologies:
But more importantly its okay for a post to be narrow in scope. The requirement should be that it is not misleading in whatever question it sets out to answer.
The post covers all the points you mention. The benchmark only covers Redis forks so it doesn’t get a whole lot more apples-to-apples than this.
> So there are 3 offerings by 3 companies, all compatible with eachother and based off open source Redis: Elasticache is offered as an optimized service offering of Redis; RedisLabs and Redis providing a core product and monetized offering, and KeyDB which remains a fast cutting edge (open source) superset of Redis. This blog looks specifically at performance, however there is a section at the end of this blog that also outlines other key differences.
The last paragraphs give a good overview of the differences.
First, you mentioned memcached, but the article doesn't cover that at all. Perhaps you meant to say elasticache?
Second, I think the closing section at https://docs.keydb.dev/blog/2020/04/15/blog-post/#how-are-ke... does a reasonable job of answering the "fundamental differences of each technology and the trade offs". If it's not more in depth, that's mostly because it's comparing Redis with two forks of the codebase that haven't diverged that much.
The article mentioned KeyDB is multithreaded fork of Redis. It multithreads several aspects including placing the event loop on multiple threads, with network IO, and query parsing done concurrently.
Redis (single-threaded) is just fine for all but the biggest scale, is well-tested, and because it's a simpler model less likely to have all the terribly complicated edge-case bugs the introduction of concurrency implies.
Also there's a matter of need. It does one no good if a KV store can churn through 200 kops/s if you can only deliver 100kops/s. If one or two redis instances (plus replicas, etc., or maybe a minimal three-master cluster) can serve the entire workload, saving two or three nodes by "upgrading" to the likes of KeyDB doesn't seem worth it. I mean, unless you're aiming to get to the melt-the-metal it's so "hot" all the time performance.
KeyDB is compatible with existing Redis libraries and tools. If you use some of the newer features your library may lack direct support but most have extensiblity features built in.
I’m not familiar with Memtier, but those YCSB numbers for, say, the read-only Workload C look surprisingly low for an in-memory store. 25k/second/thread? Is that just hitting the limits of how fast you can round-trip a request over the network?
KeyDB has some hacker news posts in the past where it was shared but these modifications were not accepted into redis. At the time Redis was not considering multi-threading.
At the end of the day an engineer shouldn’t pick database because it’s the fastest but because it fits their use case the best.
I get that reddis, memcached, and keydb are all roughly in memory hashmaps but beyond that knowing what makes them different, how hard are they to operate, what they are good at and bad at is going to be more helpful in selecting which technology to use than a simple throughout benchmark.