This is why, at a certain level, after a certain length of time, most software companies (including startups, although it could be argued you're transitioning from startup to real company at that stage) need the oft-maligned "neckbeard" type folks (vmg sports some stubble, perhaps he's a closet neckbeard? :D).
Yes, it's great to have the latest JavaScript ninja working on your front-end and whiz-bang Ruby folks on the back-end but eventually you're gonna run into problems that require RealHardComputerScience(TM) to fix. Or, you just throw more hardware at it and forget about it, and end up paying for that oversight over-and-over-and-over (it looks like it didn't take him much time to fix it)
Not to take anything away from vmg (what a great pull request!)... but "RealHardComputerScience" is writing a hash function, not swapping out one hash function for another based on some profiling. ;-)
It really involves both: particularly if you want high speed hashing, you need to pay close attention to how your hash function is executed by the hardware.
Not to be mean, but how was any of this hard? True, I work on server-side development and I'm not a neckbeard, but this is part of my day-to-day job and I think the approach is pretty straightforward.
Step 1. Design a test performance data set. Great if it comes from your production data!
Step 2. Run your algorithm and attach a profiler.
Step 3. Look at any method about 5% utilization. Can you use a better algorithm, more compact data structure, improve memory locality, or tighten the loop? Googling for fast hash functions would give you a good answer here.
Step 4. Repeat steps 2-3 until you give up.
The difference with this pull request is the level of documentation that went into it. We're impressed because nobody shows these levels of documentation and humor when writing about their change. That's what's impressive.
On one hand I agree with you completely about its apparent "obviousness".
On the other, I've seen so many devs that wouldn't even have been able to go through the process of identifying properly what to change, let alone do it, that I can recognize this pull as great.
The reality is that a majority of coders out there do not know how to properly do their job and they do not even know it (thus they don't try to improve).
(but to be fair, that's also what allow us to ask for the kind of salary we get)
I think this has more to do with the fact that in most workplaces once something works it's taboo to 'waste' more time on it (or even if it's not taboo, it becomes low priority compared to more 'critical' issues), than it does with the particular changes requiring super skills. Though I'd love to work somewhere that was even set up cleanly enough to do meaningful test profiling.
Yeah, when dablooms was posted on HN I wrote my own version to learn it. The first thing I did was use murmer. I am clean shaven.
Few companies need RealHardComputerScience guys. Instead, they need people who can apply the results of CS research to messy business problems. JS rockstar ninja brogrammers can't really do that.
Agreed. As a Ruby/JS programmer I absolutely do not think that JS or Ruby folks can't do hard CS stuff, just that it's really easy to be a commercially successful Ruby or JS programmer and not have any idea what a hash algorithm is. Or know anything about bloom filters.
The problem domain often doesn't expose you to that, and that's unfortunate. There's a lot of research over the last 60 years in CS that can make our lives much easier (obviously lots of research is ongoing, too).
There's a certain elitist tone in your comment which I found unwarranted: RealHardComputerScience can happen in JavaScript or Ruby just as easily as C - often easier as you can focus on the computer science rather than memory management.
I'd also second the other comments that this wasn't a great example of it: profiling and swapping hash functions is classic software engineering.
Yes, it's great to have the latest JavaScript ninja working on your front-end and whiz-bang Ruby folks on the back-end but eventually you're gonna run into problems that require RealHardComputerScience(TM) to fix. Or, you just throw more hardware at it and forget about it, and end up paying for that oversight over-and-over-and-over (it looks like it didn't take him much time to fix it)