Otherwise, very thorough and well done benchmark from the looks of it. Redis my beloved not holding up so well against some others these days it looks like.
Also, while I appreciate the thoroughness, I think it would be very useful to reduce the number of graphs significantly. Maybe 10x fewer. Just present the key ones that tell the story, and put the rest in another folder.
Requests are scheduled on half of these. Despite that, a plateau is hit after 8 threads? Is this a 16-core 32-thread type of a setup?
Also, consider redoing this in linear scale.
Edit: Oddly enough, no? 1 thread per core as per https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/cpu-opti...
Most of the graphs plateau around 6 threads, for pretty much all the caches under test. I wonder if there is some interesting architectural issue with cache-sharing on this particular platform?
The main difference appears to be that Garnet is more parallel, according to this student's report of benchmarking various keystores (see the "CPU usage" sections in the PDF) https://www.diva-portal.org/smash/record.jsf?pid=diva2%3A196...
They also have a very clever internal design and do some other tricks like strategically avoiding async/await and moving I/O operations onto the network request thread.
That shouldn't be representative of any modern deployments and not even declaring this outside of the code itself is IMO misleading.
Please fix the documentation or better, run that one and update the graphs.
randomtoast•6mo ago
tidwall•6mo ago