Benchmarker: https://github.com/Vesiro/vesiro-benchmarker Indexer: https://github.com/Vesiro/vesiro-indexer
We’re building a product designed to improve Elasticsearch performance, and needed a consistent way to create test indexes and rerun search workloads after making changes to our setup. You can use vesiro-benchmarker to run repeatable search benchmarks against an existing Elasticsearch index, and vesiro-indexer is designed to create Elasticsearch indexes from Common Crawl data.
For us, reproducibility in our benchmarking is very important. This is why we chose Common Crawl as a data source, as it enables other people to obtain the source data and inspect how we build test indexes. It also doesn’t hurt that there is tons of Common Crawl data, so we can index very large amounts.
Depending on your needs, you might not need both tools. If you already have your own indexes and just need a simple way to start benchmarking, the benchmarker will be enough. If you are only looking for a way to set up test indexes you might be fine just using the indexer.
We are interested to hear how these tools fit the way other teams benchmark Elasticsearch. Feedback on the query workloads or the Common Crawl indexing setup would be particularly helpful.