I asked Claude to envision what a benchmark results page for Solr might look like, and here is what he came up with:
http://www.cominvent.com/pub/solr-benchmarks3.html This is 100% MOCKUP, a single self-contained HTML file, no real benchmarks run. But note how the page is still interactive, lets you zoom in on results, filter benchmarks etc :) My thinking is, we can build a test-runner that is scheduled by Jenkins/GithubWorkflow every night. Each run will provision some EC2 nodes, run a number of benchmarks on branch_10x and branch_9x, tear down the nodes and dump result json files into an S3 bucket. Then post-process to build a results site like the above mock, published under e.g. https://solr.apache.org/benchmarks. My company would be willing to sponsor the AWS bill for the first year, then perhaps someone else could take over. Jan
