janhoy commented on issue #131: URL: https://github.com/apache/solr-orbit/issues/131#issuecomment-6254886868
Yea, depending on what budget / donations we get, we can run often or seldom. At a minimum run for each release. And perhaps once a week from nightly main? Still many things to sort out - What workloads give us the coverage we need for various types of queries - Can we run a mix of single-node and multi-shard workloads? Whilch will be most beneficial to run multi-shard? - Are the workloads we inherit from OSB meaningful in terms of size and thus realistic load on a system? - What box size (CPU, pyhsical RAM) and Solr settings (heap etc) is correct for each workload? - We need to collect telemetry from both the load runner host(s) as well as solr nodes in order to know the environment, ideally also info about merges during indexing and shard sizes etc. This will be background data to back claims that Solr x.y.z can do N QPS for workload W in condition C. Regarding telemetry, I have not yet fully understood the capabilities of OSB here, and the port lacks several telemetry providers. Also, we currently install Solr with SSH on EC2 nodes and do not run node-exporter or similar, so we are a bit short on telemetry. I hope to let "benchmarkd" handle solr provisioning and get more telemetry. It would also be cool to store all detailed telemetry time-series as Parquet in S3, and then do analytics on those by materializing an Iceberg table on top of those files and run some Spark / Trino on top. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
