Thanks for putting this together Ishan, Can you explain what "restart performance" means, and what the y axis number is?
Jan > 9. nov. 2022 kl. 07:39 skrev Ishan Chattopadhyaya <[email protected]>: > > I'm working on automating performance testing, details in > https://issues.apache.org/jira/browse/SOLR-16525. > > Even before I could complete the automation, I observed massive slowdown in > restart performance, now attributable to > https://issues.apache.org/jira/browse/SOLR-16414. This affected 9.1 release > candidate RC1, but is now fixed in 9.1 and 9x branches. > > However, while performance was back to original levels on 9.1 branch, there > was a 80-100% slowdown on the 9x branch even after this fix. > Please see: http://mostly.cool/cluster-test.json.html > The test is here: > https://github.com/fullstorydev/solr-bench/blob/ishan/repeatable-jenkins/suites/cluster-test.json > > In order to investigate the slowdown, I retroactively applied the patch > that fixed the performance problem in SOLR-16414 (removing use of > parallelStream) to the intermediate commits and plotted the graph: > http://mostly.cool/cluster-test-with-patch.html > > And now, two more commits with potential slowdowns are observed. Here are > the JIRA issues I've opened for both: > https://issues.apache.org/jira/browse/SOLR-16530 > https://issues.apache.org/jira/browse/SOLR-16531 > > In a week of working on this automation, I was able to catch 3 slowdowns on > the first thing I automated. It might be good to keep this running and test > other aspects. Going forward, I'll be automating more performance suites > and open blocker JIRA issues on significant performance degradation, > whenever observed. I'll make it easy for all of us to add suites to the > framework and have their personal branches/PRs tested through this. > > Please let me know about any thoughts / concerns / suggestions. > > Thanks, > Ishan --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
