We can put some lower limits on CPU and Memory for running a performance test. If those lower limits are not met, then the test will just skip execution.
And then we put some lower bounds (time-wise) on the time spent by different parts of the test like: - Max time taken to index 1 million documents - Max time taken to query, facet, pivot etc - Max time taken to delete 100,000 documents while read and writes are happening. For all of the above, we can publish metrics like 5minRate, 95thPercent and assert on values lower than a particular value. I know some other software compare CPU cycles across different runs as well but not sure how. Such tests will give us more confidence when releasing/adopting new features like pint compared to tint etc. Thanks SG On Sat, Jan 6, 2018 at 9:59 AM, Erick Erickson <[email protected]> wrote: > Not sure how performance tests in the unit tests would be interpreted. If > I run the same suite on two different machines how do I compare the > numbers? > > Or are you thinking of having some tests so someone can check out > different versions of Solr and run the perf tests on a single machine, > perhaps using bisect to pinpoint when something changed? > > I'm not opposed at all, just trying to understand how one would go about > using such tests. > > Best, > Erick > > On Fri, Jan 5, 2018 at 10:09 PM, S G <[email protected]> wrote: > >> Just curious to know, does the test suite include some performance test >> also? >> I would like to know the performance impact of using pints vs tints or >> ints etc. >> If they are not there, I can try to add some tests for the same. >> >> Thanks >> SG >> >> >> On Fri, Jan 5, 2018 at 5:47 PM, Đạt Cao Mạnh <[email protected]> >> wrote: >> >>> Hi all, >>> >>> I will work on SOLR-11771 >>> <https://issues.apache.org/jira/browse/SOLR-11771> today, It is a >>> simple fix and will be great if it get fixed in 7.2.1 >>> >>> On Fri, Jan 5, 2018 at 11:23 PM Erick Erickson <[email protected]> >>> wrote: >>> >>>> Neither of those Solr fixes are earth shatteringly important, they've >>>> both been around for quite a while. I don't think it's urgent to include >>>> them. >>>> >>>> That said, they're pretty simple and isolated so worth doing if Jim is >>>> willing. But not worth straining much. I was just clearing out some backlog >>>> over vacation. >>>> >>>> Strictly up to you Jim. >>>> >>>> Erick >>>> >>>> On Fri, Jan 5, 2018 at 6:54 AM, David Smiley <[email protected]> >>>> wrote: >>>> >>>>> https://issues.apache.org/jira/browse/SOLR-11809 is in progress, >>>>> should be easy and I think definitely worth backporting >>>>> >>>>> On Fri, Jan 5, 2018 at 8:52 AM Adrien Grand <[email protected]> wrote: >>>>> >>>>>> +1 >>>>>> >>>>>> Looking at the changelog, 7.3 has 3 bug fixes for now: LUCENE-8077, >>>>>> SOLR-11783 and SOLR-11555. The Lucene change doesn't seem worth >>>>>> backporting, but maybe the Solr changes should? >>>>>> >>>>>> Le ven. 5 janv. 2018 à 12:40, jim ferenczi <[email protected]> >>>>>> a écrit : >>>>>> >>>>>>> Hi, >>>>>>> We discovered a bad bug in 7x that affects indices created in 6x >>>>>>> with Lucene54DocValues format. The SortedNumericDocValues created with >>>>>>> this >>>>>>> format have a bug when advanceExact is used, the values retrieved for >>>>>>> the >>>>>>> docs when advanceExact returns true are invalid (the pointer to the >>>>>>> values >>>>>>> is not updated): >>>>>>> https://issues.apache.org/jira/browse/LUCENE-8117 >>>>>>> This affects all indices created in 6x with sorted numeric doc >>>>>>> values so I wanted to ask if anyone objects to a bugfix release for 7.2 >>>>>>> (7.2.1). I also volunteer to be the release manager for this one if it >>>>>>> is >>>>>>> accepted. >>>>>>> >>>>>>> Jim >>>>>>> >>>>>> >>>>> >>>>> -- >>>>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker >>>>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: >>>>> http://www.solrenterprisesearchserver.com >>>>> >>>> >>>> >> >
