Re: beasting tests

> Thanks for the explanation. It makes sense that we start with a given > seed and then each iteration is different because it re-uses the same > Random instance (or whatever static state?) without re-initialization? > It doesn't reuse the same random instance - it's not that simple - it

Re: beasting tests

Thanks for the explanation. It makes sense that we start with a given seed and then each iteration is different because it re-uses the same Random instance (or whatever static state?) without re-initialization? On Wed, Apr 3, 2024 at 6:09 PM Dawid Weiss wrote: > > >> Now I just need to

Re: beasting tests

> Now I just need to understand why the test failure is no longer > reproducing lol. > This is indeed the hard part! > Also it's mildly confusing that when you specify tests.iters it prints a > single test seed if it is actually going to use many different ones? > It prints a single seed

Re: beasting tests

Thank you! Now I just need to understand why the test failure is no longer reproducing lol. Also it's mildly confusing that when you specify tests.iters it prints a single test seed if it is actually going to use many different ones? Anyway I will read more docs I am probably still confusing beast

Re: beasting tests

This section of the help file for testing explains the difference between 'beast', 'test' and various reiteration methods - https://github.com/apache/lucene/blob/main/help/tests.txt#L89-L123 In *most* cases, tests.iters will be just as good as beasting (and much faster). The only difference is

Re: beasting tests

I think you could try this: ./gradlew -p lucene/core beast -Ptests.dups=10 --tests TestByteVectorSimilarityQuery I confirmed it uses a different seed (long value) for each run by printing the seed here in

Re: beasting tests

> > ./gradlew -p lucene/core test --tests TestByteVectorSimilarityQuery > -Ptests.dups=1000 -Ptests.multiplier=3 > > and it completes very quickly reporting having run only 13 tests > The task is called 'beast', not 'test', Mike. D.

Re: beasting tests

oh! I overlooked tests.dups -- but it doesn't seem to be doing what I expected. EG I tried ./gradlew -p lucene/core test --tests TestByteVectorSimilarityQuery -Ptests.dups=1000 -Ptests.multiplier=3 and it completes very quickly reporting having run only 13 tests On Tue, Apr 2, 2024 at 4:14 PM

beasting tests

Is there a convenient way to run a test multiple times with different seeds? Do I need to write my own script? I feel like I used to be able to do this in IntelliJ, but that option seems to have vanished, and I don't see any such option in gradle testOpts either. I tried -tests.iter but that