[
https://issues.apache.org/jira/browse/LUCENE-6194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288470#comment-14288470
]
Robert Muir commented on LUCENE-6194:
-------------------------------------
Can we just remove tests.iters already now that we have 'ant beast'?
tests.iters is a huge trap trap, someone runs it with n=10000 then commits
their code and wonders why jenkins fails. The reason is, our test framework
includes a lot of class-level randomness and when you use test.iters it does
not exhaust those possibilities. This makes it useless.
On the other hand 'ant beast' is the same thing as actually running the test
many times and you can be confident that it is able to explore the entire
search space. We've also done work to make it nicer and more friendly, e.g.
setting better junit4 options for 'beast' so it only includes relevant output
and minimal noise for each iteration.
You can beast with tests.verbose and everything works as you expect.
For seeds that don't reproduce well, you can pass -Dtests.dups to ant beast and
hammer the problematic seed with multiple threads. (it currently gives an
annoying warning from junit for every iteration that am i currently unable to
remove since i dont know how to set the ignore, but it works (separate issue):
"junit4 Duplicate suite name used with XML reports: org.apache.lucene.TestDemo.
This may confuse tools that process XML reports. Set 'ignoreDuplicateSuites' to
true to skip this message."
So yeah, please, kill tests.iters :)
> Reproduce with should include -Dtests.iters=N
> ---------------------------------------------
>
> Key: LUCENE-6194
> URL: https://issues.apache.org/jira/browse/LUCENE-6194
> Project: Lucene - Core
> Issue Type: Bug
> Components: modules/test-framework
> Reporter: Michael McCandless
> Priority: Minor
>
> I was playing with the patch on LUCENE-6066, and hit a test failure, but then
> the "Reproduce with: " failed to reproduce, which threw me for a bit of a
> loop, until I realized it was missing the -Dtests.iters=N.
> To see this, apply the patch on LUCENE-6066 to trunk and then run this:
> {noformat}
> ant test -Dtestcase=TestPriorityQueue
> -Dtests.method=testRemovalsAndInsertions* -Dtests.seed=2576D26D551748E5
> -Dtests.slow=true
> -Dtests.linedocsfile=/lucenedata/hudson.enwiki.random.lines.txt
> -Dtests.locale=ru_RU -Dtests.timezone=Europe/Paris -Dtests.asserts=true
> -Dtests.file.encoding=UTF-8 -Dtests.iters=10
> {noformat}
> (or just insert a random test failure into e.g. TestDemo or something).
> It should fail, but the "Reproduce with: ..." line does not include
> -Dtests.iters=10. I think it should?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
