[
https://issues.apache.org/jira/browse/LUCENE-4463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470937#comment-13470937
]
Dawid Weiss commented on LUCENE-4463:
-------------------------------------
Let me start from the beginning. I talked about it once but I can't find it
now.
1. testmethod vs. "tests.method"
The reason for the complication in "testmethod" vs. "tests.method" is stemming
from how JUnit works. A "test description" must (in practice) be unique,
otherwise tools just go crazy. So to make a test repeat, its name must be made
unique. That's why if you do tests.iters=X every repetition of a single test
method will in fact be named uniquely as:
testMethod {#seq seed=[...]}
These are not things just added to the report, this is a "method name" as JUnit
Description object sees it. It's odd but it's a workaround that works and that
is (as far as I'm concerned) the only one possible.
So when you use "-Dtestname=X" this is an alias for "-Dtests.method=X" which
will filter out all these repeated tests (because effectively they don't match
the mattern). In order to include them, you need to add a glob like:
"-Dtests.method=X*". Hoss and I added this to the test-help to make it
clear(er) a while back.
2. Seeds and tests.dup
The "master" test seed is passed to <junit4> task once and it just stays there.
Everything else is derived from it. The "duplication" you see is a simple trick
-- we just duplicate the file name on input. Because every suite gets the same
random seed (mixed with its class name to make it more random across a single
run), a duplicated identical suite will still get the same master seed every
time.
This option was meant for accelerating a test scenario in which we want to
repeat a single suite/seed combination many times and want to do it using
multiple parallel JVMs.
3. What Robert wants (across-jvm repetition of a single suite with a different
seed each time).
Is effectively impossible right now without re-spinning <junit4> with a
different seed each time.
I don't see I could marry all this into working with both the scenario above
and with Robert wants although I admit both are useful. A script (loop) doing
an antcall would work but this seems like an overkill. Fixing this at JUnit4
level isn't trivial either because the seed is currently picked even before
junit4 is started (to select the target charset).
> add support for running the same test method many times
> -------------------------------------------------------
>
> Key: LUCENE-4463
> URL: https://issues.apache.org/jira/browse/LUCENE-4463
> Project: Lucene - Core
> Issue Type: Wish
> Components: general/build
> Reporter: Robert Muir
> Attachments: LUCENE-4463.patch
>
>
> I have a shell script for this, mike has a python script, its annoying :)
> I want to do something like this:
> ant beast -Dtestcase=XXXX -Dtestmethod=YYYY -Diterations=100
> I would be happy with a simple loop that just invokes 'test' somehow: getting
> a fresh new JVM to each iteration is desirable anyway (so you get fresh
> codecs, etc).
> the -Dtests.iters is not really useful for this because it does not allow
> -Dtestmethod and it does not give a fresh jvm.
> bonus points if it can use multiple jvms at the same time though :)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
