Yeah you explained it to me (at least once !) Filters are applied outside of the scope of > the runner >
That's the part I keep forgetting ! :) tests.dups returns the whole suite with the same random seed, so every run > is 100% identical > Ah, that's important. I just verified that and you're right. Ran a test w/ -Dtests.iters=2 and -Dtests.dups=5, every invocation used same seeds. We should document that in test-help ! So now I'm even more confused. What is tests.dups good for then? Just running the same suite multiple times, but not changing the seed? That can be achieved w/ tests.iters and tests.seed .. Dawid, can tests.dups use different seeds? Then it'd be really useful cause it can run these iters in parallel. And then we can document that tests.iters + glob is what you should call from eclipse ... Shai On Sun, Dec 9, 2012 at 12:56 PM, Dawid Weiss <[email protected]>wrote: > I think I wrote about it a few times -- in short: what you ask for is > not possible with the current JUnit architecture and Eclipse runner's > implementation. > > The problem is that every test method has a "description" object that > needs to be a unique combination of 'suite name' and 'method name' > (this is the case with methods in Java so it's not a problem for > regular tests). If you want to re-run the same test a few times in > JUnit you need to differentiate every run; otherwise method names > won't be unique and this confuses every IDE and reporting tool out > there. Randomized testing concatenates non-unique descriptions with > the seed and with a sequential number to enforce this uniqueness. > > Then there are filters. Filters are applied outside of the scope of > the runner, so when you filter a test method in Eclipse it simply > applies a pattern over a description. So a description "enhanced" with > the seed and method number has no chance of being matched and is > filtered out -- this is why you get an empty set of tests after trying > to run with tests.iters and a filter. When you specify -Dtests.method > then the filtering is applied at the runner level (as opposed to > Eclipse's filter). > > I use Eclipse as well and believe me, I tried very hard to make it > work but there's just no way to do it. > > As for tests.dups -- this will work under ant only because what it > does is it duplicates entire suites, not methods. This again is not > really JUnit compatible... > > Dawid > > > > On Sun, Dec 9, 2012 at 10:18 AM, Shai Erera <[email protected]> wrote: > > Hi > > > > Can the randomizing test framework add the glob implicitly, if a > > -Dtests.method is defined together with -Dtests.iters? > > I don't mind adding the glob, but it'd be much nicer and easier if the > > framework did it. Like, it could just check that the specific test method > > invocation startsWith tests.method? > > > > The reason I'm asking is because when I run a single test method from > > eclipse and add -Dtests.iters, it doesn't work. > > And I always forget that you cannot run them like that. You have to run > the > > entire test-class, and add the tests.method filter. > > But then I forget to remove the filter when I actually want to test the > > entire class ... or if I do remember that, I need to create > > several run configurations with names ... > > > > This is just a convenience. If JUnit doesn't allow that, I'll live > without > > it. > > > > Shai > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
