I know we could fork separate class loaders, Uwe. But I had exactly the same kind of concerns you already so accurately pinpointed; if no real gain is to be had I typically vote for simplicity.
Mike -- Ant's overhead is indeed a problem. Uwe's solution with antcontrib (which I already mentioned a while ago I believe) will make it more palatable, but it's still a half-way thing because if we had "real" seed reiteration in the runner then it could also run multiple concurrent copies of the same class (with different master seeds) in the forked JVMs. This would nicely play with what's already in the code. I will get down to it and look at the possibilities and problems again. Thanks for bringing it up, Ryan. I admit it's been on my queue for a long time but I was hesitant to open this particular can of worms... Dawid On Fri, Aug 8, 2014 at 8:13 PM, Uwe Schindler <u...@thetaphi.de> wrote: > Hi, > > I will look into that as a Groovy Skript: The main problem is: You cannot > simply use <antcall/> in a loop, because this would also execute the > dependencies on each run. > > My idea is to do the following: > - maybe subclass antcall Task with Groovy (not sure if this is needed) > - instantiate it with current project > - execute dependent targets > - execute the inner target multiple times: store the project properties first > and restore them after execution. This is done, because ANT properties can > only be set *once*. If you don't give a fixed test seed, each run would pick > a new one (because the project properties are reset, so the seed from the > previous execution is gone). > > Uwe > > ----- > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > >> -----Original Message----- >> From: Ryan Ernst [mailto:r...@iernst.net] >> Sent: Friday, August 08, 2014 5:08 PM >> To: dev@lucene.apache.org >> Subject: Re: Test iterations >> >> Thanks for the extremely thorough answer, Dawid! Entertaining as always. :) >> >> > Should we provide this "beaster" in common-build? >> >> I would use it! It sounds like there is a lot of work involved in making >> tests.iters work better with LuceneTestCase. In the mean time, this sounds >> like a quick solution that might not be as efficient (multiple JVMs), but >> still >> better than having to come up with a bash script? >> >> On Fri, Aug 8, 2014 at 7:28 AM, Michael McCandless >> <luc...@mikemccandless.com> wrote: >> > +1, this sounds awesome? >> > >> > Mike McCandless >> > >> > http://blog.mikemccandless.com >> > >> > >> > On Fri, Aug 8, 2014 at 10:06 AM, Uwe Schindler <u...@thetaphi.de> wrote: >> >> Hi, >> >> >> >> We could emulate the same thing (the repeating beaster) with pure Ant: >> >> >> >> Just repeat the "test" target, which can be done using ant-contrib's "for" >> task or (much simplier) a groovy script using antcall on the test target. >> >> Should we provide this "beaster" in common-build? >> >> >> >> "ant beast-tests -Dbeast.iter=100 -Dtestcase=..." >> >> >> >> Very easy to implement and makes it easier to use for the python haters - >> and comes embedded... >> >> >> >> Uwe >> >> >> >> ----- >> >> Uwe Schindler >> >> H.-H.-Meier-Allee 63, D-28213 Bremen >> >> http://www.thetaphi.de >> >> eMail: u...@thetaphi.de >> >> >> >> >> >>> -----Original Message----- >> >>> From: Michael McCandless [mailto:luc...@mikemccandless.com] >> >>> Sent: Friday, August 08, 2014 3:48 PM >> >>> To: Lucene/Solr dev >> >>> Subject: Re: Test iterations >> >>> >> >>> On Fri, Aug 8, 2014 at 9:35 AM, Uwe Schindler <u...@thetaphi.de> >> wrote: >> >>> > Hi Dawid, >> >>> > >> >>> > Thanks for the very good explanation! Indeed the main problem with >> >>> tests.iters is the static initializers. Maybe put that explanation >> >>> into the Wiki! I sometimes also need to remember it, so it should be >> documented. >> >>> > >> >>> > One (only theoretical) way to solve the whole thing could be: >> >>> > Load the class(es) in a separate classloader for every repeated >> >>> > execution,... but of course this will very fast blow up your >> >>> > permgen (java 6, 7) or anything else we don't know about (java 8). >> >>> > In fact the separate classloader approach is not different from >> >>> > Mike's scripts, just that Mike's script creates a new classloader >> >>> > by forking a new JVM. In fact I don't think the separate >> >>> > classloader approach would be much faster, because the class >> >>> > clones will all have separate compilation paths in Hotspot, so >> >>> > Hotspot cannot share the same assembler code. So except the JVM >> >>> > startup time, you gain nothing. Just permgen issues :-) >> >>> >> >>> The big thing the python beasting scripts avoids is all the ant >> >>> overhead to just get to the point where it actually spawns the JVM >> >>> to run the test. Really, that's all the beasting script does: >> >>> directly spawn the JVM on the test runner (after running "ant >> >>> test-compile" up >> >>> front) and then parse its output/events. >> >>> >> >>> The distributed test runner, which uses rsync/ssh to run tests on N >> >>> machines, is very different from the beasting script: it runs all >> >>> Lucene's tests (instead of a single test over and over) across N >> >>> JVMs on M machines. It "cheats" by taking the union of all CLASSPATHs >> ... >> >>> but this is a huge win because it means all testing is fully >> >>> concurrent, not just concurrent within one module. This script can >> >>> also repeat, which means once all lucene tests finish, re-en-queue all of >> them again. >> >>> >> >>> Mike McCandless >> >>> >> >>> http://blog.mikemccandless.com >> >>> >> >>> -------------------------------------------------------------------- >> >>> - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For >> >>> additional commands, e-mail: dev-h...@lucene.apache.org >> >> >> >> >> >> --------------------------------------------------------------------- >> >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For >> >> additional commands, e-mail: dev-h...@lucene.apache.org >> >> >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For >> > additional commands, e-mail: dev-h...@lucene.apache.org >> > >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional >> commands, e-mail: dev-h...@lucene.apache.org > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
