Slightly improved patch: forbidding tests.iters is not needed. It still makes sense to beast 20 rounds and each test repeated (with same static class seed) 20 times, too -> 400 reps. Also more groovy-like loop.
----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -----Original Message----- > From: Uwe Schindler [mailto:u...@thetaphi.de] > Sent: Saturday, August 09, 2014 8:08 PM > To: dev@lucene.apache.org > Subject: RE: Test iterations > > Hi, > > attached you will find the beaster: > > - Only modifies common-build.xml, so no inherit down (makes no sense > otherwise, as you would never run "ant beast-test" from top-level. So you > have to go to correct submodule and run "ant beast-test -Dbeast.iters=n - > Dtestcase=..." from there > - Uses "antcall" in a loop, invoking the internal dependency-less "-test" > target. My first impl used the test-macro directly, but this did not work, > because test-macro sets non-local properties, which are then available on > second round, causing errors or use always same seed. Antcall creates a new > project each time and runs tests. > > I can open an issue or just commit this :-) > > Uwe > > ----- > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > > > -----Original Message----- > > From: Uwe Schindler [mailto:u...@thetaphi.de] > > Sent: Friday, August 08, 2014 8:13 PM > > To: dev@lucene.apache.org > > Subject: RE: Test iterations > > > > Hi, > > > > I will look into that as a Groovy Skript: The main problem is: You > > cannot simply use <antcall/> in a loop, because this would also > > execute the dependencies on each run. > > > > My idea is to do the following: > > - maybe subclass antcall Task with Groovy (not sure if this is needed) > > - instantiate it with current project > > - execute dependent targets > > - execute the inner target multiple times: store the project > > properties first and restore them after execution. This is done, > > because ANT properties can only be set *once*. If you don't give a > > fixed test seed, each run would pick a new one (because the project > > properties are reset, so the seed from the previous execution is gone). > > > > Uwe > > > > ----- > > Uwe Schindler > > H.-H.-Meier-Allee 63, D-28213 Bremen > > http://www.thetaphi.de > > eMail: u...@thetaphi.de > > > > > > > -----Original Message----- > > > From: Ryan Ernst [mailto:r...@iernst.net] > > > Sent: Friday, August 08, 2014 5:08 PM > > > To: dev@lucene.apache.org > > > Subject: Re: Test iterations > > > > > > Thanks for the extremely thorough answer, Dawid! Entertaining as > > > always. :) > > > > > > > Should we provide this "beaster" in common-build? > > > > > > I would use it! It sounds like there is a lot of work involved in > > > making tests.iters work better with LuceneTestCase. In the mean > > > time, this sounds like a quick solution that might not be as > > > efficient (multiple JVMs), but still better than having to come up with a > bash script? > > > > > > On Fri, Aug 8, 2014 at 7:28 AM, Michael McCandless > > > <luc...@mikemccandless.com> wrote: > > > > +1, this sounds awesome? > > > > > > > > Mike McCandless > > > > > > > > http://blog.mikemccandless.com > > > > > > > > > > > > On Fri, Aug 8, 2014 at 10:06 AM, Uwe Schindler <u...@thetaphi.de> > > wrote: > > > >> Hi, > > > >> > > > >> We could emulate the same thing (the repeating beaster) with pure > > Ant: > > > >> > > > >> Just repeat the "test" target, which can be done using ant-contrib's > "for" > > > task or (much simplier) a groovy script using antcall on the test target. > > > >> Should we provide this "beaster" in common-build? > > > >> > > > >> "ant beast-tests -Dbeast.iter=100 -Dtestcase=..." > > > >> > > > >> Very easy to implement and makes it easier to use for the python > > > >> haters - > > > and comes embedded... > > > >> > > > >> Uwe > > > >> > > > >> ----- > > > >> Uwe Schindler > > > >> H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de > > > >> eMail: u...@thetaphi.de > > > >> > > > >> > > > >>> -----Original Message----- > > > >>> From: Michael McCandless [mailto:luc...@mikemccandless.com] > > > >>> Sent: Friday, August 08, 2014 3:48 PM > > > >>> To: Lucene/Solr dev > > > >>> Subject: Re: Test iterations > > > >>> > > > >>> On Fri, Aug 8, 2014 at 9:35 AM, Uwe Schindler <u...@thetaphi.de> > > > wrote: > > > >>> > Hi Dawid, > > > >>> > > > > >>> > Thanks for the very good explanation! Indeed the main problem > > > >>> > with > > > >>> tests.iters is the static initializers. Maybe put that > > > >>> explanation into the Wiki! I sometimes also need to remember it, > > > >>> so it should be > > > documented. > > > >>> > > > > >>> > One (only theoretical) way to solve the whole thing could be: > > > >>> > Load the class(es) in a separate classloader for every > > > >>> > repeated execution,... but of course this will very fast blow > > > >>> > up your permgen (java 6, 7) or anything else we don't know about > (java 8). > > > >>> > In fact the separate classloader approach is not different > > > >>> > from Mike's scripts, just that Mike's script creates a new > > > >>> > classloader by forking a new JVM. In fact I don't think the > > > >>> > separate classloader approach would be much faster, because > > > >>> > the class clones will all have separate compilation paths in > > > >>> > Hotspot, so Hotspot cannot share the same assembler code. So > > > >>> > except the JVM startup time, you gain nothing. Just permgen > > > >>> > issues :-) > > > >>> > > > >>> The big thing the python beasting scripts avoids is all the ant > > > >>> overhead to just get to the point where it actually spawns the > > > >>> JVM to run the test. Really, that's all the beasting script does: > > > >>> directly spawn the JVM on the test runner (after running "ant > > > >>> test-compile" up > > > >>> front) and then parse its output/events. > > > >>> > > > >>> The distributed test runner, which uses rsync/ssh to run tests > > > >>> on N machines, is very different from the beasting script: it > > > >>> runs all Lucene's tests (instead of a single test over and over) > > > >>> across N JVMs on M machines. It "cheats" by taking the union of > > > >>> all > > CLASSPATHs > > > ... > > > >>> but this is a huge win because it means all testing is fully > > > >>> concurrent, not just concurrent within one module. This script > > > >>> can also repeat, which means once all lucene tests finish, > > > >>> re-en-queue all > > of > > > them again. > > > >>> > > > >>> Mike McCandless > > > >>> > > > >>> http://blog.mikemccandless.com > > > >>> > > > >>> ---------------------------------------------------------------- > > > >>> ---- > > > >>> - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For > > > >>> additional commands, e-mail: dev-h...@lucene.apache.org > > > >> > > > >> > > > >> ----------------------------------------------------------------- > > > >> ---- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > > >> For additional commands, e-mail: dev-h...@lucene.apache.org > > > >> > > > > > > > > ------------------------------------------------------------------ > > > > --- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For > > > > additional commands, e-mail: dev-h...@lucene.apache.org > > > > > > > > > > -------------------------------------------------------------------- > > > - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For > > > additional commands, e-mail: dev-h...@lucene.apache.org > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For > > additional commands, e-mail: dev-h...@lucene.apache.org
beaster.patch
Description: Binary data
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
