Hi, attached you will find the beaster:
- Only modifies common-build.xml, so no inherit down (makes no sense otherwise, as you would never run "ant beast-test" from top-level. So you have to go to correct submodule and run "ant beast-test -Dbeast.iters=n -Dtestcase=..." from there - Uses "antcall" in a loop, invoking the internal dependency-less "-test" target. My first impl used the test-macro directly, but this did not work, because test-macro sets non-local properties, which are then available on second round, causing errors or use always same seed. Antcall creates a new project each time and runs tests. I can open an issue or just commit this :-) Uwe ----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: [email protected] > -----Original Message----- > From: Uwe Schindler [mailto:[email protected]] > Sent: Friday, August 08, 2014 8:13 PM > To: [email protected] > Subject: RE: Test iterations > > Hi, > > I will look into that as a Groovy Skript: The main problem is: You cannot > simply > use <antcall/> in a loop, because this would also execute the dependencies > on each run. > > My idea is to do the following: > - maybe subclass antcall Task with Groovy (not sure if this is needed) > - instantiate it with current project > - execute dependent targets > - execute the inner target multiple times: store the project properties first > and restore them after execution. This is done, because ANT properties can > only be set *once*. If you don't give a fixed test seed, each run would pick a > new one (because the project properties are reset, so the seed from the > previous execution is gone). > > Uwe > > ----- > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: [email protected] > > > > -----Original Message----- > > From: Ryan Ernst [mailto:[email protected]] > > Sent: Friday, August 08, 2014 5:08 PM > > To: [email protected] > > Subject: Re: Test iterations > > > > Thanks for the extremely thorough answer, Dawid! Entertaining as > > always. :) > > > > > Should we provide this "beaster" in common-build? > > > > I would use it! It sounds like there is a lot of work involved in > > making tests.iters work better with LuceneTestCase. In the mean time, > > this sounds like a quick solution that might not be as efficient > > (multiple JVMs), but still better than having to come up with a bash script? > > > > On Fri, Aug 8, 2014 at 7:28 AM, Michael McCandless > > <[email protected]> wrote: > > > +1, this sounds awesome? > > > > > > Mike McCandless > > > > > > http://blog.mikemccandless.com > > > > > > > > > On Fri, Aug 8, 2014 at 10:06 AM, Uwe Schindler <[email protected]> > wrote: > > >> Hi, > > >> > > >> We could emulate the same thing (the repeating beaster) with pure > Ant: > > >> > > >> Just repeat the "test" target, which can be done using ant-contrib's > > >> "for" > > task or (much simplier) a groovy script using antcall on the test target. > > >> Should we provide this "beaster" in common-build? > > >> > > >> "ant beast-tests -Dbeast.iter=100 -Dtestcase=..." > > >> > > >> Very easy to implement and makes it easier to use for the python > > >> haters - > > and comes embedded... > > >> > > >> Uwe > > >> > > >> ----- > > >> Uwe Schindler > > >> H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de > > >> eMail: [email protected] > > >> > > >> > > >>> -----Original Message----- > > >>> From: Michael McCandless [mailto:[email protected]] > > >>> Sent: Friday, August 08, 2014 3:48 PM > > >>> To: Lucene/Solr dev > > >>> Subject: Re: Test iterations > > >>> > > >>> On Fri, Aug 8, 2014 at 9:35 AM, Uwe Schindler <[email protected]> > > wrote: > > >>> > Hi Dawid, > > >>> > > > >>> > Thanks for the very good explanation! Indeed the main problem with > > >>> tests.iters is the static initializers. Maybe put that explanation > > >>> into the Wiki! I sometimes also need to remember it, so it should be > > documented. > > >>> > > > >>> > One (only theoretical) way to solve the whole thing could be: > > >>> > Load the class(es) in a separate classloader for every repeated > > >>> > execution,... but of course this will very fast blow up your > > >>> > permgen (java 6, 7) or anything else we don't know about (java 8). > > >>> > In fact the separate classloader approach is not different from > > >>> > Mike's scripts, just that Mike's script creates a new classloader > > >>> > by forking a new JVM. In fact I don't think the separate > > >>> > classloader approach would be much faster, because the class > > >>> > clones will all have separate compilation paths in Hotspot, so > > >>> > Hotspot cannot share the same assembler code. So except the JVM > > >>> > startup time, you gain nothing. Just permgen issues :-) > > >>> > > >>> The big thing the python beasting scripts avoids is all the ant > > >>> overhead to just get to the point where it actually spawns the JVM > > >>> to run the test. Really, that's all the beasting script does: > > >>> directly spawn the JVM on the test runner (after running "ant > > >>> test-compile" up > > >>> front) and then parse its output/events. > > >>> > > >>> The distributed test runner, which uses rsync/ssh to run tests on N > > >>> machines, is very different from the beasting script: it runs all > > >>> Lucene's tests (instead of a single test over and over) across N > > >>> JVMs on M machines. It "cheats" by taking the union of all > CLASSPATHs > > ... > > >>> but this is a huge win because it means all testing is fully > > >>> concurrent, not just concurrent within one module. This script can > > >>> also repeat, which means once all lucene tests finish, re-en-queue all > of > > them again. > > >>> > > >>> Mike McCandless > > >>> > > >>> http://blog.mikemccandless.com > > >>> > > >>> -------------------------------------------------------------------- > > >>> - To unsubscribe, e-mail: [email protected] For > > >>> additional commands, e-mail: [email protected] > > >> > > >> > > >> --------------------------------------------------------------------- > > >> To unsubscribe, e-mail: [email protected] For > > >> additional commands, e-mail: [email protected] > > >> > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [email protected] For > > > additional commands, e-mail: [email protected] > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] For additional > > commands, e-mail: [email protected] > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected]
beaster.patch
Description: Binary data
--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
