Hi Ovid - yes, we use a separate .t script per test class. This creates a more traditional TAP output (one stream per class) which works better with Smolder etc, and avoids the caveats with running potentially conflicting tests in the same process. The chances of one test interfering with another in a mysterious way explode if we run all 200+ in the same process. We don't have a magic reset_singletons. :)
On Nov 6, 2012, at 12:21 AM, Ovid wrote: > Hi Jonathan, > > I have just one question. You wrote that you're using Test::Class and "many > of the tests start by loading a bunch of the same modules". That confuses me > as Test::Class was originally designed to speed up test suites and did so by > loading everything *once* in the same process. > > Are you using a separate .t script per test class? That would cause the > reloading. Otherwise, a single .t script loading all of your test classes > (Test::Class::Load helps) should help you load all classes at once. Assuming > they have a sane setup and all call their own test control methods > (startup/setup/teardown/shutdown), you could drop something like this in your > base class: > > # see also http://www.slideshare.net/Ovid/a-whirlwind-tour-of-testclass > sub setup : Tests(setup) { > my $test = shift; > $test->reset_singletons; > } > > Yes, it's a hack to work around the fact that you have a bunch of singletons > with mutable state, but it seems like this would be much easier (though > perhaps less fun :) than your preforking solution. > > Cheers, > Ovid > -- > Twitter - http://twitter.com/OvidPerl/ > Buy my book - http://bit.ly/beginning_perl > Buy my other book - http://www.oreilly.com/catalog/perlhks/ > Live and work overseas - http://www.overseas-exile.com/ > From: Jonathan Swartz <swa...@pobox.com> > To: perl-qa <perl-qa@perl.org> > Sent: Tuesday, 6 November 2012, 2:03 > Subject: preforking prove > > We have a large slow test suite at work (Test::Class, 225 classes, about 45 > minutes run time). Many of the tests start by loading a bunch of the same > modules. Obviously we could speed things up if we could share that loading > cost. > > I'm aware of Test::Aggregate and Test::Aggregate::Nested, but a number of our > tests run into the caveats (e.g. they change singletons -- yes, this is not > ideal, but also not easily changeable). > > I thought of an alternative to Test::Aggregate called "preforking prove", or > pfprove, which does the following: > > * Accepts the same arguments as prove > * Preloads the module(s) in -M > * Launches a Starman server in the background > * For each test file, makes a request to the Starman server. The Starman > child runs the test and exits. > > The idea is that you preload all the common stuff in the Starman parent, so > that forking each child and running each test is fast (at least on Linux). > > Potential advantages over Test::Aggregate: > * runs one test per process, so avoids some caveats > * keeps the TAP in traditional form (one TAP stream per file) > * works well with parallel prove > > Potential disadvantages: > * lots of extra complexity (requires Starman or similar, need to make sure it > shuts down, need to handle errors, etc.) > > Curious what you think. Is there something like this out there already? > Potential problems? > > If I do this I might call it Test::Aggregate::Preforking, just to keep it in > the same category. > > Thanks! > Jon > > >
