My test suite at work was starting to get me down. It was taking
forever to run. Not just the whole suite mind you, but individual
scripts were taking several seconds just to start up.
As my application has grown, so has its library (including a lot of
interconnected Class::DBI modules), and this has lead to the long
startup times. The work-around (at least on the web side) has been to
run the app in a persistent environment such as mod_perl or
PersistentPerl.
After some playing around I've now also managed to get my test suite
running under PersistentPerl. This has dramatically increased the
performance of my test scripts.
For instance, in a typical directory containing 16 test scripts
(about 2000 lines of test code), here are the different execution
timings:
- without perperl ~ 70 seconds
- with perperl (first run) ~ 11 seconds
- with perperl (subsequent runs) ~ 4.5 seconds
And on a single script (about 50 lines of code):
- without perperl ~ 5 seconds
- with perperl (first run) ~ 5 seconds
- with perperl (subsequent runs) ~ 1 second
To "restart" PersistentPerl, you have to kill its backend processes,
e.g.:
$ killall perperl_backend
You have to do this if any code outside of the test script itself has
changed. However you *don't* have to restart PersistentPerl if only the
test script has changed.
So for the common case of fixing bugs in your test script and rerunning
it, you still get the fast startup time. But if you change something in
a module somewhere (such as to fix a bug that your test revealed), you
have to restart PersistentPerl again.
But where this system saves the most time is when you've finished with
your current round of test-driven development and you want to run the
entire suite to make sure that you haven't broken anything elsewhere. My
test suite used to take about 10 minutes to run. Now it takes about 2
minutes.
Because Test::Harness forks a new perl interpreter for each test script,
and because it bases its caching system on the name of the script,
there is a bit of scaffolding involved in this setup.
First I created a wrapper script which uses PersistentPerl to run a
single test script, but which always re-executes the script (via 'do'):
$ cat perperl-runscript
#!/usr/bin/perperl -- -M1
use strict;
use Test::PerPerlHelper; # see below
my $script = shift;
do $script or die $@;
(The '-M1' switch on the shbang line tells perperl to only spawn one
backend process.)
Next, I created a wrapper around prove to tell it to use
perperl-runscript as its perl interpreter:
$ cat perperl-prove
#!/bin/sh
export HARNESS_PERL=perperl-runscript
prove $*
There is also some code that has to be added to each test script in
order to get it to work properly under PersistentPerl:
* have to reset Test::Builder counter at the beginning of the script
* for scripts with no_plan, have to register a PersistentPerl cleanup
handler to display the final 1..X line
* need to prevent Test::Builder from duplicating STDOUT and STDERR,
since this seems to be incompatible with PersistentPerl
I've made a little module to do this:
use strict;
use warnings;
package Test::PerPerlHelper;
use Test::More ();
use Test::Builder ();
sub import {
my $class = shift;
$class->plan(@_);
}
sub plan {
my $class = shift;
if (@_) {
my $Test = Test::Builder->new;
if (eval {require PersistentPerl} && PersistentPerl->i_am_perperl) {
# Ugly hack to prevent Test::Builder
# calling $self->_dup_stdhandles
{
local $^C = 1;
$Test->reset;
}
Test::Builder::_autoflush(\*STDOUT);
Test::Builder::_autoflush(\*STDERR);
$Test->output(\*STDOUT);
$Test->failure_output(\*STDERR);
$Test->todo_output(\*STDOUT);
$Test->no_ending(1);
my $pp = PersistentPerl->new;
$pp->register_cleanup(sub {
$Test->_ending;
});
}
$Test->plan(@_);
}
}
1;
In a test script, you use it like this:
$ cat 01-test.t
use Test::More;
use Test::PerPerlHelper 'no_plan';
ok(1);
ok(2);
ok(3);
This should be compatible with regular (non-PersistentPerl) use as well.
The bit in Test::PerPerlHelper that sets $^C to prevent Test::Builder
from duplicating its filehandles is obviously ugly and wrong. But
PersistentPerl doesn't seem to like STDOUT and STDERR being redirected
(it hangs). Maybe Test::Builder could offer an option to disable this
feature? Or is there another solution?
Limitations and Caveats with the system:
* Scripts that muck about with STDIN, STDOUT or STDERR will probably
have problems.
* The usual persistent environment caveats apply: be careful with
redefined subs, global vars; 'require'd code only gets loaded on the
first request, etc.
* Test scripts have to end in a true value.
If there's interest, I'll try to package all this up into a CPAN module.
Michael
---
Michael Graham <[EMAIL PROTECTED]>
