Working on Parrot, I frequently do coverage analysis of the tests I and
others have written for Parrot's configuration and build tools,
components of the project which are written in Perl 5. I display the
results on my server:
http://thenceforward.net/parrot/coverage/configure-build/coverage.html
Because these tests are not set up quite like tests in CPAN modules, I
have to get a bit fancy in how I use Devel::Cover. Here's the shell
function I typically use:
# coverconf: configure-build coverage
coverconf ()
{
cd $RECONFDIR && \
cover -delete $COVERAGEDIR && \
PERL5OPT=-MDevel::Cover=-db,$COVERAGEDIR/ \
perl $RECONFDIR/Configure.pl --test --configure_trace "$@" && \
cover $COVERAGEDIR \
-ignore_re 'Configure\.pl' \
-ignore_re '^t\/' \
-ignore_re '^\/usr\/local\/bin' \
-ignore_re '^lib\/(?!Parrot)' && \
chmod 0755 $COVERAGEDIR/
}
The --test option to Configure.pl runs a series of tests before
Configure.pl itself runs, then runs more tests afterwards.
The problem I encounter is that, about 30% of the time, I will get test
files, that will complete with a message like this:
t/configure/029-option_or_data....................dubious
Test returned status 0 (wstat 11, 0xb)
after all the subtests completed successfully
All the individual tests within this file complete successfully. The
tests in this file are very similar to those in other test files which
complete without generating this error message. And when I run the
tests without Devel::Cover, they all pass with flying colors.
The error message occurs under some circumstances, such as above, but
not under other, very similar circumstances. Example:
PERL5OPT=-MDevel::Cover=-db,$COVERAGEDIR/ \
prove t/configure/*.t "$@"
In fact, when I run the command above, file '029' completes without an
error message ... but a later test file now starts to generate the error
message!
t/configure/104-init_miniparrot...................dubious
Test returned status 0 (wstat 11, 0xb)
after all the subtests completed successfully
When one of these error messages is generated, the shell function above
terminates at one of the '&&' marks and my lovely web pages displaying
coverage analysis fail to be regenerated.
I have sometimes been able to get the files displaying the error
messages to cease doing so by code voodoo, e.g., opening up the file,
throwing in a few more newlines somewhere, then saving the file. But
that only works sometimes.
So, can anyone explain why a test file, all of whose individual tests
repeatedly pass, cause this "Test returned status 0" error message?
Thank you very much.
Jim Keenan
