For certain modifications to the Parrot project's smoke testing
mechanism, I need to create an archive from an aggregation of test
harnesses and then send that archive to our smoke server.
Currently, we create an archive from just a single run of a given
testing target. So we have a 'make smolder' target which archives the
same set of tests whose results 'make test' sends to the console.
But we also have a 'make fulltest' target which runs a series of
individual 'make' targets and reports their summaries to the console one
at a time. Each of these targets can take different option which modify
%ENV. We want to be able to print to the console what appears to be one
big test run with a single summary, i.e., we want to aggregate the
results of individual harness runs.
The following code succeeds with respect to this objective:
$formatter = TAP::Formatter::Console->new( {
verbosity => $ENV{HARNESS_VERBOSE},
jobs => $ENV{TEST_JOBS} || 1,
directives => 1,
timer => $ENV{HARNESS_TIMER} || 0,
} );
$aggregator = TAP::Parser::Aggregator->new;
$aggregator->start();
foreach my $set (@targets) {
# rewrite environment
&{$set->{rule}} if defined $set->{rule};
print STDERR "$set->{label}: running with: $ENV{TEST_PROG_ARGS}\n";
my $harness = TAP::Harness->new( { formatter => $formatter } );
$harness->aggregate_tests($aggregator, @{$set->{tests}});
}
$aggregator->stop();
$formatter->summary($aggregator);
my $exit_value = $aggregator->failed ? 1 : 0;
exit($exit_value);
However, we don't simply want to print this information to the console.
When our harness is passed '--archive --send-to-smolder', we want the
*aggregated* test output to be archived and transmitted to our smoke
server. Unfortunately, I have not figured out how to make
TAP::Harness::Archive do this. Here is the code I have so far:
my %env_data = collect_test_environment_data();
my $harness = TAP::Harness::Archive->new( {
verbosity => $ENV{HARNESS_VERBOSE},
archive => 'parrot_test_run.tar.gz',
merge => 1,
jobs => $ENV{TEST_JOBS} || 1,
extra_properties => \%env_data,
extra_files => [ 'myconfig', 'config_lib.pir' ],
} );
$aggregator = TAP::Parser::Aggregator->new;
$aggregator->start();
foreach my $set (@targets) {
# rewrite environment
&{$set->{rule}} if defined $set->{rule};
$harness->aggregate_tests($aggregator, @{$set->{tests}});
}
$aggregator->stop();
$harness->summary($aggregator);
send_archive_to_smolder(%env_data) if $smolder;
Everything seems fine until I get to the very last line. The subroutine
send_archive_to_smolder() (which essentially wraps around an
LWP::UserAgent call) fails because the archive file
'parrot_test_run.tar.gz' has not been created. My impression is that
the use of TAP::Parser::Aggregator has somehow superseded the archiving.
Has anyone tried to combine these two modules with good results?
Thank you very much.
Jim Keenan
