I have tried the following test using the perl@8773 binary and the
Test::Harness.pm modules from: 0) the perl@8773 kit, 1) the perl@8585 kit,
2) the perl@6382 kit, 3) the perl 5.6.1 trial 2 kit. The result
is the same in all cases - 100% false ok reports.
Consider the following test script:
$ type test.pl
use strict;
use Test;
plan test => 5;
my $data = "fidlylyl dee\nHowdy!\nextra text\n";
my $reference = "Twas brillig and the\nslithey toves\ndid gyre and gimbal\nin the
wabe\n";
ok($data eq $reference,1,"# Expected: $reference\n# Got: $data\n");
ok($data eq $reference,0,"# Expected: $reference\n# Got: $data\n");
ok($data ne $reference,1,"# Expected: $reference\n# Got: $data\n");
ok($data ne $reference,0,"# Expected: $reference\n# Got: $data\n");
ok(eval{$data eq $reference},1,"# Expected: $reference\n# Got: $data\n");
__END__
If I run the output into a file like so:
$ perl test.pl > test.out
$ search test.out ok
not ok 1
not ok 2
ok 3
not ok 4
not ok 5
I obtain the expected result: one ok and four not ok's [*see also below].
If, however, I run the test as it would be run from a MakeMaker generated
descrip.mms file like so:
$ perl -e "use Test::Harness qw(&runtests $verbose); $verbose=0; runtests @ARGV;"
test.pl
test.p..............ok
All tests successful.
Files=1, Tests=5, 1 wallclock secs ( 0.00 cusr + 0.00 csys = 0.00 CPU)
The reason for the _apparent_ passage is the problem with record oriented
I/O that Craig Berry recently mentioned. For example if I switch to
$verbose=1 and look at the file:
$ perl -e "use Test::Harness qw(&runtests $verbose); $verbose=1; runtests @ARGV;"
test.pl > test_harn.out
FAILED--1 test script could be run, alas--no output ever seen
%SYSTEM-F-ABORT, abort
$ dir test_harn.out
TEST_HARN.OUT;2 TEST_HARN.OUT;1
$ sea TEST_HARN.OUT ok
not ok 1
not ok 2
ok 3
not ok 4
not ok 5
Note that I have two output files: the ;2 version is 2 blocks and the ;1
version is 1 block.
Note also the following misleading analogy to running
`perl test.pl | grep ok` on unix:
$ pipe perl test.pl | search sys$input ok
ok 1
ok 2
ok 3
ok 4
ok 5
all the "not\n" lines go missing because of the record oriented nature of
the PIPE command.
Recommendation: do not use Test::Harness with perl@8773, run module tests
scripts by hand until a patch can be worked out. Although I have noticed
that this problem exists on VMS for a wide variety of different Harness.pm
files I have not determined which version(s) of perl other than perl@8773
might be so affected.
Peter Prymmer
