On Sat, 23 Feb 2008, James Keenan via RT wrote: > The patch attached, diff.trunk.tcif.txt, addresses both RT 47391 > (t/configure/1*.t tests incorrectly rely on init::defaults_ and RT 47503 > (Remove config::init::defaults From configure tests). It accomplishes > the following:
Thanks for working on this. Alas, it didn't work. I don't have any more
time to debug this any further, but I'll share some observations I had:
> * In each test, the configuration step is first run just as it will be
> by Configure.pl. The results of that step are recorded on disk. The
> test file then reads from the disk the results of all previous
> configuration steps and uses that in tests for the step currently being
> tested. Thus, for the testing of a given configuration step, individual
> tests have available to them all the information found in the
> Parrot::Configure object at that same point during regular configuration.
This sounds to me as if it assumes all the tests will be running in order
every time. For example, suppose I have a test failure with test number
200. Does that mean I have to rerun all 199 preceding tests every time I
wish to change something in test 200? Conversely, suppose all 200 tests
run, but I want to go back and investigate a failure in test 10. The
stored results will reflect all 200 tests, so they won't be the same the
next time I run test 10. Will that be an issue?
More generally: Where is this cache stored? How do we know if it's
still valid or not? What invalidates this cache?
> * The harness which runs pre- and post-configuration tests,
> Parrot::Configuration::Options::Test, has been revised to require an
> up-to-date version of the 'prove' utility (from TAP::Parser 3.05 or
> higher) so that options passed to Configure.pl on the command line can
> be passed through to the test files in t/steps/.
This is a problem too. Since Test::Harness 3.* changed the default
output format, installing it has wider implications than I want to think
about at the moment. Unless I am mistaken, the following lines added to
Configure::Options::Test will cause Configure.pl to always fail unless
the user has TAP::Harness installed. This means Test::Harness 3.* would
now be required for everyone.
+BEGIN {
+ eval { use TAP::Harness 3.05 (); };
+ if ($@) {
+ print STDERR "[EMAIL PROTECTED] Must have modern 'prove' tests
using Parrot::Configure::Options::Test\n";
+ exit;
+ }
+}
> Please review. This patch is necessarily large and is likely to give
> the 'patch' program fits. But you can play in a sandbox for the SVN
> branch in which I developed this approach. To see
> configuration-testing-in-parallel in action, call 'svn co
> https://svn.perl.org/branches/tcif/' and run 'perl Configure.pl --test'
> with whatever command-line options you customarily use when configuring
> in trunk.
Ok. I set up a fresh perl installation into which I could install
TAP::Harness without messing up anything else, checked out a fresh copy
of parrot, and applied your patch. Apart from some irrelevant issues
with MANIFEST, MANIFEST.skip and some languages stuff, it all applied
just fine.
However, actually trying it out, I get *lots* and *lots* of errors on
my Solaris system. I've attached the compressed output (uncompressed
it's over 80k!) of
Configure.pl --cc=gcc --link=gcc --ld=gcc --cxx=g++ --test
The first major failure is here:
t/steps/inter_progs-01.........................Can't call method "data" on an
undefined value at (eval 20) line 7.
# Looks like you planned 11 tests but only ran 2.
# Looks like your test died just after 2.
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 9/11 subtests
t/steps/inter_progs-02.........................Can't call method "data" on an
undefined value at (eval 20) line 7.
# Looks like you planned 11 tests but only ran 2.
# Looks like your test died just after 2.
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 9/11 subtests
[etc for the rest of inter_progs-??.t]
Alas, I can't see how to narrow this down further, since I can't see how
to use prove -v on individual tests that are expecting to somehow get
the original Configure.pl command line, and the current output is
unhelpful anyway. I tried adding print commands to the test, but all
the problems are occurring inside a
capture( sub {...} )
construct, so it's annoying to try to get information out in any useful
way.
$ prove --version
TAP::Harness v3.09 and Perl v5.10.0
$ prove -v t/steps/inter_progs-01.t
t/steps/inter_progs-01......
1..11
ok 1 - use config::inter::progs;
ok 2 - Parallel::Trace constructor succeeded
not ok 3 - Step stored
# Failed test 'Step stored'
# at t/steps/inter_progs-01.t line 21.
# got: '2'
# expected: '1'
ok 4 - inter::progs constructor returned defined value
ok 5 - The object isa inter::progs
ok 6 - inter::progs has description
ok 7 - Tie::Filehandle::Preempt::Stdin->can('READLINE')
ok 8 - The object isa Tie::Filehandle::Preempt::Stdin
# Looks like you planned 11 tests but only ran 8.
# Looks like you failed 1 test of 8 run.
# Looks like your test died just after 8.
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 4/11 subtests
Test Summary Report
-------------------
t/steps/inter_progs-01.t (Wstat: 65280 Tests: 8 Failed: 1)
Failed test: 3
Non-zero exit status: 255
Parse errors: Bad plan. You planned 11 tests but ran 8.
Files=1, Tests=8, 3 wallclock secs ( 0.15 usr 0.01 sys + 1.39 cusr 0.22
csys = 1.77 CPU)
Result: FAIL
There are many, many, many other failures. Many are probably related to
this failure to get the right compiler.
There are also many strange warnings:
t/steps/inter_make-01..........................Use of uninitialized value
$make_c in substitution (s///) at config/inter/make.pm line 109.
Use of uninitialized value $make_c in substitution (s///) at
config/inter/make.pm line 109.
ok
t/steps/inter_make-02..........................Use of uninitialized value
$make_c in substitution (s///) at config/inter/make.pm line 109.
# Failed test 'Step stored; has previously been tested'
# at t/steps/inter_make-02.t line 18.
# got: '1'
# expected: '2'
# Looks like you failed 1 test of 9.
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/9 subtests
I don't know what's happening here. I don't know why it's complaining
about an uninitialized variable. I also don't know what the complaint
about 'Step stored' means. This was the first time run on a clean
directory tree. I don't know what the got: '1' expected '2' actually
means.
I also can't figure out what's happening to the command line. Changing
the command line to give a bogus C compiler doesn't seem to actually
change anything:
Configure.pl --cc=none --link=none --ld=none --cxx=none --test
surely ought to run into problems, but doesn't seem any worse!
Anyway, here is the test "Summary" report. Alas, with T::H: 3, it's not
a terribly compact report, and there's no actual summary at the end, but
here it is anyway. The full output is attached.
Test Summary Report
-------------------
t/steps/inter_progs-01.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 11 tests but ran 2.
t/steps/inter_progs-02.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 11 tests but ran 2.
t/steps/inter_progs-03.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 11 tests but ran 2.
t/steps/inter_progs-04.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 10 tests but ran 2.
t/steps/inter_make-02.t (Wstat: 256 Tests: 9 Failed: 1)
Failed test: 3
Non-zero exit status: 1
t/steps/auto_gcc-01.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 9 tests but ran 2.
t/steps/auto_gcc-02.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 9 tests but ran 2.
t/steps/auto_gcc-03.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 10 tests but ran 2.
t/steps/auto_gcc-04.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 12 tests but ran 2.
t/steps/auto_gcc-05.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 10 tests but ran 2.
t/steps/auto_gcc-06.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 12 tests but ran 2.
t/steps/auto_gcc-07.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 13 tests but ran 2.
t/steps/auto_gcc-08.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 12 tests but ran 2.
t/steps/auto_gcc-09.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 13 tests but ran 2.
t/steps/auto_gcc-10.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 11 tests but ran 2.
t/steps/auto_gcc-11.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 11 tests but ran 2.
t/steps/auto_gcc-12.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 12 tests but ran 2.
t/steps/auto_gcc-13.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 13 tests but ran 2.
t/steps/auto_backtrace-02.t (Wstat: 256 Tests: 12 Failed: 1)
Failed test: 3
Non-zero exit status: 1
t/steps/auto_msvc-01.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 8 tests but ran 2.
t/steps/auto_msvc-02.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 10 tests but ran 2.
t/steps/auto_msvc-03.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 11 tests but ran 2.
t/steps/auto_msvc-04.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 24 tests but ran 2.
t/steps/auto_warnings-02.t (Wstat: 256 Tests: 9 Failed: 1)
Failed test: 3
Non-zero exit status: 1
t/steps/auto_alignptrs-01.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 9 tests but ran 2.
t/steps/auto_alignptrs-02.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 9 tests but ran 2.
t/steps/auto_alignptrs-03.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 9 tests but ran 2.
t/steps/auto_alignptrs-04.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 10 tests but ran 2.
t/steps/auto_alignptrs-05.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 8 tests but ran 2.
t/steps/auto_alignptrs-06.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 8 tests but ran 2.
t/steps/auto_alignptrs-07.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 10 tests but ran 2.
t/steps/auto_headers-02.t (Wstat: 256 Tests: 10 Failed: 1)
Failed test: 3
Non-zero exit status: 1
t/steps/auto_sizes-01.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 9 tests but ran 2.
t/steps/auto_sizes-02.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 29 tests but ran 2.
t/steps/auto_byteorder-01.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 11 tests but ran 2.
t/steps/auto_byteorder-02.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 15 tests but ran 2.
t/steps/auto_byteorder-03.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 8 tests but ran 2.
t/steps/auto_va_ptr-01.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 4 tests but ran 2.
t/steps/auto_format-01.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 13 tests but ran 2.
t/steps/auto_format-02.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 12 tests but ran 2.
t/steps/auto_isreg-02.t (Wstat: 256 Tests: 10 Failed: 1)
Failed test: 3
Non-zero exit status: 1
t/steps/auto_isreg-03.t (Wstat: 256 Tests: 10 Failed: 1)
Failed test: 3
Non-zero exit status: 1
t/steps/auto_arch-02.t (Wstat: 256 Tests: 9 Failed: 1)
Failed test: 3
Non-zero exit status: 1
t/steps/auto_arch-03.t (Wstat: 256 Tests: 11 Failed: 1)
Failed test: 3
Non-zero exit status: 1
t/steps/auto_snprintf-01.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 7 tests but ran 2.
t/steps/auto_snprintf-02.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 12 tests but ran 2.
t/steps/auto_snprintf-03.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 9 tests but ran 2.
t/steps/auto_perldoc-02.t (Wstat: 256 Tests: 22 Failed: 1)
Failed test: 3
Non-zero exit status: 1
Files=261, Tests=2680, 448 wallclock secs (12.22 usr 3.23 sys + 349.94 cusr
57.21 csys = 422.60 CPU)
Result: FAIL
Pre-configuration tests did not complete successfully; Configure.pl will not
continue. at lib/Parrot/Configure/Options/Test.pm line 167.
--
Andy Dougherty [EMAIL PROTECTED]
configure-test.out.bz2
Description: Output of perl Configure.pl --test
