On Tue, Jul 29, 2008 at 4:28 AM, James Keenan via RT
<[EMAIL PROTECTED]> wrote:
> I have prepared a patch which represents an 'svn diff' between trunk and
> the 'parallel' branch at the point of the branch's inception. Rather
> than swamp your inboxes, I'm posting it here:
>
> http://thenceforward.net/parrot/diff.trunk.parallel.txt
>
> This patch is more for review than application. Because the number of
> t/steps/*.t test files has been reduced from 227 to 74, there's been a
> tremendous amount of text movement. So it's not surprising that when I
> do the 'svn merge', I'll have plenty of conflicts to resolve. And I
> doubt that 'patch' will apply the patch cleanly.
>
> For those of you who are interested in benchmarking the difference, I
> recommend you do a checkout of the parallel branch and run 'perl
> Configure.pl --test=configure', as I've included a line reporting the
> elapsed time. Then do a fresh checkout from trunk, copy the version of
> lib/Parrot/Configure/Options/Test.pm found in 'parallel' into trunk and
> running Configure.pl there in the same way. Since, as I said in a
> previous post, there is considerable variance in the time Configure.pl
> takes, you should probably configure in each branch 3 times to get a
> good idea of any speed improvements.
>
i haven't reviewed the patch, but i have run the tests. here's the
results from both branches, synchronized to r29844.
trunk:
All tests successful, 16 tests skipped.
Files=287, Tests=3946, 145 wallclock secs ( 0.00 cusr + 0.00 csys = 0.00 CPU)
All tests successful, 16 tests skipped.
Files=287, Tests=3946, 140 wallclock secs ( 0.00 cusr + 0.00 csys = 0.00 CPU)
All tests successful, 16 tests skipped.
Files=287, Tests=3946, 142 wallclock secs ( 0.00 cusr + 0.00 csys = 0.00 CPU)
c:\usr\local\parrot\trunk>ack -af t\configure t\steps | wc -l
301
branches/parallel:
Files=133, Tests=3332, 76 wallclock secs ( 0.00 cusr + 0.00 csys = 0.00 CPU)
Failed 1/133 test programs. 1/3332 subtests failed.
Files=133, Tests=3332, 76 wallclock secs ( 0.00 cusr + 0.00 csys = 0.00 CPU)
Failed 1/133 test programs. 1/3332 subtests failed.
Files=133, Tests=3332, 75 wallclock secs ( 0.00 cusr + 0.00 csys = 0.00 CPU)
Failed 1/133 test programs. 1/3332 subtests failed.
c:\usr\local\parrot\parallel>ack -af t\configure t\steps | wc -l
147
the failing test:
t/steps/auto_ctags-01........................ok 1/31
# Failed test 'Got expected result'
# at t/steps/auto_ctags-01.t line 65.
t/steps/auto_ctags-01........................NOK 17/31# got: 'yes'
# expected: 'no'
# Looks like you failed 1 test of 31.
t/steps/auto_ctags-01........................dubious
Test returned status 1 (wstat 256, 0x100)
DIED. FAILED test 17
Failed 1/31 tests, 96.77% okay
the source looks like:
is($step->result(), q{no}, "Got expected result");
this test assumes i don't have ctags installed, but i do. of course,
configure knows better than any tester, so it's better to test that
the 'result' method returns one of the allowed values, and doesn't
assume one or the other. something like:
like($step->result(), qr/(yes|no)/, "Got allowed value from
'result' method");
i expect there may be similar tests of ->result() that need
modification, though i haven't verified that.
overall, i'm impressed at the 50% speedup. given that these are
relatively fast tests, it makes sense that since there are half as
many files (and therefore half as many invocations of perl), the tests
take half the time to run. other than my concern about the testing
strategy that led to my failing test, i think it's ready for trunk.
it seems that only files in t/steps/ have been refactored, and that
t/configure/ has been left alone. is that correct, and if so, can it
be similarly refactored? another 50% speedup would put these tests in
the 30-40s range, which is much nicer than the 2+ minutes it takes in
trunk today.
~jerry
