tl;dr: TAP::Harness uses lots of RAM to store test results
Particularly lots and lots for millions of little "ok\n"s
It would be nice if it didn't use lots of RAM when storing things that mostly
pass.
So, the long(er) story. I use to be able to build and test blead on my
Raspberry Pi. Now it runs of memory (well, swap), somewhere near the end
of the tests, and fails to finish. So, I wondered why, and tried bisecting
the problem to see if it could be narrowed down to a particular change.
Running this:
export TEST_JOBS=1; bisect.pl -j1 --start
9f68b0f7acd1bcb04e7baa4bdb7cfec8e5d985c8 --end blead^ -Doptimize=-g
-Dusethreads -Duse64bitint -- sh -c 'cd t && ./perl harness; echo $?; ls -l
test_state'
*eventually* produced this:
HEAD is now at 3283393 Restrict the valid identifier syntax, fix some
identifier bugs.
bad - non-zero exit from sh -c cd t && ./perl harness; echo $?; ls -l test_state
32833930e32dc619abdaaab54e88de2a2765fb86 is the first bad commit
commit 32833930e32dc619abdaaab54e88de2a2765fb86
Author: Brian Fraser <frase...@gmail.com>
Date: Tue Mar 5 18:18:49 2013 -0300
Restrict the valid identifier syntax, fix some identifier bugs.
Fixes:
* Length-one identifiers are now restricted to
[\p{XIDS}\p{POSIX_Punct}\p{POSIX_Digit}\p{POSIX_Cntrl}]
plus, if under 'no utf8', the 128 non-ASCII characters in the
Latin1 range.
* Identifiers that start with ASCII letters can be followed with
XIDC characters
(The committer made some small edits in the pod)
:100644 100644 e8f5402a77dd011f065f7d91db354fdfd6da6830
8ac08abf50107cbfba3a8296df34556449134ba6 M gv.c
:040000 040000 9ae3a108394119259ca10d38ef44ef23c237fb91
3a55dc5feeea97da7346fc74df0601d3200e32e3 M pod
:040000 040000 695729610bd917237944558cfe729d529dfb6514
714c16587ae882fc635c17fd82362e5ff804f16b M t
:100644 100644 27485462ffca6939c895961092a743775401fa14
1ea0da7fde5aa50521ffe68378a87980048578cd M toke.c
bisect run success
That took 274968 seconds
(yes, that's just over 3 days).
If I reverse just the test cases added by that commit:
git show 32833930e32dc619abdaaab54e88de2a2765fb86^:t/uni/variables.t
>t/uni/variables.t
Test Summary Report
-------------------
uni/variables.t (Wstat: 0
Tests: 5 Failed: 0)
TODO passed: 5
Files=2388, Tests=620014, 18222 wallclock secs (2020.18 usr 55.35 sys +
13406.55 cusr 290.67 csys = 15772.75 CPU)
Result: PASS
whereas with that test restored:
../lib/charnames.t ................................................ ok
Could not execute (./perl -I.. -MTestInit=U1 ../lib/diagnostics.t): open3: fork
failed: Cannot allocate memory at ../lib/TAP/Parser/Iterator/Process.pm line
168.
make: *** [test_harness] Error 12
If I double the amount of swap:
All tests successful.
Files=2388, Tests=685775, 18699 wallclock secs (2216.11 usr 60.91 sys +
14095.49 cusr 300.45 csys = 16672.96 CPU)
Result: PASS
So, yes, it's to do with the amount of memory available. That test itself
is innocent. All it does wrong is to add a *lot* of testcases:
-plan (tests => 5);
+plan (tests => 65850);
Which makes me wonder - just how much memory is TAP::Parser using.
In particular, is TAP::Parser using the same amount of memory to store 65850
"ok"s as it would to store some mix of 65850 "ok"s and "not ok"s?
Which I'm starting to think, for large test suites, isn't that efficient.
Most tests pass most of the time. So is it possible for TAP::Parser to use
a more efficient format in memory to "archive" results for tests where
every single subtest was an "ok"?
Nicholas Clark
