I recognize that the CPAN testing architecture is under heavy load but there's something non-deterministic about its behavior that my poor brain just doesn't grasp. I'm never sure how long one should wait for results to show up, and because I'm waiting on a recent submission, I decided to perform a really simple study.
I've probably repeated below what others already know; it eases my frustration a bit to look at some numbers rather than just repeatedly reloading web pages. I submitted version 0.001_01 of Math::Rational::Approx on 15 April. Very few tests showed up on cpantesters.org. My initial hypotheses was that there's some blockage, so I compared the numbers with a module which is going through rapid development and thus pokes at the system fairly often, namely Type::Tiny. I've compared the log.txt views with the cpantesters.org pages. I'm assuming that tail.txt is an accurate measure of the testing activity. I recognize that's not correct in the long term (it shows fewer tests for version 0.001 of MRA which was released last September than appear on cpantesters.org) but over a few days I hope it will suffice. For the log.txt data I counted PASS and FAIL rows from pages with URL's like the following http://217.199.168.174/cgi-bin/cpantestersmatrix.pl?dist=Math-Rational-Approx%200.01_01;reports=1 I retrieved all of the data sometime around April 20, 14:00 GMT. The current CPAN testers page for Math::Rational::Approx has a time tag of Wednesday, 17th April 2013 (7:23pm). Here are the results (in #pass/#fail) for Math::Rational::Approx | Version | Date | CPAN Testers | tail.txt | | 0.01_01 | 15 April | 3/0 | 55/2 | tail.txt contains 3 tests which were submitted after 17 April, the time tag on the CPAN testers page. The CPAN testers page for Type::Tiny is tagged as Saturday, 20th April 2013 (1:21pm). Here are the results (in #pass/#fail) for Type::Tiny | Version | Date | CPAN Testers | tail.txt | | 0.003_05 | 19 April | 0/0 | 35/0 | | 0.003_04 | 18 April | 16/0 | 75/0 | | 0.003_03 | 17 April | 54/0 | 86/0 | | 0.003_02 | 16 April | 81/0 | 123/0 | | 0.003_01 | 16 April | 75/0 | 132/0 | | 0.001 | 15 April | 96/0 | 184/0 | A few things pop out: 1 The CPAN testers page for MRA seems stuck. Type::Tiny's is up-to-date. 2 Even though the CPAN testers page for Type::Tiny has an up-to-date time tag, it is missing lots of old reports. I expected that "backlogs" would be cleared out in FIFO fashion, but that doesn't seem to be the case, else * #2 wouldn't be the case. * MRA (which was submitted prior to Type:Tiny 0.003_01) would have more tests on CPAN testers Part of the (my?) problem is that I don't know how long to wait for accurate results to percolate to cpantesters.org before starting to worry. Is 5 days too long? too short? Is there some means of providing an indication of how caught up the processing is, perhaps by noting the last date at 100%? The latest CPAN Tester's summary indicates disk i/o is the bottleneck for digesting reports; does this mean that AWS is no longer part of the problem? Would throwing money at new hardware help? In any case, thanks for all of the good work. Diab
