On Sun, Sep 19, 2010 at 10:23:16AM +0900, Taro Nishino wrote: > Dear Tim, > > I'm sorry I didn't write back sooner.
Hello Taro. Your reply was very quick, thanks. > On Sat, 18 Sep 2010 11:54:20 +0100, Tim Bunce <tim.bu...@pobox.com> wrote: > > > > Devel-NYTProf-4.05: > > > - MSWin32-x86-multi-thread / 5.8.8: > > > - FAIL > > > http://www.cpantesters.org/cpan/report/09e91bd1-6c98-1014-b60a-40f6a66d8bb6 > > > > Thank you for testing Devel-NYTProf-4.05! > > > > I'm puzzled by the test failures though. They don't seem to be consistent. > > > > Are the failures repeatable? > > Is there sufficient disk space? > > Is there sufficient memory? > The failures aren't repeatable in the least. They strongly depend upon > much of situations when the tests are executed under at least my > circumstance. In fact, on inspection, it is almost impossible for the > tests to pass at a stretch by means of "make test", much less using > CPAN.pm. In short, you can't count on the CPAN Testers reports. Are you doing automated smoke testing or reporting manual testing? And are you using a virtual machine? > So, I executed the dubious part of the tests separately by using "prove" > as follows: > > C:\Perl\cpan\build\Devel-NYTProf-4.05-oF8sEy>prove t/test0*.t [...] > All tests successful. > C:\Perl\cpan\build\Devel-NYTProf-4.05-oF8sEy>prove t/test1*.t [...] > All tests successful. [...] > > As you can see, all the dubious parts pass. Are you saying that the tests pass when run manually using "prove", but fail when run using "make test"? > I'm sorry to have caused you so much trouble. No trouble at all Taro. I'm just trying to understand if there's a problem with NYTProf or something else, and what the causes are. > > Is there sufficient disk space? > > Is there sufficient memory? > > The disk space is sufficient, but the memory has no more than 500M. > While I'm not sure it's a cause of the trouble above, I feel like the > tests themselves are too sensitive to be executed under the 500M memory. > IMHO the test in general should be executed as small memory as possible. The way NYTProf tests work does run an extra perl process. Some of the tests that failed use very little memory themselves (e.g., t/test13.p) and certainly less that some that passed. 500M sounds like a reasonable amount of memory but perhaps that extra perl process, plus the "make" process, is what's pushing the memory usage too high. But maybe memory is not the problem, or not the only one. Two of the five failing tests start with "Failed to open output '...': No error". That open happens very early in NYTProf, before the perl code being profiled has even compiled. There are also several instances of "Unable to open ... for writing: Permission denied" plus one "rename(...): Permission denied". All very odd. > Finally, to be honest, I'm losing confidence in all my reports sent out > before. Questions for cpantesters-discuss: - Are there any guidelines for minimum available memory for cpan-testing? - Could the ENVIRONMENT AND OTHER CONTEXT section show available memory? (I presume there's some command to find it on windows.) - Any thoughts about possible causes of the failures in this report? Tim.
