On Tue, Apr 22, 2014 at 09:46:26AM -0700, Karen Etheridge wrote: > On Tue, Apr 22, 2014 at 12:51:02PM +0100, David Cantrell wrote: > > A 'FAIL' result doesn't mean "this code is broken". It doesn't even mean > > "this code is broken on this version of perl and this OS". It means "it > > didn't pass its tests on this particular setup". > > That's why in cpXXXan I don't pay any attention to failures at all. > Are you saying you only look for the presence of a PASS, not the absence of > a FAIL, when considering whether to index a distribution for a particular > cpXXXan?
Yup. Consider, for example, a report like this: http://www.cpantesters.org/cpan/report/03359184-b19f-3f77-b713-d32bba55d77f where the tests failed because they were run on a machine without enough memory available. It isn't possible to automatically and reliably categorise failure reports into valid and invalid^W^W^Wuseful and not useful for users, so I don't even try. But we can be pretty damned certain that *pass* reports are correct. -- David Cantrell | A machine for turning tea into grumpiness Compromise: n: lowering my standards so you can meet them
