On Sat, Sep 16, 2006 at 04:36:50PM -0500, Michael G Schwern wrote: > The tenor of the opinions in the passing TODO tests bother me a bit. It > seems folks have forgotten why the CPAN installation chain exists. > Especially the assertion that its perfectly ok for modules to start failing > even though there's nothing wrong with them (unless you reeeeaally stretch > things). The install chain is a special place and very conservative, often > radically so. >
I'm sure hoping that some of the things that I've been planning on in the Perl 5 core have not been interpreted incorrectly. I've been planning for quite some time to add additional TODO tests for bugs currently in the Perl 5 bug queue. We've even altered the Test::Harness to report on passing TODO tests. This, however, is of primary importance for the developers and not for the typical user. When you have a bug queue the size of the Perl 5 bug queue, you are bound to have multiple tickets related to a single problem. Possibly a bug will manifest itself with multiple symptoms. The primary purpose of adding the TODO tests are to alert the developers when a bug is coincidentally fixed. Smoke testing will discover the passing TODO's to alert developers that another bug has been fixed. It should not, however, cause any alarm to the typical user compiling, testing, and installing Perl. It shouldn't cause additional bug reports for simply confirming whether a specific bug exists or not. The other purpose of TODO's is to provide a ready made test case when the bug is fixed. If you are following one of the many test driven development models, when you have a bug, the first step is to write a test case. Using TAP, however, we have a way to capture the test case without causing the tests to fail for the typical user. The bug fix can come at a different time, or even from a different developer than the one that wrote the test case. This is ideal for an open source where developers have differing skill sets and motivations regarding bug fixes and testing. So, if you as a developer want to cause a TODO test case to behave like a normal test case, there is a simple answer. Don't make the test case be a TODO. The choice is up to the developer. It should never be up to the user. This is expected as part of the TAP protocol and would have the possiblity of effected many unsuspecting developers and users. Steve Peters [EMAIL PROTECTED]
