> Did you change PHP version too? That's a PHP test, the result shouldn't > change unless you change PHP version too with 1.3.33?
ah, I had php installed for 1.3.32 but not 1.3.33 :) btw, the php stuff we've been doing is coming along quite well. you might be interested in the tarball chris and I are working on for apachecon http://www.modperlcookbook.org/~geoff/slides/nyphp/perl-php-test.tar.gz note you'll need to have current A-T cvs installed, as I didn't take any protection against missing versions, etc. > Welll... we started having this debate a while back :) indeed :) > > Here's my take: I think it's correct to: > > 1) only test for new features in versions on which they are known to be > present I think we agreed on that (eventually :) > > 2) test for bugs in all versions unconditionally in all affected > versions > > I think it's the desired outcome that if you test 1.3.32 for > CAN-2004-0940, it should fail: 1.3.32 is after all vulnerable to > CAN-2004-0940. Why hide that by skipping the test? Likewise, if you're > running 1.3.32 you *should* be told that there is a nasty mod_rewrite > regression in that version. > > Maybe I'm hawking my corporate agenda here a little too, because it > makes httpd-test slightly more useful to me since I can test for 1.3.x + > backported patch, whereas if the test was skipped for <1.3.33 it won't > demonstrate that the code is patched. > > Does that make sense? sure. what it really feels like is that we (as a community) need a new function of sorts. that is, skip just glosses over a failure, and todo is only forward looking (throwing unexpectedly succeeded warnings) - we need some kind of 'known issue' marker that understands an issue can never be fixed (unlike todo which assumes that it can be fixed in the future). but I guess that's another topic altogether :) so I guess I'm inclined to agree with your logic then - it's better to have regressions fail loudly and pique some interest than to just gloss over them, especially for security-type things. thanks for being patient with me while I caught up, then :) --Geoff
