* Andreas Koenig <andreas.koenig.7os6v...@franz.ak.mind.de> [2013-01-02T15:20:00] > MPR <mplistarch...@gmail.com> writes: > > > > You say "usually". Is there a case where it might be useful to test > > and send reports against an old version? > > When a test passes with perl 5.14 and fails with 5.16, then we will > probably be interested in the results with 5.15 (e.g. > http://matrix.cpantesters.org/?dist=Attribute-Util%201.06 )
I'm not sure that this is really a good example. The question is whether anyone should be *still* testing and sending reports against old dev versions. So, the way this would be useful is if: 1. we have known passes for 5.14 2. we have known failures for 5.16 3. we have /no existing reports/ for 5.15 4. it's useful to know what commit changed things from pass to fail The first three conditions need to be true, *and* somebody needs to be smoking *both* the "still works" earlier 5.15.x *and* the "now fails" 5.15.(x+1), and that still only gets you to a 30 day window of commits to test. It seems like a lot of noise with only a very, very small chance of producing signal. Meanwhile, we have git-bisect to quickly get to the actual commit! I'm delighted to have those smoke reports rolling in during 5.15, though, so we CPAN authors can see what 5.16 might break. Once 5.16 is out the door, though, it hardly seems to have any value. -- rjbs
signature.asc
Description: Digital signature
