On Tue, 20 Feb 2001 00:21:44 -0800 (PST), Richard Soderberg <[EMAIL PROTECTED]>
wrote:
> On Tue, 20 Feb 2001, H.Merijn Brand wrote:
>
> > On Mon, 19 Feb 2001 18:52:46 -0500 (EST), [EMAIL PROTECTED] wrote:
> > > This is a build failure report for perl from [EMAIL PROTECTED],
> > > generated with the help of perlbug 1.33 running under perl v5.7.0.
> > >
> > > Daily build and smoke test by smokingjacket v0.01.
> >
> > Is it wrong to presume that people/builders/testers/smokers building bleadperl
> > *ALSO* /read/ p5p?
>
> Not specifically, no.
As smokers is starting, we *do* indeed need another way to report the test
results. My mktest (http://home.hccnet.nl/h.m.brand/mktest) is a nice way to
start the tests and make summarized results, but we'd like to *not* get them in
the perlbug database, at least /not/ the way normal bugs are reported.
Do we
[ ] 1. Need another DB?
Don't think so, too much admin tasks
[ ] 2. Archive the results for later perusal
Probably
[ ] 3. Need a search mechanism for that?
Probably
[ ] 4. Need some meta-summary?
Think of something like: At .patch = 8873, the only failres
reported are 'db-*, op/each, numconvert and pragma' all other
tests pass on all received test results.
[ ] 5. Need a weekly/monthly digest?
Like Simon's doing for p5p and p6 (thank you)
> > Why is it that if Jarkko /explicitly/ tells that db-* fails with the latest
> > tarball that bug reports keep streaming in which report exactly what was
> > announced to fail?
>
> FWIW, smokingjacket rsyncs perl-current - unrelated to the tarball.
>
> > A PLEA TO ALL:
> >
> > *READ* what Jarkko writes as comments with the latest tarballs and *DO NOT*
> > report bugs that are announced, in order to prevent the perlbug database to
> > flood with duplicates that are *known* in advance.
>
> Unfortunately, I'm not aware of code that allows me to easily seperate
> known test failures from unexpected test failures barring some interesting
> modifications to the test suite.
>
> > At this time I do /not/ have time to mark them duplicate, but think about the
> > poor soul that has to detect those ... Yes, it's a lot of work, and it's not so
> > long ago that we did a cleanup with common effort. Let's keep the open bug
> > count as low as possible, which also motivates bug diggers to pick up open bugs
> > to fight.
>
> How, then, should we be doing daily build failures? It sounds as if
> perhaps the smoke test builds should not be going into the bugdb, but
> instead to the smokers(whatever it's called now, sorry) list. I'd be more
> than willing to move my apparently redundant bug reports there - I'll get
> to see them, interested parties who wish to see the burning parts of the
> tree can subscribe and watch, and I'll know within a day when that bug
> gets fixed.
>
> The issue here seems to be that as we automate more and more the process
> of reporting build failures, we're flooding the bugdb with lots of
> hard-to-seperate-out reports. Perhaps it's time to add a category to the
> bugdb to support these tests? I don't expect that automatic build testing
> is going to cease any time soon, but nor do I believe that each automatic
> build report should become an open bug.
>
> Suggestions?
>
> R.
--
H.Merijn Brand Amsterdam Perl Mongers (http://www.amsterdam.pm.org/)
using perl-5.005.03, 5.6.0, 5.6.1, 5.7.1 & 623 on HP-UX 10.20 & 11.00, AIX 4.2
AIX 4.3, WinNT 4, Win2K pro & WinCE 2.11 often with Tk800.022 &/| DBD-Unify
ftp://ftp.funet.fi/pub/languages/perl/CPAN/authors/id/H/HM/HMBRAND/
