On Mon, Sep 28, 2009 at 5:01 PM, Maciej Stachowiak <m...@apple.com> wrote:
> > On Sep 28, 2009, at 4:47 PM, David Levin wrote: > > I don't believe that the test was checked in a flaky state. It was solid > for a long time and then something happened... > > > What's "the test" in this context? The network / credentials one? > Yep. > > > I'll try to add more logging to this test this evening (after my turn at > helping chromium stay up to date with WebKit is over). > > I'll ping Drew about the other test. > > > It sounds like that test was always buggy, if Drew is right about the > cause. > > - Macie > > > Dave > > > On Mon, Sep 28, 2009 at 4:40 PM, Eric Seidel <e...@webkit.org> wrote: > >> On Fri, Sep 25, 2009 at 2:42 PM, Maciej Stachowiak <m...@apple.com> wrote: >> > I like Dave Levin's idea that the first action should be to instrument >> the >> > tests so we can find out why they intermittently fail. Especially if the >> > failure is reproducible on the bots but not on developer systems. >> >> I like this idea too. I don't like the reality that flakey tests are >> a burden on all developers caused by one. >> >> > Using the >> > skip list should be a last resort, because that hides the failure >> instead of >> > helping us diangose the cause. >> >> I (respectfully) disagree. I think we shouldn't be so afraid to skip >> tests. We don't allow people to check in compiles which fail. We >> don't allow people to check in tests which fail on other platforms >> (without skipping them) or on every other run. Why should we allow >> people to check in tests which fail every 10 runs? Or worse, why >> should we leave a known flakey test checked in/un-attended which fails >> every 10 runs? >> >> If we can't easily roll-out the failing tests (or the commit which >> cause them to start failing), we should skip them to keep the bots (a >> shared resource) green, so as not to block other work on the project. >> No? >> >> I very much like WebKit's "everyone is responsible for the whole >> project" culture, but I disagree that the burden of diagnosis should >> be on the person trying to make a completely unrelated checkin (as is >> the case when we leave flakey tests enabled in the tree). >> >> -eric >> >> p.s. I now have two "skipping flakey tests" changes up for review: >> https://bugs.webkit.org/show_bug.cgi?id=29322 >> https://bugs.webkit.org/show_bug.cgi?id=29344 >> _______________________________________________ >> webkit-dev mailing list >> webkit-dev@lists.webkit.org >> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev >> > > >
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev