I used to agree as well, but my opinion is now more nuanced. I've experienced projects where a test keeps failing day after day, and after a while developers stop looking at the test results with the same level of discipline.
Perhaps Sling is small enough (and the developers are pro-active enough) that this isn't an issue. But it certainly is on some other, larger, more disperse projects (such as CQ). In those, moving a failing test to an issue (which can be assigned to an individual) can produce better results than everyone simply getting used to the build being red. Cheers, Jeff. > -----Original Message----- > From: Carsten Ziegeler [mailto:cziege...@apache.org] > Sent: 03 June 2013 07:01 > To: dev@sling.apache.org > Subject: Re: Disabling flaky tests > > I agree as well, especially for the error handling as this is partially not > a problem of the test but really a bug in Sling - we have an issue for > that, it just needs to be done :) > > Carsten > > > 2013/6/3 Felix Meschberger <fmesc...@adobe.com> > > > I agree here: Disabling the test and having an issue keeps the build green > > but bears the danger of forgetting about it ... > > > > Regards > > Felix > > > > Am 02.06.2013 um 16:04 schrieb Eric Norman: > > > > > Personally, I'm not a big fan of hiding flaky/failing tests since it > > tends > > > to remove some of the motivation to stabilize/fix them in a timely > > manner. > > > > > > That's my 2 cents. > > > > > > Regards, > > > Eric > > > > > > On Fri, May 31, 2013 at 12:14 PM, Robert Munteanu <romb...@apache.org > > >wrote: > > > > > >> Hi, > > >> > > >> It seems that the ErrorHandlingTest fails sporadically when run inside a > > >> full maven build. I've tried locating the root cause for a couple of > > >> hours but failed. For this test, and for future flaky/failing tests, I > > >> suggest that we > > >> > > >> 1. Create an issue for the failing test > > >> 2. Disable the test and mark it with the issue key > > >> 3. Re-enable the test when it is stable/passing ( which may be > > >> considerably later than step 2) > > >> 4. Close the issue after the test is re-enabled > > >> > > >> This has the advantage of keeping the build green and making it easier > > >> to find regressions since a failing or unstable build will actually mean > > >> something. > > >> > > >> What do you think? > > >> > > >> Robert > > >> > > >> > > > > > > > -- > Carsten Ziegeler > cziege...@apache.org
