I've talked about updating the jenkins builds in the past to break them up into a flaky and non-flakey sets. (it is something I've setup on an internal jenkins instance). If we do that the the flakey tests should have jiras until they get fixed.
I can't do this right now but could make some time to work on in sometime next week. Jon. On Thu, Nov 8, 2012 at 12:09 PM, lars hofhansl <[email protected]> wrote: > I'd say we keep the pressure up for failing tests... I.e. we file jiras. > IMHO, a failing test should either be fixed or disabled, otherwise it just > adds noise. > > > (This is true for even occasionally failing tests. We have > 1000 tests, if > we have many tests that fail just once/100 runs, we get frequent build > failures.) > > Just my $0.02. > > > -- Lars > > > > ________________________________ > From: Stack <[email protected]> > To: HBase Dev List <[email protected]> > Sent: Thursday, November 8, 2012 11:26 AM > Subject: Re: [DISCUSSION] Policy proposal for JIRAs opened for unit test > failures without patches attached > > On Wed, Nov 7, 2012 at 4:26 PM, Andrew Purtell <[email protected]> wrote: >> There has been a recent uptick in JIRAs opened for unit test failures >> without patches attached. Since these merely duplicate information readily >> available on our Jenkins, we should institute a policy of closing them as >> Invalid if a patch is not attached to the JIRA in a timely manner (within >> hours). Simply pointing out a failing test is not >> a consequential contribution. We should also update the How To Contribute >> documentation accordingly. >> > > I can go either way. > > On the one hand our JIRA has loads of issues opened against failing > tests that we need to clear up as now as either fixed, invalid, or > still pertinent. Would be better if failing tests were just addressed > near immediately. > > On the other hand, one day we'll be in a situation where we'll want to > look at tests that failed in the past but that are currently not > failing so it'd be good to keep record of the old test in JIRA. > > I suppose I'd lean toward no special 'unit test' rule that precludes > creating issues for failing tests mostly because if a new user, it'd > be hard to explain the rule they'd be violating. > St.Ack -- // Jonathan Hsieh (shay) // Software Engineer, Cloudera // [email protected]
