On Jul 10, 2014, at 16:12, Sanne Grinovero <sa...@infinispan.org> wrote:
> On 10 July 2014 15:13, Mircea Markus <mmar...@redhat.com> wrote: >> >> On Jul 10, 2014, at 15:03, Sanne Grinovero <sa...@infinispan.org> wrote: >> >>> The important point for me is that patches don't get merged if they >>> introduce any regression. I hope that rule stays? >>> BTW this matches with the "classic" approach as far as I know it. >> >> yes. A patch might pass the test when integrated and cause intermittent >> failures later on, so it's not straight forward to avoid patches introducing >> regressions, nor to identify which patch has caused it so that we can roll >> it back. > > It's self-speaking that it's hard to immediately evaluate if a patch > is going to cause intermittent / time-bound failures in the future: we > have no crystal ball, so I'm not making absurd demands. There will > always be cases in which developers will need to ask forgiveness. > > But if anything in the test run fails during a review of a patch, and > this patch still gets merged because "the cause is likely unrelated", > or worse "we'll fix that problem later" that's unacceptable and needs > to be investigated further before being merged. > I always did that, and spending a lot of time on things which are not > directly related on my goals, for sake of respect of other developers > in the team and I expect no less from everyone else: when the > testsuite doesn't pass I can't make progress on my own work and need > to necessarily shift to firefighthing. Or go on holidays. > That's what necessarily needs to happen when a bad patch slips in past > our guard, even if doing so you need to reschedule other tasks: > because otherwise it's other people, and more and more people needing > to reschedule their own tasks. Worse yet, the other people who will > need to look at it will not have any of the context to understand what > might be going on. So I hope we're on the same page with this, because > ultimately it's about respect for each person contributing or working > on it. I understand your frustration and you are absolutely right. > > I'm also puzzled on why exactly the current situation is not > sustainable. Sure there is a lot of technical debt to pay, but you > know debts come with interests and it's hard.I've also suggested many > ways to improve our testsuite to make our life easier, like use > Byteman, mock the timers via a TimeService, get rid of TestNG to move > to something more reliable, remove unnecessary dependencies from each > module, use more mocking in areas where we don't actually have an > interest in testing JGroups.. IMO the problem is that ATM we lack the discipline, as a team, to maintain the suite green. In the last 7 years I've been working on JBossCache and then ISPN it has always been like this - intermittent failures - and always to big and too ugly of a task to fix it and maintain it green, especially with the schedule we had ahead of us. Now that the team is big, this compromise - intermittent failures - really impacts the productivity of a lot of people. The suite should stay green and we should treat any failure that keep us away from that as an blocker - a thing we didn't really do up to now. > Nothing like that was done, so you can't say in all fairness that we > tried to do better. My PR which is the first step to get rid of TestNG > from the Query modules is open since 43 days.. so don't expect me to > fix any problem quickly, it's not particularly motivating. > > You can try playing with processes, and I don't disagree with idea, > but when I'm able to find a regression is is that I will do: I'll > revert all commits until the first stable point I find. I've > suggested this approach to you as well, not sure why you don't apply > it more regularly? Because the last green commit will trigger intermittent failures, try it. > There is no shame in reverting commits, especially > if we do it regularly. > It's stable today, so I'll make a note of this commit id, my crystal > ball is telling that it will be useful soon ;-) > > Sanne > > >> >>> >>> On 10 July 2014 14:04, Mircea Markus <mmar...@redhat.com> wrote: >>>> I just had a chat with Dan and we don't think the current process for the >>>> test suite works. Not hard to see why, the suite is almost never green. So >>>> we will adopt a more classic and simple approach: if a test fails a >>>> blocker JIRA is created for it and assigned to a component lead then team >>>> member who'll start working on it *immediately*. Dan will be watch dog >>>> starting today so please expect blocker JIRAs coming your way and treat >>>> them accordingly. >>>> >>>> Cheers, >>>> -- >>>> Mircea Markus >>>> Infinispan lead (www.infinispan.org) >>>> >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev@lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev@lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> Cheers, >> -- >> Mircea Markus >> Infinispan lead (www.infinispan.org) >> >> >> >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev Cheers, -- Mircea Markus Infinispan lead (www.infinispan.org) _______________________________________________ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
