On Tue, Jul 15, 2014 at 2:51 PM, Wayne Witzel <wayne.wit...@canonical.com> wrote:
> If we aren't stopping the line when our CI is in the red, then what is the > point of even having CI at all? If we are not prepared to adjust the > culture of our development. To truly halt forward progress in favor of > chasing down regressions then I struggle to find the benefit that CI and > testing is giving us at all. Just confirming that master is broken and we > are still ignoring it? If master is broken, we all our broken. No > development you are doing should proceed that is based on a broken master. > That work cannot at any point be considered in good working condition. A > problem in master is everyone's problem. > > Bugs that are found throughout the normal operations and usage of juju > should be assigned a priority and queued, but regression is a sign of a > greater problem that should be resolved immediately. Allowing regressions > to not stop the line is taking the stance that we don't care about > deterioration in our code base. > +100 to this. Regressions are a Big Deal and should be treated as such; leaving other merges queued until such a time as the regression is fixed (or backed out for rework) is entirely reasonable (and I think we've got enough evidence that the alternative really doesn't fly effectively). Cheers William > > > On Tue, Jul 15, 2014 at 9:37 AM, Nate Finch <nate.fi...@canonical.com> > wrote: > >> I don't think we need to stop the world to get these things fixed. It is >> the responsibility of the team leads to make sure someone's actively >> working on fixes for regressions. If they're not getting fixed, it's our >> fault. We should have one of the team leads pick up the regression and >> assign someone to work on it, just like any other high priority bug. >> >> >> >> On Mon, Jul 14, 2014 at 3:05 PM, Curtis Hovey-Canonical < >> cur...@canonical.com> wrote: >> >>> Devel has been broken for weeks because of regressions. We cannot >>> release devel. The stable 1.20.0 that we release is actually older >>> than it appears because we had to search CI for an older revision that >>> worked. >>> >>> We have a systemic problem: once a regression is introduced, it blocks >>> the release for weeks, and we build on top of the regression. We often >>> see many regressions.The regression mutate as people merge more >>> branches. >>> >>> The current two regressions are: >>> * win juju client still broken with unknown >>> from 2014-06-27 which has varied as a compilation >>> problem or panic during execution. >>> https://bugs.launchpad.net/juju-core/+bug/1335328 >>> >>> * FAIL: managedstorage_test trusty ppc64 >>> from 2014-06-30 which had a secondary bug that broke compilation. >>> https://bugs.launchpad.net/juju-core/+bug/1336089 >>> >>> I think the problem is engineers are focused on there feature. They >>> don't see the fallout from their changes. They may hope the fix will >>> arrive soon, and that maybe someone else will fix it. >>> >>> I propose a change in policy. When a there is a regression in CI, no >>> new branches can be merged except those that link to the blocking bug. >>> This will encourage engineers to fix the regression. One way to fix >>> the regression is to identify and revert the commit that broken CI. >>> >>> >>> -- >>> Curtis Hovey >>> Canonical Cloud Development and Operations >>> http://launchpad.net/~sinzui >>> >>> -- >>> Juju-dev mailing list >>> Juju-dev@lists.ubuntu.com >>> Modify settings or unsubscribe at: >>> https://lists.ubuntu.com/mailman/listinfo/juju-dev >>> >> >> >> -- >> Juju-dev mailing list >> Juju-dev@lists.ubuntu.com >> Modify settings or unsubscribe at: >> https://lists.ubuntu.com/mailman/listinfo/juju-dev >> >> > > > -- > Wayne Witzel III > wayne.wit...@canonical.com > > -- > Juju-dev mailing list > Juju-dev@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju-dev > >
-- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
