+1. Having put all the effort in to to get to a clean test suite, it would
be a great shame to let it slip. The effort to remain clean is much less
than the effort to get to that state in the first place: failing tests are
easier to fix when they first occur and everything is fresh in the author's
mind, than they are 6 months later. If you can't merge to master until the
tests are clean, there is a much greater incentive to take the high road
and fix problems immediately.

As a second point: if you come to an interesting Open Source project you've
never seen before, then a failing build or failing test suite is a good
reason to reject it without further thought. Cleanliness at that level is
an indication of a well-functioning and professional community, whose
software is worthy of further consideration.

Nick

On Wed, 16 Aug 2017 at 08:43 Dale Harvey <d...@arandomurl.com> wrote:

> +1, have found this useful / necessary with PouchDB over the years
>
> Describing our workflow as it may be useful (its also very similiar to
> mozillas). We have Tier 1
> supported platform, node stable, stable releases of browsers etc a checkin
> is not allowed to make
> those go red, if something lands that makes them red it gets immediately
> reverted
>
> Tier 2 systems, like CouchDB 2.0 for the period it was under development.
> We take a best effort
> at keeping it green but it doesnt prevent patches from landing. If it goes
> red it is someones job to get
> it green, if it stays red for a long period of time it becomes unsupported.
> (We are still in the process of moving
> couch2 to tier1)
>
> This allows us to stay pretty stable while not be too restrictive for
> building new features / supporting new
> platforms. The important thing however is the zero policy rule on Tier 1,
> once there is any reason for
> people to ignore failing tests they decay super quickly and it gets very
> hard to bring them back, for a while
> we had the policy of fixing things in master but there are always
> complications, backing out, fixing the tests
> and relanding is far more reliable
>
> On 16 August 2017 at 07:46, Joan Touzet <woh...@apache.org> wrote:
>
> > Hi committers,
> >
> > I'd like to propose a change to our policy on version control, namely
> > that no check-ins be allowed on the master branch unless CI test runs
> > against that PR are clean.
> >
> > We've worked hard as a group to get runs clean. We need to protect
> > that achievement and investment in our test suite. That means not
> > letting rogue check-ins slip by because we are ignoring a red X in
> > GitHub (GH) from the Travis run.
> >
> > Things I see as exceptions:
> > * Changes to things clearly not related to the test suite, i.e.
> >   documentation, support scripts, rel/overlay/etc/ files, etc.
> > * Changes already agreed upon in a previous PR/discussion for
> >   administrative tasks
> >
> > Interesting situation right now for a discussion: Garren has a PR up[1]
> > that enables the mango tests to be part of the standard Travis/Jenkins
> > runs. Unfortunately, it doesn't pass on one of our platforms right now
> > and that needs investigation. Should we allow the PR to land and fix
> > the problems in master, or should the PR hold-up until it can land along
> > with the fixes for the failing mango tests? I can see both sides of this
> > argument.
> >
> > It may or may not be possible for our GH setup to actually prevent such
> > checkins (the Apache GH setup is somewhat restricted, and various things
> > like commit hooks and webhooks have to be configured by INFRA, not us).
> >
> > I'd like to further discuss whether people feel such a hook would be
> > acceptable, onerous or otherwise. Personally, I worry that such a setup
> > might prevent us from checking in some of the exceptions above, but if
> > there is a way around it, we could proceed down that path.
> >
> > What do you think, sirs?[2]
> > Joan
> >
> >
> > [1]: https://github.com/apache/couchdb/pull/753
> > [2]: It's a Mystery Science Theatre 3000 Joel reference. :)
> >
>

Reply via email to