+1, have found this useful / necessary with PouchDB over the years Describing our workflow as it may be useful (its also very similiar to mozillas). We have Tier 1 supported platform, node stable, stable releases of browsers etc a checkin is not allowed to make those go red, if something lands that makes them red it gets immediately reverted
Tier 2 systems, like CouchDB 2.0 for the period it was under development. We take a best effort at keeping it green but it doesnt prevent patches from landing. If it goes red it is someones job to get it green, if it stays red for a long period of time it becomes unsupported. (We are still in the process of moving couch2 to tier1) This allows us to stay pretty stable while not be too restrictive for building new features / supporting new platforms. The important thing however is the zero policy rule on Tier 1, once there is any reason for people to ignore failing tests they decay super quickly and it gets very hard to bring them back, for a while we had the policy of fixing things in master but there are always complications, backing out, fixing the tests and relanding is far more reliable On 16 August 2017 at 07:46, Joan Touzet <woh...@apache.org> wrote: > Hi committers, > > I'd like to propose a change to our policy on version control, namely > that no check-ins be allowed on the master branch unless CI test runs > against that PR are clean. > > We've worked hard as a group to get runs clean. We need to protect > that achievement and investment in our test suite. That means not > letting rogue check-ins slip by because we are ignoring a red X in > GitHub (GH) from the Travis run. > > Things I see as exceptions: > * Changes to things clearly not related to the test suite, i.e. > documentation, support scripts, rel/overlay/etc/ files, etc. > * Changes already agreed upon in a previous PR/discussion for > administrative tasks > > Interesting situation right now for a discussion: Garren has a PR up[1] > that enables the mango tests to be part of the standard Travis/Jenkins > runs. Unfortunately, it doesn't pass on one of our platforms right now > and that needs investigation. Should we allow the PR to land and fix > the problems in master, or should the PR hold-up until it can land along > with the fixes for the failing mango tests? I can see both sides of this > argument. > > It may or may not be possible for our GH setup to actually prevent such > checkins (the Apache GH setup is somewhat restricted, and various things > like commit hooks and webhooks have to be configured by INFRA, not us). > > I'd like to further discuss whether people feel such a hook would be > acceptable, onerous or otherwise. Personally, I worry that such a setup > might prevent us from checking in some of the exceptions above, but if > there is a way around it, we could proceed down that path. > > What do you think, sirs?[2] > Joan > > > [1]: https://github.com/apache/couchdb/pull/753 > [2]: It's a Mystery Science Theatre 3000 Joel reference. :) >
