On Tue, 2016-06-07 at 22:24 +0200, Sébastien Wilmet wrote: [...] > > This of course, requires real thought and engineering, but I wanted > > to > > at least offer some technical input, some starting point that could > > be > > explored - this wont be solved without real engineering, trial and > > error. > It's similar to the "master/master-stable-next/master-stable" > branches > concept that I thought about in this previous sub-thread. See those > two > mails: > https://mail.gnome.org/archives/desktop-devel-list/2016-June/msg00002 > .html > https://mail.gnome.org/archives/desktop-devel-list/2016-June/msg00008 > .html > > It doesn't look like rocket science to me, and it would get us closer > to > continuous delivery.
Honestly I hadn't put a huge amount of thought to this, but I did figure in that once there is API churn, there will be multiple variations/combinations that are desirable to build - one might want to build a certain app which has not yet adapted to an API change, in which case they need an amalgamation of branches which gets them their app building without the API change in a lower level library/service. On the other hand someone who wants bleeding edge of the given library/service will want it with the API change included... but I admit these considerations are possibly not worthwhile for our basic goals: improve CI for development cycles and help have a more friendly starting point for onboarding newcomers. I also woke up this morning to an interesting conversation on #gnome- hackers, which I missed due to timezone skew, but made me think of something cheaper also more inline with what you suggest with master- stable, master-stable-next etc. So here it goes, after reading the interesting conversation I think that we are close to something that is a step forward which will not cost very much at all, I think that the main point we disagree on is reverts in *master*, and the idea that *master* is something that can be forced to be always stable. For some of us, Milan and I at least, I think we disagree most particularly on dirtying master, this creates work especially if we have to pick up the pieces after some API change or such, and it dirties the mainline development history which is a bad thing in general, it makes bisections dirty and history tracking dirty, we've come to have some discipline over the years to even avoid any whitespace in commits because we value the history that much, we want to know that every commit in mainline is relevant. Simply put, if master is broken, it needs to be fixed the correct way, and a maintainer has never advertised master as being stable until we tag it as stable, we want to keep this. So what I thought up after reading that IRC conversation, is that there is no reason at all why all of this has to be done in master, and no justification to want this behavior on master either - we just want to be able to provide a closest-to-master as possible build that always builds, and we want that to be the "easy way" that we advertise to newcomers - more mature developers when they grow up might want to build master anyway and see the actual breakages in real time. Sorry I dont have time to spend hours reducing this huge text and being all concise, but I want to contribute to this, so here goes: Cheap proposal for a build-bot/release-team/build-sherif owned integration branch ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ So what I would propose here is a setup with some automation, similar to the proposed master-stable but I would call it 'integration' and with a few changes. Starting with our current workflow, all our projects/modules use the master branch as mainline development, feature branches are merged into master and eventually come release time, master is tagged and branched as stable and we release tarballs for each release tag. [ feature ] | \ | \ [ master ] | | [ release tag ] \ | \ [ feature ] | \ \ | \ \ | \ \ | [ release branch ] \| | | | [ bugfix ] -------> | | | | [ bugfix release ] I dont think the above should change at all; patch flow for production continues to flow in this traditional way that we have come to work with very well. Master remains mainline. What I would suggest, in addition to our setup, is that there is a new 'integration' branch for every module in the moduleset, this integration branch is automatically managed. Workflow for contributors can look more like this: [ integration ] [ master ] [ bugzilla patches ] | | / | | / patches possibly | | / made against | | / integration branch | |/ | | recommended | [ release tag ] build branch | |\ for | <----------- | \ newcommers | Auto CI | \ | | \ No commits | | \ here | | [ release branch ] | | | | | | Some assumptions I would point out about the above: o The integration branch has a strict commit policy, which is to say only the continuous automated merge process is every allowed to commit here - I would even go as far as to lock out maintainers from messing with the integration branch - This is really for an automated service, and build sheriffs can control the integration branch. In this case I really want a hard block, if I ever accidentally try to push something to 'integration' directly, I want an error, 'integration' is not where patches and work enters the work flow. o Commits are never *reverted* from the integration branch, instead of this, the history of the integration branch is automatically *rebuilt* from master, and done so on a continuous basis. This can mean that the build bots or build sheriffs can launch a script which rebuilds the history of 'integration' based on master, but excluding the offending commit, IMO it's interesting to see 'integration' as the closest possible mirror to master which 'builds at all times' Similar to Sébastien's suggestions about 'master-stable', the default for running jhbuild would be to use 'integration' to build the bleeding edge, with the added side effect of blocking all commits from any committer. A contributor can already choose pretty easily with jhbuild to build 3 or 4 modules directly from master if that's what bakes their cake, but it's not the default behavior. I think in the big picture, with this proposal we maintain the prestine meaning of what master/mainline should be, and we get all the benefits of a constantly building bleeding edge - in any case 'integration' would be exactly the same thing that people in these threads are proposing for 'master', it satisfies their requirement without interfering with mainline. It adds a little bit more work in setting this up: o Need a script to automatically/continuously rebuild 'integration' from master o CI of the integration branch should be able to blacklist broken commits and "omit" them from the history o Some complex hooks need to be implemented gnome-wide in the git repositories, ensuring that any direct commits to the integration branch are blocked, with the exception of build sheriffs and the CI service. There are also added benefits, making things easier for people babysitting the build: Build sheriffs never need to chase down maintainers explicitly before blacklisting a broken commit, the blacklist (formerly 'reverts in master') would simply effect the integration branch; maintainers should be notified via bug reports that certain commits have been 'rejected from integration' - this of course raises a red flag to the maintainers because, any patches still rejected by integration cannot logically be merged come release time. I think the above outlined overhead may cost us... say 2 or 3 weeks of scripting and setup from a single dedicated full time engineer, modulo a week or so depending on whether we can find someone who is already familiar with the gnome-continuous current setup and who can easily push this through in our infrastructure. Any thoughts ? Cheers, -Tristan _______________________________________________ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list