Re: [ Revised Proposal ] Continuous Builds in GNOME
Hi Philip, On Tue, 2016-06-21 at 04:28 +, philip.chime...@gmail.com wrote: [...] > Tristan, I wonder if I can present a different perspective that might > reconcile these differences. > > I'm an active contributor to the gtk-osx module, which is a > collection of scripts and jhbuild modulesets for building (some of) > the GNOME stack on Mac OSX. While I personally believe very much in > the goal of the GNOME desktop as a unified platform, I also put a lot > of work into keeping the the GNOME stack libraries buildable on Mac, > so that people can provide builds of GNOME apps (such as Gedit) on > Mac as well as build their own cross-platform apps using GNOME tools. > I think you even reviewed some of my patches to make Glade buildable > on Mac not too long ago. > > I hope you'll agree that this is a diverse use of GNOME modules, > focused on consumers outside of the GNOME desktop. Sure, Running GNOME apps built on the GNOME stack on other operating systems qualifies, and helps to raise our standards of quality and flexibility; at least ensuring that some of our apps and libraries arent tied too tightly to 'linux only' or even 'linux + systemd only'. But I'm also interested in seeing our stack being the stack of choice for developing appliances and systems in the IVI and mobile spaces for instance. It was not all that long ago that GTK+ was a viable competitor to Qt for embedded. > Please believe me then when I say that _nothing_ would make me more > happy, in the capacity of gtk-osx contributor, than a CI system, and > with it the ability to assume that the entire stack is buildable at > any given time. Yes, some things would make us happy even if they are not entirely viable, and this is the point I've been trying to make. The _only_ way this can be even halfway viable is if we were to make the statement that all projects under the GNOME umbrella have the single priority of integrating into a single unified user experience and that this goal should override all other technical considerations - this is a statement I perceive as hostile towards projects which only want to be the best solution for the functionalities they provide. And the people who just want to work on the best solution, are the people we should be trying to attract and engage. That said, there are two types of build breakages, there are simple mistakes and there are intentional API breakages, I dont even want to talk about mistakes, apparently there are many of those and that should stop, if we want to be industry leaders; please dont go committing stuff that breaks even the build of your own module. Regarding intentional breakages; while I personally would love it if the whole platform were to be meticulously API and ABI stable at all times this is just not possible. Firstly, we don't hold all platform libraries to the high standards as we do glib for instance, and secondly; in order to improve the platform we need to refactor vigorously. If we cannot enforce a discipline where all API breaks come with an entirely new, parallel installable library (not soname bump), then this refactoring will equate to API breaks, and if we're not making them, then we are stagnating and falling behind. If you've been following planet GNOME you will be aware that in fact the GTK+ team is considering a plan to break API more often; with a rolling 'unstable' release; if this route is taken then you can expect a lot more cross-module breakages, how can one expect that all of GNOME projects 'master' branch always be in sync with API changes from underlying modules ? The best we can do is introduce breakages early in the cycle, allowing a reasonable grace period for depending projects to adapt to those. On the other hand, having a reference 'integration' branch which always does build, or a 'master-stable' as Sébastien has called it (it's pretty much the same proposal when you boil it down), would allow us to always easily see the delta of what modules have adapted to API changes, and would also provide an entry point for newcomers to have something as close to master as possible which builds. Anyway, at this point I'm just reiterating the same things I've been saying since our first thread in January - and for doing so I am perceived as the enemy of CI, I am not; I just dont see how everything 'master' can possibly be expected to always be buildable, and trying to come up with some approach which could. Cheers, -Tristan ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: [ Revised Proposal ] Continuous Builds in GNOME
On Sun, Jun 19, 2016 at 7:34 AM Tristan Van Berkom < tristan.vanber...@codethink.co.uk> wrote: > On Sat, 2016-06-18 at 08:15 +0100, Emmanuele Bassi wrote: > [...] > > Tristan: I understand you don't give a crap about GNOME as a holistic > > project; you've made it *painfully* clear over the years. I'd like to > > point out that without GNOME working as a single unit nothing we do, > > or did up until now, really matters in the grand scheme of things. > > The > > *only* successful thing out of GNOME is the GNOME desktop environment > > and its whole stack of technologies; everything else has been a > > failure, mostly caused by this inane obsession of considering the > > GNOME platform as a bag of loosely coupled projects that somebody has > > to work hard into identifying and keep working together, granting the > > position of gatekeeper to a bunch of middle men. This position is > > ignorant of the history of the project, especially of its failures; > > it's insulting of the work of hundreds of people who want to keep > > this > > thing working and keep it approachable to newcomers; and completely > > invalidated by the reality of what happens every day in the > > commercial > > space. I also don't understand your opposition to keeping the only > > integration branch we have, the 'master' branch, building constantly, > > considering this is a basic engineering practice, but at this point I > > can barely muster the strength to even care. In any case, as I said > > to > > Milan, I'll gladly consider any project you maintain at the same > > level > > as an external dependency, and thus just tag it. Feel free to go > > through the Continuous manifest and Jhbuild moduleset and point out > > which modules you still maintain. > > > Emmanuele, > > Thank you for sharing this. You make the perfect example. > > You will note, that my "astronauting" is in fact a sincere attempt to > reconcile both of these views with some plausible technical approach, > so while your goals are clearly not the same as mine; I am being > sensitive to your goals, in fact I want some team within GNOME to be > successful at integrating all of these projects into some coherent user > experience - I only want that team to accept that the components they > intend to integrate have bigger plans than *just* being a part of this > integrated whole. > > You on the other hand seem threatened by my views and consequently act > intolerant towards the existing diversity of goals of projects within > GNOME - I don't think this intolerance is an attitude we should have to > deal with, we should be instead looking for creative ways to reconcile > both views. > > Yes, the GNOME Desktop Environment as a unified thing has never been a > personal goal for me to work towards, rather the most important to me > is to build a solid and stable platform with good development tools, > and I want it to be the platform of choice in the FOSS world - if the > GNOME Desktop Environment turns out to be the only consumer of the work > I've done over the years, this will be the biggest failure of all in my > eyes. > > You may have accepted defeat but as long as I spend any energy in GNOME > at all, I simply can not. > Hi, Tristan, I wonder if I can present a different perspective that might reconcile these differences. I'm an active contributor to the gtk-osx module, which is a collection of scripts and jhbuild modulesets for building (some of) the GNOME stack on Mac OSX. While I personally believe very much in the goal of the GNOME desktop as a unified platform, I also put a lot of work into keeping the the GNOME stack libraries buildable on Mac, so that people can provide builds of GNOME apps (such as Gedit) on Mac as well as build their own cross-platform apps using GNOME tools. I think you even reviewed some of my patches to make Glade buildable on Mac not too long ago. I hope you'll agree that this is a diverse use of GNOME modules, focused on consumers outside of the GNOME desktop. Please believe me then when I say that _nothing_ would make me more happy, in the capacity of gtk-osx contributor, than a CI system, and with it the ability to assume that the entire stack is buildable at any given time. I realize that having a CI system on the proprietary Mac OSX is quite another controversial step beyond having a CI system in the first place, but I would be happy just to have commits that break the build on Linux reverted automatically so that when I trepidatiously type "jhbuild build" on my Mac and hover over the Enter key thinking "do I really want to commit to another three evenings of build bugs," I can at least be assured that any failures are going to be Mac build bugs (fixing which is the reason I contribute to gtk-osx) and not general build breakage (which I feel I shouldn't have to spend my limited time dealing with as there are automated tools, such as CI, for catching this sort of thing!) I hope to illustrate with this, that a CI system, and stabilit
Re: [ Revised Proposal ] Continuous Builds in GNOME
On Sat, 2016-06-18 at 08:15 +0100, Emmanuele Bassi wrote: [...] > Tristan: I understand you don't give a crap about GNOME as a holistic > project; you've made it *painfully* clear over the years. I'd like to > point out that without GNOME working as a single unit nothing we do, > or did up until now, really matters in the grand scheme of things. > The > *only* successful thing out of GNOME is the GNOME desktop environment > and its whole stack of technologies; everything else has been a > failure, mostly caused by this inane obsession of considering the > GNOME platform as a bag of loosely coupled projects that somebody has > to work hard into identifying and keep working together, granting the > position of gatekeeper to a bunch of middle men. This position is > ignorant of the history of the project, especially of its failures; > it's insulting of the work of hundreds of people who want to keep > this > thing working and keep it approachable to newcomers; and completely > invalidated by the reality of what happens every day in the > commercial > space. I also don't understand your opposition to keeping the only > integration branch we have, the 'master' branch, building constantly, > considering this is a basic engineering practice, but at this point I > can barely muster the strength to even care. In any case, as I said > to > Milan, I'll gladly consider any project you maintain at the same > level > as an external dependency, and thus just tag it. Feel free to go > through the Continuous manifest and Jhbuild moduleset and point out > which modules you still maintain. Emmanuele, Thank you for sharing this. You make the perfect example. You will note, that my "astronauting" is in fact a sincere attempt to reconcile both of these views with some plausible technical approach, so while your goals are clearly not the same as mine; I am being sensitive to your goals, in fact I want some team within GNOME to be successful at integrating all of these projects into some coherent user experience - I only want that team to accept that the components they intend to integrate have bigger plans than *just* being a part of this integrated whole. You on the other hand seem threatened by my views and consequently act intolerant towards the existing diversity of goals of projects within GNOME - I don't think this intolerance is an attitude we should have to deal with, we should be instead looking for creative ways to reconcile both views. Yes, the GNOME Desktop Environment as a unified thing has never been a personal goal for me to work towards, rather the most important to me is to build a solid and stable platform with good development tools, and I want it to be the platform of choice in the FOSS world - if the GNOME Desktop Environment turns out to be the only consumer of the work I've done over the years, this will be the biggest failure of all in my eyes. You may have accepted defeat but as long as I spend any energy in GNOME at all, I simply can not. Regards, -Tristan ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: [ Revised Proposal ] Continuous Builds in GNOME
On Sat, Jun 18, 2016 at 08:15:19AM +0100, Emmanuele Bassi wrote: > So, I want to stop this thread on my side because not only is making > me not want to contribute anything ever again, but it also is > absolutely pointless busy work that relies on two people architecture > astronauting without the minimal intention of building anything or > even consensus. Citing myself from several mails ago: "With master/master-stable, it's a more relaxed atmosphere." https://mail.gnome.org/archives/desktop-devel-list/2016-June/msg2.html There is a proposal that is simple to understand: when tagging a module in GContinuous, this would mirror what jhbuild builds by default. With a branch. That's it. Then there are more complicated proposals where everything is fully automated, delivering a stable desktop continuously to end users. I didn't talk about it, but it would enable to do canary releasing and A/B testing. And there is a gray scale between those two proposals. -- Sébastien ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: [ Revised Proposal ] Continuous Builds in GNOME
So, I want to stop this thread on my side because not only is making me not want to contribute anything ever again, but it also is absolutely pointless busy work that relies on two people architecture astronauting without the minimal intention of building anything or even consensus. I consider the matter that started this thread — ensuring that jhbuild does non-srcdir builds by default — solved, and any talk about continuous integration in GNOME absolutely moot until somebody comes up with an actual CI server and tool that is *not* Continuous, and shows that it works well enough that we can start talking about commit and integration policies. I also consider the matter of fixing the Continuous builds solved, in the sense that I'll continue tagging badly behaving modules and file bugs. That is, obviously, if I can muster caring about this stuff — something that I keep finding hard to conjure out of thin air and my limited amount of time, given the feedback I've received. Instead of talking about these two matters, let me talk to you about the only two tools we have at our disposal, and why I (possibly naively) thought they were important enough to the GNOME project to raise their continued working state as an important topic. Let's start with jhbuild. I personally don't care if you're a long-standing contributor to the GNOME project and you feel like you don't need to use jhbuild because you either don't give a crap about your system's state, or because you're competent enough to deal with it; jhbuild *is* the tool that we provide to newcomers to the project. It's the major blocker for people that come to use and wish to build and contribute to GNOME. Since GNOME is a free software project and contributors are a slowly renewable resource, introducing barriers like having stuff *not* build is a perfect way to stagnate the contributors pool, and turn enthusiasts and people willing to contribute into people that will negatively impact the perception of GNOME — and with good reason. Our Git repositories, and jhbuild, are part of our developer outreach. Ensuring that potential new contributors can follow a simple step tutorial without the build crapping its pants two thirds into the process is the way we grow our contributor's base. As much as I wish to retain one existing maintainer, I'd rather have 10 new contributors every month. Let's move on to Continuous. As much as it's a build bot, and a continuous delivery testing mechanism, Continuous is much more, and much simpler, than that: it's our Minimum Viable Product. It's the closest thing we have to present to OSVs and OEMs and say: "You can take this list of modules, with the given commit ids, and build options; build them; and plop the resulting file system tree into a product — and we can guarantee you that it'll boot, log in, and give you the exact same results we seen". You literally have *no* idea how powerful this is. It allows companies to create a whole operating system without having to deal with the whims and policies of Linux distributions. It gives us the ability to QA what we create. It gives Linux distributions the ability to determine a baseline for their own packaging, and check if something is a bug they introduced or if it's upstream. Continuous is GNOME, and GNOME is Continuous. Both of these things are so important because they demonstrate that GNOME is bigger than the sum of its parts. To return to the original topic: Milan, I understand you don't like any policy imposed on your projects. That's perfectly understandable, so I will consider Evolution and all its dependent modules as external dependencies, and treat them as such. I'd even be happy to only build the stable branches in Continuous, so we can keep the burden at a minimum, and we'll ensure that GNOME does not use unstable API until it's been released, but for the time being I'll just keep tagging stuff — both in Continuous and in Jhbuild. Tristan: I understand you don't give a crap about GNOME as a holistic project; you've made it *painfully* clear over the years. I'd like to point out that without GNOME working as a single unit nothing we do, or did up until now, really matters in the grand scheme of things. The *only* successful thing out of GNOME is the GNOME desktop environment and its whole stack of technologies; everything else has been a failure, mostly caused by this inane obsession of considering the GNOME platform as a bag of loosely coupled projects that somebody has to work hard into identifying and keep working together, granting the position of gatekeeper to a bunch of middle men. This position is ignorant of the history of the project, especially of its failures; it's insulting of the work of hundreds of people who want to keep this thing working and keep it approachable to newcomers; and completely invalidated by the reality of what happens every day in the commercial space. I also don't understand your opposition to keeping the only integration branch we
Re: [ Revised Proposal ] Continuous Builds in GNOME
First, I've been on vacation this week and have been able to put time into this, with the hope that next week I will be able to focus on $dayjob without regretting too much not being able to take part in this important debate. I hope we are going somewhere. Also, in closing in this mail, I will just highlight my motivations for bringing up this proposal in the first place. Sri, Please just gloss over the technical stuff and read the end, I think you are in a position to effect change and this is written with you in mind (from the "Why do I think..." heading). On Fri, 2016-06-17 at 15:17 +0200, Milan Crha wrote: > On Fri, 2016-06-17 at 17:11 +0900, Tristan Van Berkom wrote: > > > > I dont believe you have any such functionality in jhbuild. > > > > At best, you can specify the sha1 of the commit you want to build > > of > > a given module, this would not let you single out one commit, > > omitting it from history in integration and at least attempt to > > apply > > subsequent commits without that one included, re-including that > > commit in the correct place in history only when CI infrastructure > > would find integration unbroken. > Hi, > I never thought of a single commit removal from a set of commits. To > be > honest, that sounds even more crazy (not 'crazy' like 'insane'). :) > We are talking about master, the development version of the software. > If you'd like to tell me that you are at commit X and feature which I > added at commit X-3 does not work for you, then trying to figure out > why exactly you do not see that feature is a real nightmare and waste > of time for both the reporter and the developer, especially when you > want to remove only single commit from a history. Not talking that > following commit can "build on top of the previous commit", which is > quite usual. > > > > > The thing is, either way; your work flow will not be interrupted at > > all with this approach, ... > See above. Milan, I think that if your commit broke integration with the wider GNOME ecosystem, that will matter to you and you will be quite aware of which commit in master broke it and why until it is fixed. I think we have to ensure that people take integration breaking commits seriously and ultimately, an integration breaking commit would block any stable release. That said, in your above scenario; a disparity between master and integration is a serious issue, but consider the alternative: If a new comer wants to get involved in GNOME, they probably have a specific itch to scratch, say with Evolution or EDS; they will either be able to build everything up to EDS & Evolution easily; have an easy experience, and be able to submit a patch to bugzilla, or, they will fail to build and get side tracked because the bleeding edge of master for all GNOME modules doesn't build. In other words (and this has absolutely nothing to do with using jhbuild or not) when you build everything from master and it breaks before you are even able to work on the issue you wanted to work on, this is discouraging; this means that we are offloading our technical debts to newcomers, forcing them to deal with our bugs before being able to even submit a patch they wanted to contribute. Would you prefer that a potential EDS contributor be able to submit a patch to bugzilla, even if it does not match up *exactly* against master ? Or would you prefer that your potential contributor get discouraged while trying to build EDS against the latest gcr, libsecret, libgdata or such, or just get bogged down trying to fix an EDS bug to adjust to some API churn in a dependent GNOME library ? I think receiving a patch for what the contributor wanted to work on is preferable, even if you happen to fall on a patch that doesn't apply exactly against master, in the 1% of the time that EDS integration is broken and lagging behind. [...] > I said earlier in the thread that the current situation works for me > the best. There is clearly stated what the continuous builds from, at > which exact commit it "stopped updating" certain module, because of > some breakage, and everything references the real master branch. That > the continuous uses jhbuild might be just a coincidence, even it > sounds > like the moduleset is targeted for the Continuous builds, when it can > influence an environment for any developer using the jhbuild (I do > not > use it, this is how I understood how it works in the jhbuild world). I'll just note again this is not about jhbuild particularly, jhbuild in master tries to build everything from master, sometimes it breaks and I think that's natural in the present state of affairs. gnome-continuous does *not* build jhbuild modulesets, it uses it's own json format for, also building everything from master: https://git.gnome.org/browse/gnome-continuous/tree/manifest.json This is about getting the bleeding edge, or something as close as possible to the bleeding edge to be always buildable; for two reasons: a.) T
Re: [ Revised Proposal ] Continuous Builds in GNOME
On Fri, 2016-06-17 at 17:11 +0900, Tristan Van Berkom wrote: > I dont believe you have any such functionality in jhbuild. > > At best, you can specify the sha1 of the commit you want to build of > a given module, this would not let you single out one commit, > omitting it from history in integration and at least attempt to apply > subsequent commits without that one included, re-including that > commit in the correct place in history only when CI infrastructure > would find integration unbroken. Hi, I never thought of a single commit removal from a set of commits. To be honest, that sounds even more crazy (not 'crazy' like 'insane'). :) We are talking about master, the development version of the software. If you'd like to tell me that you are at commit X and feature which I added at commit X-3 does not work for you, then trying to figure out why exactly you do not see that feature is a real nightmare and waste of time for both the reporter and the developer, especially when you want to remove only single commit from a history. Not talking that following commit can "build on top of the previous commit", which is quite usual. > The thing is, either way; your work flow will not be interrupted at > all with this approach, ... See above. > I realize my answer is quickly written and I would like to keep my > mind open and receptive, but I would also like to hear a more > detailed counter proposal on your end too, weighing the benefits of > managing this in a jhbuild moduleset vs a unified project wide git > branch I didn't think of my workflow changes in any deep way, also because I do not use jhbuild (that's not a tool for me). I just want to keep things simple as much as possible. Not only for me, but for the others too. Adding a new branch adds some non-trivial complexity. Consider the early adopters, whom will fill bugs against the developer version of some module. The developer will not only ask "what steps to do to reproduce the issue", but also "which branch was used and with what commits". And it happens often that the developers do not react on bugs "quickly", thus the commit numbers in the integration branch can be easily lost (branch rebuilt), thus making the bug report more or less useless. I said earlier in the thread that the current situation works for me the best. There is clearly stated what the continuous builds from, at which exact commit it "stopped updating" certain module, because of some breakage, and everything references the real master branch. That the continuous uses jhbuild might be just a coincidence, even it sounds like the moduleset is targeted for the Continuous builds, when it can influence an environment for any developer using the jhbuild (I do not use it, this is how I understood how it works in the jhbuild world). I do not care about the new branch, if you (plural 'you') will decide to use it, then I've nothing significant against. I only think it'll add too much confusion for the developers, that it'll strike back in some way in the future. I only do not like the idea of random reverts, the master branch is the pure bleeding edge, thus if the automatically managed branch is a way to go for you (plural 'you'), to avoid those reverts, then feel free to do it. Maybe, only, it would be ideal if the new branch won't influence daily `git pull` on the projects, due to bandwidth limitation, connection speed and similar constraints. That would, maybe, better fit to create a new project for the automatic builds, with the "interesting" git sub-modules there at certain commits. There might not be much difference between the new project with submodules and 10K branches in each module, right? Your new tool would still work on the git commits. That's still too complex, too limiting for a daily work on the master (not for me, but for jhbuild users to whom some modules will automatically stop updating at certain commit which can break other modules, like with the API changes, but the developer of the affected module might not ever notice, because he/she will stay at the commit which precedes the API change). There had been said in this thread already that having a continuous integration (automatic builds) is not easy, especially for such a large project like GNOME. Whenever a "Continuous" was said in this thread a shortly after followed "jhbuild". It's understood. The jhbuild is the current tool for the automatic builds. Supporting random automatic build systems, which is what you said could be done with the 'integration' branches, is not needed from my point of view, due to the GNOME project complexity. Thus target jhbuild only. Using modulesets (it's called like that, right?) for jhbuild and writing an export/import utility to transform the jhbuild's moduleset into another automated build integration could be also doable, hypothetically speaking. If any such thing would be needed in the future. That is, let's target jhbuild first. The jhbuild already has a functionality
Re: [ Revised Proposal ] Continuous Builds in GNOME
On Fri, 2016-06-17 at 07:55 +0200, Milan Crha wrote: > On Thu, 2016-06-16 at 14:16 +0900, Tristan Van Berkom wrote: > > > > o The integration branch of every module is continuously > > reconstructed based on that module's master branch. > Hi, > as the 'integration' branch targets only jhbuild, then it doesn't > make > sense to add it at each git module, no? I dont think it targets only jhbuild, but jhbuild is an obvious plausible consumer of it - rather it is the stomping grounds for automated CI to be developed and the reference point for automatically filing bugs against modules for broken commits; jhbuild is the consumer of this which eases workflow for new comers. > Thinking of it, instead of > playing with a jhbuild only, you propose to add one more layer (and > place of possible issues) between the git master <-> jhbuild > connection. The proposal, as I understand it, is: > git master <-> git integration <-> jhbuild > where the git integration can be sometimes skipped in the jhbuild. > But > as you already have a functionality to skip some commits inside the > jhbuild, I dont believe you have any such functionality in jhbuild. At best, you can specify the sha1 of the commit you want to build of a given module, this would not let you single out one commit, omitting it from history in integration and at least attempt to apply subsequent commits without that one included, re-including that commit in the correct place in history only when CI infrastructure would find integration unbroken. > and the integration branch is meant to do exactly that, then > why to have one software which would eat only space on a disk I think disk space is really not a concern here, integration branches would be rebuilt, they should not be accumulative, in the usual case where no commits need to be omitted; the ref is just a ref to master. If disk space is an issue somehow (because git will normally keep history even when branches are re-written, the old sha1's still exist), a mirror could also be an option, but I think branches are better. > in each > git module with a new branch history, instead of writing the > automated > build tool which would cooperate directly with the jhbuild setup? You > also want to add some restrictions on the git integration branch, > which > is even more unnecessary work. Honestly, I see where you are going, but I tend to disagree; managing this sort of thing in a jhbuild xml file, who's state needs to be managed ideally by a cooperation of gnome-continuous build bots (probably multiple in tandum to test on all supported arches) sounds like a lot of work, not to mention it does not allow for singling out commits, it can only block integration on the first integration breaking commit. > Not having the branch will make things easier for the developers and > new comers too, as there will be no confusion for them. Having > somewhere a super-detailed description of the integration branch > intention is nice, but it'll be easy to forget of it during the time. The thing is, either way; your work flow will not be interrupted at all with this approach, it offers greater flexibility for CI systems to be improved on looking forward and IMO actually seems less complex in a first implementation than trying to manage this all in a single jhbuild moduleset. I realize my answer is quickly written and I would like to keep my mind open and receptive, but I would also like to hear a more detailed counter proposal on your end too, weighing the benefits of managing this in a jhbuild moduleset vs a unified project wide git branch, how much work it would really take and how this alternative route would impact our workflow; for instance the first thing I can see is that I would have to regularly refresh my integration modulesets if I wanted to have a working build - while making jhbuild simply target integration by default would not involve this extra step to confuse new comers either. This is not a super detailed proposal, but lets start with a base minimum of at least trying to cover all the bases. Best, -Tristan ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: [ Revised Proposal ] Continuous Builds in GNOME
On Thu, 2016-06-16 at 14:16 +0900, Tristan Van Berkom wrote: > o The integration branch of every module is continuously > reconstructed based on that module's master branch. Hi, as the 'integration' branch targets only jhbuild, then it doesn't make sense to add it at each git module, no? Thinking of it, instead of playing with a jhbuild only, you propose to add one more layer (and place of possible issues) between the git master <-> jhbuild connection. The proposal, as I understand it, is: git master <-> git integration <-> jhbuild where the git integration can be sometimes skipped in the jhbuild. But as you already have a functionality to skip some commits inside the jhbuild, and the integration branch is meant to do exactly that, then why to have one software which would eat only space on a disk in each git module with a new branch history, instead of writing the automated build tool which would cooperate directly with the jhbuild setup? You also want to add some restrictions on the git integration branch, which is even more unnecessary work. Not having the branch will make things easier for the developers and new comers too, as there will be no confusion for them. Having somewhere a super-detailed description of the integration branch intention is nice, but it'll be easy to forget of it during the time. Bye, Milan ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: [ Revised Proposal ] Continuous Builds in GNOME
On Wed, 2016-06-15 at 17:49 +, Sriram Ramkrishna wrote:
[...]
> I think with a couple of iteration we can work out an agreement. :)
Hi Sri,
I can see you read through this diagonally, so I'll try a TL;DR version
below...
> So to wit, let me summarize your points:
>
> * Master is unstable and can be in unbuildable state until we apply a
> tag making it stable.
Basically what it is today, mostly always buildable modulo transitional
periods in times of API churn, and modulo some mistakes which may take
time to fix for maintainers in different timezones with dayjobs.
> * We have a 'integration' branch or something like that which is
> always in buildable state but lags behind master by some undetermined
> number of commits.
The integration branch as proposed does not lag behind except by
commits which break integration, in all other respects it is always
exactly master.
> I'm a little unclear what a maintainer's workflow is, and how can we
> ascertain that they push to this integration branch when there is no
> social policy in place to do so. I mean some maintainers can
> completely ignore that if they wanted to.
Nobody ever pushes to 'integration' (at least not developers and
maintainers), and this is why I want some strict git hooks in place to
ensure that any pushes to the integration branch are immediately
rejected ('git push origin integration' shows you an error).
TL;DR version is basically:
o Yes, this is going to take some actual work to implement, technical
work, but not an astronomical amount, maybe a few weeks.
o The integration branch of every module is continuously
reconstructed based on that module's master branch.
This must be an automated process, which runs perhaps every 10 min
or so, updating integration branches of all modules whenever new
commits are available on master.
At first it wont be completely automated, it will require some
human intervention to assist in maintaining the blacklist of
broken commits (we have a problem to solve which is deciding
which commits to blacklist in the cases of API churn).
Ideally though, the buildbots should be able to reconstruct
integration by performing many variants and deciding on the
blacklist with least changes omitted.
o Notifications are delivered, perhaps bugs filed, against commits
in master which dont integrate properly into 'integration'
No need for a grace period, when your commit to master breaks
integration, it is omitted from integration and you get a bug
report.
o Default for jhbuild is 'integration' of everything, which is
almost always buildable (in fact it could be made to be literally
always buildable with a bit more effort, assuming some tries
are performed by the build bots before constructing the integration
branches).
In terms of workflow this would mean people would always work
against the integration branch, except that developers/maintainers
of a given specific module would checkout a few modules as master.
Commits only ever enter the history via regular master or stable
branches.
Basic gist of the idea is like this.
Cheers,
-Tristan
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: [ Revised Proposal ] Continuous Builds in GNOME
On Tue, Jun 7, 2016 at 9:45 PM Tristan Van Berkom wrote: > On Tue, 2016-06-07 at 22:24 +0200, Sébastien Wilmet wrote: > [...] > > > This of course, requires real thought and engineering, but I wanted > > > to > > > at least offer some technical input, some starting point that could > > > be > > > explored - this wont be solved without real engineering, trial and > > > error. > > It's similar to the "master/master-stable-next/master-stable" > > branches > > concept that I thought about in this previous sub-thread. See those > > two > > mails: > > https://mail.gnome.org/archives/desktop-devel-list/2016-June/msg2 > > .html > > https://mail.gnome.org/archives/desktop-devel-list/2016-June/msg8 > > .html > > > > It doesn't look like rocket science to me, and it would get us closer > > to > > continuous delivery. > > Honestly I hadn't put a huge amount of thought to this, but I did > figure in that once there is API churn, there will be multiple > variations/combinations that are desirable to build - one might want to > build a certain app which has not yet adapted to an API change, in > which case they need an amalgamation of branches which gets them their > app building without the API change in a lower level library/service. > On the other hand someone who wants bleeding edge of the given > library/service will want it with the API change included... but I > admit these considerations are possibly not worthwhile for our basic > goals: improve CI for development cycles and help have a more friendly > starting point for onboarding newcomers. > > I also woke up this morning to an interesting conversation on #gnome- > hackers, which I missed due to timezone skew, but made me think of > something cheaper also more inline with what you suggest with master- > stable, master-stable-next etc. > > So here it goes, after reading the interesting conversation I think > that we are close to something that is a step forward which will not > cost very much at all, I think that the main point we disagree on is > reverts in *master*, and the idea that *master* is something that can > be forced to be always stable. > I think with a couple of iteration we can work out an agreement. :) So to wit, let me summarize your points: * Master is unstable and can be in unbuildable state until we apply a tag making it stable. * We have a 'integration' branch or something like that which is always in buildable state but lags behind master by some undetermined number of commits. I'm a little unclear what a maintainer's workflow is, and how can we ascertain that they push to this integration branch when there is no social policy in place to do so. I mean some maintainers can completely ignore that if they wanted to. I think in general we still need to figure out how to socially have compliance as I can envision some maintainers not wanting extra work for sake of argument. [snip] > Any thoughts ? > > Thanks for taking the time to putting your thoughts and contributing positively to this discussion and making suggestion, well appreciated by the rest of us. I don't think we have gotten any feedback on this and I'm bumping this again to see if people can look at Tristan's proposal as something workable and acceptable as a base of a conversation around doing this. sri ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
[ Revised Proposal ] Continuous Builds in GNOME
On Tue, 2016-06-07 at 22:24 +0200, Sébastien Wilmet wrote: [...] > > This of course, requires real thought and engineering, but I wanted > > to > > at least offer some technical input, some starting point that could > > be > > explored - this wont be solved without real engineering, trial and > > error. > It's similar to the "master/master-stable-next/master-stable" > branches > concept that I thought about in this previous sub-thread. See those > two > mails: > https://mail.gnome.org/archives/desktop-devel-list/2016-June/msg2 > .html > https://mail.gnome.org/archives/desktop-devel-list/2016-June/msg8 > .html > > It doesn't look like rocket science to me, and it would get us closer > to > continuous delivery. Honestly I hadn't put a huge amount of thought to this, but I did figure in that once there is API churn, there will be multiple variations/combinations that are desirable to build - one might want to build a certain app which has not yet adapted to an API change, in which case they need an amalgamation of branches which gets them their app building without the API change in a lower level library/service. On the other hand someone who wants bleeding edge of the given library/service will want it with the API change included... but I admit these considerations are possibly not worthwhile for our basic goals: improve CI for development cycles and help have a more friendly starting point for onboarding newcomers. I also woke up this morning to an interesting conversation on #gnome- hackers, which I missed due to timezone skew, but made me think of something cheaper also more inline with what you suggest with master- stable, master-stable-next etc. So here it goes, after reading the interesting conversation I think that we are close to something that is a step forward which will not cost very much at all, I think that the main point we disagree on is reverts in *master*, and the idea that *master* is something that can be forced to be always stable. For some of us, Milan and I at least, I think we disagree most particularly on dirtying master, this creates work especially if we have to pick up the pieces after some API change or such, and it dirties the mainline development history which is a bad thing in general, it makes bisections dirty and history tracking dirty, we've come to have some discipline over the years to even avoid any whitespace in commits because we value the history that much, we want to know that every commit in mainline is relevant. Simply put, if master is broken, it needs to be fixed the correct way, and a maintainer has never advertised master as being stable until we tag it as stable, we want to keep this. So what I thought up after reading that IRC conversation, is that there is no reason at all why all of this has to be done in master, and no justification to want this behavior on master either - we just want to be able to provide a closest-to-master as possible build that always builds, and we want that to be the "easy way" that we advertise to newcomers - more mature developers when they grow up might want to build master anyway and see the actual breakages in real time. Sorry I dont have time to spend hours reducing this huge text and being all concise, but I want to contribute to this, so here goes: Cheap proposal for a build-bot/release-team/build-sherif owned integration branch ~~ So what I would propose here is a setup with some automation, similar to the proposed master-stable but I would call it 'integration' and with a few changes. Starting with our current workflow, all our projects/modules use the master branch as mainline development, feature branches are merged into master and eventually come release time, master is tagged and branched as stable and we release tarballs for each release tag. [ feature ] | \ | \ [ master ] | | [ release tag ] \ | \ [ feature ] |\ \ | \ \ |\ \ | [ release branch ] \| | | | [ bugfix ] ---> | | | | [ bugfix release ] I dont think the above should change at all; patch flow for production continues to flow in this traditional way that we have come to work with very well. Master remains mainline. What I would suggest, in addition to our setup, is that there is a new 'integration' branch for every module in the moduleset, this integration branch is automatically managed. Workflow for contributors can look more like this: [ integration ] [ master ] [ bugzilla patches ] | |/ | | / patches possibly | |/made against | | / int
Re: Continuous Builds in GNOME
On Tue, Jun 07, 2016 at 07:34:42AM +0200, Milan Crha wrote: > > > There is a solution: bump the major version of Camel or EDS each time an > > > API or ABI break is unavoidable, making the new major version > > > parallel-installable with the previous ones. And that, every 6 months if > > > needed (or one year if the development cycle is longer for the > > > Evolution-related projects). > > there is no need to give me pointers to things and practices which are > already used in the project. It can be useful for some other projects, > for some maintainers following this thread, thought. > > I mean, that's exactly what we do (unless accidentally overlooked). If > the API/ABI changes, a soname version is bumped for that respective > part, at least once between releases (I just did a soname version bump > of Camel for 3.21.3 in one commit, then I did another change in the > Camel API in another commit, but I didn't bump the soname version, > because that all will be available for the early adopters in one single > release, 3.21.3). What I meant is that for the Camel library, incompatible versions should be parallel-installable, like GTK+ 2 and GTK+ 3. That way, some modules could still depend on the previous major version of Camel, and you can port the modules that you want to the new major version of Camel, one at a time if you want. From a quick glance at the evolution-data-server repo, this isn't done. The difference would be that all modules still build and work fine. -- Sébastien ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Continuous Builds in GNOME
On Tue, Jun 07, 2016 at 05:24:21PM +0900, Tristan Van Berkom wrote: > One approach might be a setup where we have an RC branch opened for > integration of changes from master at the beginning of each cycle - > this could be a sort of "soft master" that builds but might not be > bleeding edge in every way, it could benefit new comers as they could > still submit patches against the RC branch and it should at least > successfully build all projects from scratch. Also it could benefit the > release team inasmuch as we could have constant awareness of exactly > how much of the bleeding edge currently integrates at a given time. > > I strongly disagree with holding project maintainers responsible for > creating feature branches and burdening them with the duty of updating > other modules not under their control, especially for reasons already > outlined in [0], however perhaps a uniform 'integration' CI branch > could be automated to a certain extent, as gnome-continuous currently > blacklists not-building modules, it could instead be made to guess what > changes break a build and recreate the integration branch without the > said failing patch, performing tries with merges from master until > something builds and integration can again include the new changes. > > This of course, requires real thought and engineering, but I wanted to > at least offer some technical input, some starting point that could be > explored - this wont be solved without real engineering, trial and > error. It's similar to the "master/master-stable-next/master-stable" branches concept that I thought about in this previous sub-thread. See those two mails: https://mail.gnome.org/archives/desktop-devel-list/2016-June/msg2.html https://mail.gnome.org/archives/desktop-devel-list/2016-June/msg8.html It doesn't look like rocket science to me, and it would get us closer to continuous delivery. -- Sébastien ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Continuous Builds in GNOME
On Fri, 2016-06-03 at 01:53 +, Sriram Ramkrishna wrote: > I found this discussion really fascinating and so I wanted to > continue it, separately from Emmanuele's thread so that issue is > resolved without bifurcating the discussion. > > My thoughts are that we really shouldn't be looking at something and > say 'well, we can't do it we don't have the resources' especially > when there is a clear return on investment. I think this would be an > interesting challenge and we should find a way to figure out how we > can attract the talent, machines and money to do this. After all, we > have a foundation dedicated to supporting the development of GNOME. Hi Sri, I am not happy with how the previous discussion evolved. I think I raised some technically valid points and historical insight, e.g. [0] and [1] and I get the impression that those who are pushing this agenda are not willing to digest the facts and converse constructively, but lets re-examine this. I am certainly not against continuous integration, but I think we need to recognize first that we are in an extremely different position from companies, organizations and projects which perform CI successfully - this is not a solved problem that we can simply apply to GNOME as a wider umbrella of various interconnected projects. CI works very well where development is done in silo, all developers are instantly reachable or replaceable, and all components are designed to work in harmony for a very singular purpose. None of these ingredients are present in the wider GNOME project. So how can we improve things ? What technical approach can we take to better leverage CI for the purpose of furthering the collective goals of projects in GNOME ? Before we can even answer that question, we have to ask also, what are the goals we want to achieve with CI, what do we want to improve ? I think this falls under 2 distinct categories: A.) Finding actual bugs and integration breakages early enough in the cycle to have a stable build come release time. B.) Making it easier for new comers to build and contribute to projects in GNOME. I think first, we need to dispel the belief that the bleeding edge of every module is going to build at all times - especially to new comers who have the idea that they can build all of GNOME and it just doesn't pan out, they get a bad experience because we advertise jhbuild of master as something that should just work. We need to be more honest here. What we have currently with gnome-continous is already helpful for (A), but can be improved. For (B) we don't have a solution right now, but there may be some technical approaches to help solve (B). One approach might be a setup where we have an RC branch opened for integration of changes from master at the beginning of each cycle - this could be a sort of "soft master" that builds but might not be bleeding edge in every way, it could benefit new comers as they could still submit patches against the RC branch and it should at least successfully build all projects from scratch. Also it could benefit the release team inasmuch as we could have constant awareness of exactly how much of the bleeding edge currently integrates at a given time. I strongly disagree with holding project maintainers responsible for creating feature branches and burdening them with the duty of updating other modules not under their control, especially for reasons already outlined in [0], however perhaps a uniform 'integration' CI branch could be automated to a certain extent, as gnome-continuous currently blacklists not-building modules, it could instead be made to guess what changes break a build and recreate the integration branch without the said failing patch, performing tries with merges from master until something builds and integration can again include the new changes. This of course, requires real thought and engineering, but I wanted to at least offer some technical input, some starting point that could be explored - this wont be solved without real engineering, trial and error. One line of thinking I think we really need to distance ourselves from, is the idea that the "social problem" of projects in GNOME is one that needs itself to be fixed. Yes it is a problem, but rather a riddle or puzzle to be solved. Lets embrace the community as it is and find a technical solution that works *for* the community, not try to teach the community that their goals should be more in line with what some particular participants think. I say this, because I sense there is some impatience coming from participants who seem to hold the belief that a uniform and beautiful desktop experience is the singular goal of all projects participating in GNOME, while on the other hand, projects who have been solving real problems and are targeting a much wider audience than just this suffer. The result is that we begin to alienate smart and diligent developers who have been long standing dedicated contributors,
Re: Continuous Builds in GNOME
On Mon, 2016-06-06 at 20:32 +, Sriram Ramkrishna wrote: > I think that is the point of the try server, right? You push it to > the try server and make sure that it builds correctly for everyone... Hi, if you mean that "jhbuild" is a synonym for "everyone", then you surely didn't open [1], to which I gave a link to earlier. There jhbuild succeeded, my local build succeeded, Fedora build succeeded, still there was an environment where it failed. And it failed properly, I'd say. Your idea of the "try server" makes me think of the way WebKit has it done. You attach a patch to their bugzilla, set some flags and a bot runs some testing and builds in various environments with that patch applied. All automated. Anything like that can work for smaller patches, but once you think of the bigger changes, like the aforementioned Camel API change [2], you get into a very different world. > A lack of diligence on your part for instance costs time and > headache somewhere else. Right, I agree, there will be always someone unhappy. The question of this thread, and the whole idea of the reverts, is who that will be. > Nope and Emmanuele does stipulate that we need someone to monitor the > build and act accordingly. However, the rest of you have an > obligation when that person nags you about it to respond in a timely > manner. I hope it was understood from my mail that I am not part of that "the rest of you" group. What you said makes me feel that you think that everyone else is "a bad guy". That's really not true. The "timely manner" is the keyword too. Sometimes it takes time before the maintainer notices that the Continuous broke. Consider different timezones, weekends, public holidays in different countries and so on. Everything matters when it comes to "timely manner". I know, this thread is also about addressing just this in some not so cruel way to anyone. > I think we all agree that there should be change, we just need to > agree what is the minimal set of things we can accomplish today and > then iterate there. We just need to establish a place where we can > at least start on a path and break current status quo. I'm sorry, but the current state works for me. Of course, I do not work on the Continuous, I have no idea how much time and headache it costs to keep it working with so many involved projects. Thus my point of view is not objective, it's rather subjective. > In general, I really don't like the idea that we're all a set of > fiefdoms with no influence on each other. Some projects are core parts, where others can use them. Some are on "leafs", final applications, which cannot be reused (in a library sense) by others. Those are very different types of projects. As had been said in this thread, Continuous doesn't build each hosted project in the GNOME infrastructure. Bye, Milan > > [1] https://bugzilla.gnome.org/show_bug.cgi?id=766540 > > [2] https://bugzilla.gnome.org/show_bug.cgi?id=764065 ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Continuous Builds in GNOME
On Mon, 2016-06-06 at 22:21 +0200, Sébastien Wilmet wrote: > On Mon, Jun 06, 2016 at 10:10:58PM +0200, Sébastien Wilmet wrote: > > > > There is a solution: bump the major version of Camel or EDS each time an > > API or ABI break is unavoidable, making the new major version > > parallel-installable with the previous ones. And that, every 6 months if > > needed (or one year if the development cycle is longer for the > > Evolution-related projects). Hi, there is no need to give me pointers to things and practices which are already used in the project. It can be useful for some other projects, for some maintainers following this thread, thought. I mean, that's exactly what we do (unless accidentally overlooked). If the API/ABI changes, a soname version is bumped for that respective part, at least once between releases (I just did a soname version bump of Camel for 3.21.3 in one commit, then I did another change in the Camel API in another commit, but I didn't bump the soname version, because that all will be available for the early adopters in one single release, 3.21.3). > Mmh maybe it's not possible for EDS, since it's a daemon (or contains > a daemon) and thus only one instance can run on a system. I don't > know much about how EDS, Camel etc are designed. The Camel is a library, which happens to be part of the evolution-data- server. The evolution-data-server consist also of some D-Bus services, which allow clients to connect to them and have address books and calendars, tasks, memos. The Camel library is used for Mail and it doesn't provide any such D-Bus service as of today. The D-Bus services are also versioned, thus, if needed, the version can be bumped. Nonetheless, the evolution-data-server doesn't support parallel builds into the same prefix, neither the evolution. We diverged from the initial thread topic a bit here, I'm afraid. Bye, Milan ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Continuous Builds in GNOME
On 06/06/2016 01:35 PM, Sriram Ramkrishna wrote: > I would think the docs team would be also interested. I wonder if we > could create coverage maps using this.. Generally the docs team doesn't write API documentation, but I certainly wouldn't turn anyone away that wants to write API docs for me. -- Christian ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Continuous Builds in GNOME
On Mon, Jun 6, 2016 at 1:03 PM Christian Hergert wrote: > On 06/06/2016 12:48 PM, Michael Catanzaro wrote: > > I agree with you that when landing a big API change, if you don't want > > to use side branches, then a revert is less preferable to tagging in > > Continuous and branching in jhbuild. (In such cases, it'd be great if > > you could handle that before pushing the API change, though. :) > > A couple weeks ago I wrote a quick hack to parse Continuous build logs > and extract CFLAGS for the built files. It can use this to then go > perform static analysis on the module with clang and extract/resolve all > function calls. > This is pretty cool! > > The goal was to be able to tell us which API are used the most (and > therefore requires our attention to documentation), but we could also > use it to tell developers everywhere in g.g.o that their API is being used. > > I would think the docs team would be also interested. I wonder if we could create coverage maps using this.. > I expect to build this out as I work more on documentation stuff for > Builder. > Excellent! sri > > -- Christian > > ___ > desktop-devel-list mailing list > desktop-devel-list@gnome.org > https://mail.gnome.org/mailman/listinfo/desktop-devel-list > ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Continuous Builds in GNOME
Hi Milan! On Mon, Jun 6, 2016 at 10:05 AM Milan Crha wrote: > On Mon, 2016-06-06 at 09:49 -0500, Michael Catanzaro wrote: > > A revert is not supposed to be "punishment" in any way... rather, > > consider it as assistance to make sure GNOME stays buildable. :) > > Hi, > maybe it's not supposed to be, but it is in my eyes. I try my best to > not break builds, I'm grateful when other folks let me know when the > builds are broken for them. Neither jhbuild catches everything [1], > which you seem to suggest. I broke build in general, not only in the > jhbuild, in the not so distant past, which was a shame and it was all > my fault. I'm sorry for that, though in that particular case the change > on the evolution-data-server side was correct, only the evolution build > broke, because it wasn't updated appropriately. The revert on the eds > side would not make any sense. It was correct, as I said. (API changed > in some way and I didn't realize that the evolution uses the API in a > way which breaks it. Just my fault.) > I think that is the point of the try server, right? You push it to the try server and make sure that it builds correctly for everyone and then you'll know whether something is broken or not before it get committed to master. Perhaps, reverting might seem like over kill, but I think we need to have a social contract to at least use the try server and make sure that it builds there before pushing it to master. After all, we all committed to a six month release cycle and this doesn't add too much additional overhead, and helps enables development across the stack without anybody getting stuck. A lack of diligence on your part for instance costs time and headache somewhere else. Just another perspective on the issue. > I do not fully understand why reverting in some random project (and > making sort of hostile environment) is easier than the current state, > when the same person "tags" what commit is supposed to be used in > Continuous in the jhbuild. You still need the person, the person cannot > be a monkey, it will think before doing anything (I mean, it's not a > mechanical job where one rule fits each case). > Nope and Emmanuele does stipulate that we need someone to monitor the build and act accordingly. However, the rest of you have an obligation when that person nags you about it to respond in a timely manner. > > I appreciate how Emmanuele and others (are there any others?) handle > the situation right now. I do not want to add them more work, really > not. > I think we all agree that there should be change, we just need to agree what is the minimal set of things we can accomplish today and then iterate there. We just need to establish a place where we can at least start on a path and break current status quo. > > Right, it's unrealistic in some cases to land all of that at one time. > Side branches is a nice idea, but it won't work, because you do not > have any influence on the other project maintainers usually, thus even > a bugzilla requests can lurk there for a long time. Having a side > branch only means that the maintainers whom do not follow particular > mailing list will notice only later, rather than sooner. > Well that's when we ask the release team for help. They are in my opinion currently under utilized for this kind of thing. In general, I really don't like the idea that we're all a set of fiefdoms with no influence on each other. We all here to move this platform forward. We should all be making sure that we support each other, because that support helps GNOME and moves our purpose forward. This shouldn't be about my little piece of soil with the myopic view that this is all I care about unless you are a contributor to the module. If I come to you with a build breakage issue then I expect a return courtesy because my request comes with the intent that we want to improve things. > > There is a bug to make more structures in Camel GObject based [2], to > be able to introspect it in a much easier way and so on. That change > will not be only a simple API change, it'll break core Camel things, > everything what uses it. If you open the [2], then I listed affected > projects at the end of the comment #5. It counts 18 projects. Maybe > there are more. I do not think I'd be able to coordinate the change in > a side branch for all of them, I (we) will surely provide patches in > the bugzilla around the time of the change landing, then it'll be up to > the respective maintainers to pick or reject them. What will the > Continuous do during this "broken period" is something I do not know. I > only know that the change will be for good. Introspection support for > the Mail part is good, I believe. Trying to revert such change in the > eds would hurt very much, no doubt. > I would say that is a matter of negotiation. If we are tolerating a full break in the builds, then be it. You schedule that time, and then give people a chance to test the patches
Re: Continuous Builds in GNOME
On Mon, Jun 06, 2016 at 10:10:58PM +0200, Sébastien Wilmet wrote: > There is a solution: bump the major version of Camel or EDS each time an > API or ABI break is unavoidable, making the new major version > parallel-installable with the previous ones. And that, every 6 months if > needed (or one year if the development cycle is longer for the > Evolution-related projects). Mmh maybe it's not possible for EDS, since it's a daemon (or contains a daemon) and thus only one instance can run on a system. I don't know much about how EDS, Camel etc are designed. -- Sébastien ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Continuous Builds in GNOME
On Mon, Jun 06, 2016 at 07:05:49PM +0200, Milan Crha wrote: > There is a bug to make more structures in Camel GObject based [2], to > be able to introspect it in a much easier way and so on. That change > will not be only a simple API change, it'll break core Camel things, > everything what uses it. If you open the [2], then I listed affected > projects at the end of the comment #5. It counts 18 projects. Maybe > there are more. I do not think I'd be able to coordinate the change in > a side branch for all of them, I (we) will surely provide patches in > the bugzilla around the time of the change landing, then it'll be up to > the respective maintainers to pick or reject them. What will the > Continuous do during this "broken period" is something I do not know. I > only know that the change will be for good. Introspection support for > the Mail part is good, I believe. Trying to revert such change in the > eds would hurt very much, no doubt. There is a solution: bump the major version of Camel or EDS each time an API or ABI break is unavoidable, making the new major version parallel-installable with the previous ones. And that, every 6 months if needed (or one year if the development cycle is longer for the Evolution-related projects). Distros might not like that, but packagers should be encouraged to drop old major versions if it is no longer used by any package. (As a more practical matter, instead of hardcoding the API/major version a bit everywhere in the build system, see how it is done at: https://developer.gnome.org/programming-guidelines/stable/parallel-installability.html.en with @LIBRARY_API_VERSION@ in the Makefile.am etc. That way the API version can be bumped easily). One of the initial goals of developing a shared library is to reduce memory consumption. But with systems like Flatpak, that goal is fading away: there will be a new GNOME runtime version every 6 months. And with today's computers, is it more important to save, say, 100MB of memory, or to fluidize the development and ease big refactorings? -- Sébastien ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Continuous Builds in GNOME
On 06/06/2016 01:02 PM, Christian Hergert wrote: > A couple weeks ago I wrote a quick hack to parse Continuous build logs > and extract CFLAGS for the built files. It can use this to then go > perform static analysis on the module with clang and extract/resolve all > function calls. And the missing link: https://github.com/chergert/sightline ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Continuous Builds in GNOME
On 06/06/2016 12:48 PM, Michael Catanzaro wrote: > I agree with you that when landing a big API change, if you don't want > to use side branches, then a revert is less preferable to tagging in > Continuous and branching in jhbuild. (In such cases, it'd be great if > you could handle that before pushing the API change, though. :) A couple weeks ago I wrote a quick hack to parse Continuous build logs and extract CFLAGS for the built files. It can use this to then go perform static analysis on the module with clang and extract/resolve all function calls. The goal was to be able to tell us which API are used the most (and therefore requires our attention to documentation), but we could also use it to tell developers everywhere in g.g.o that their API is being used. I expect to build this out as I work more on documentation stuff for Builder. -- Christian ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Continuous Builds in GNOME
On Mon, 2016-06-06 at 19:05 +0200, Milan Crha wrote: > Right, it's unrealistic in some cases to land all of that at one > time. > Side branches is a nice idea, but it won't work, because you do not > have any influence on the other project maintainers usually, thus > even > a bugzilla requests can lurk there for a long time. Having a side > branch only means that the maintainers whom do not follow particular > mailing list will notice only later, rather than sooner. Hi, If you have a patch that e.g. adapts another module to your API change, but the relevant maintainers are not responsive on Bugzilla, you should feel free to just push ahead with your changes. If you're not comfortable pushing to other modules, you can contact the release team and we'll push your work for you. We understand this happens sometimes; everyone's busy, but nobody wants your work to get blocked in such cases. (I remember recently we had this issue with e-d-s and california, although california is neither in jhbuild nor continuous, so we're not going to nag you if it breaks.) I agree with you that when landing a big API change, if you don't want to use side branches, then a revert is less preferable to tagging in Continuous and branching in jhbuild. (In such cases, it'd be great if you could handle that before pushing the API change, though. :) Michael ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Continuous Builds in GNOME
On Mon, 2016-06-06 at 09:49 -0500, Michael Catanzaro wrote: > A revert is not supposed to be "punishment" in any way... rather, > consider it as assistance to make sure GNOME stays buildable. :) Hi, maybe it's not supposed to be, but it is in my eyes. I try my best to not break builds, I'm grateful when other folks let me know when the builds are broken for them. Neither jhbuild catches everything [1], which you seem to suggest. I broke build in general, not only in the jhbuild, in the not so distant past, which was a shame and it was all my fault. I'm sorry for that, though in that particular case the change on the evolution-data-server side was correct, only the evolution build broke, because it wasn't updated appropriately. The revert on the eds side would not make any sense. It was correct, as I said. (API changed in some way and I didn't realize that the evolution uses the API in a way which breaks it. Just my fault.) I do not fully understand why reverting in some random project (and making sort of hostile environment) is easier than the current state, when the same person "tags" what commit is supposed to be used in Continuous in the jhbuild. You still need the person, the person cannot be a monkey, it will think before doing anything (I mean, it's not a mechanical job where one rule fits each case). I appreciate how Emmanuele and others (are there any others?) handle the situation right now. I do not want to add them more work, really not. > we can create a list of maintainers who don't want this Nah, that's one more place to look at and to forget about. That adds work to those nice folk(s) whom take care of the Continuous. > In this case, when the API change breaks something in core or gnome- > apps, then the module and its dependencies really need to be updated > in jhbuild at (approximately) the same time, so there's at most a > small window of time where jhbuild is broken. > ... Right, it's unrealistic in some cases to land all of that at one time. Side branches is a nice idea, but it won't work, because you do not have any influence on the other project maintainers usually, thus even a bugzilla requests can lurk there for a long time. Having a side branch only means that the maintainers whom do not follow particular mailing list will notice only later, rather than sooner. There is a bug to make more structures in Camel GObject based [2], to be able to introspect it in a much easier way and so on. That change will not be only a simple API change, it'll break core Camel things, everything what uses it. If you open the [2], then I listed affected projects at the end of the comment #5. It counts 18 projects. Maybe there are more. I do not think I'd be able to coordinate the change in a side branch for all of them, I (we) will surely provide patches in the bugzilla around the time of the change landing, then it'll be up to the respective maintainers to pick or reject them. What will the Continuous do during this "broken period" is something I do not know. I only know that the change will be for good. Introspection support for the Mail part is good, I believe. Trying to revert such change in the eds would hurt very much, no doubt. > Yeah, of course runtime bugs are problems, but not the problem we are > trying to solve here. :) Heh, true, though the runtime bugs are more important. I believe. I know, that's one reason why you want to have the jhbuild working, to make it easier for the contributors to test and develop. Which is a good thing for everyone. Bye, Milan [1] https://bugzilla.gnome.org/show_bug.cgi?id=766540 [2] https://bugzilla.gnome.org/show_bug.cgi?id=764065 ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Continuous Builds in GNOME
Hi, A revert is not supposed to be "punishment" in any way... rather, consider it as assistance to make sure GNOME stays buildable. :) But if you really don't like it and don't want other folks reverting your commits when you're not around, we can create a list of maintainers who don't want this; that way, on the rare occasions when something breaks, we can branch your module in jhbuild and tag it in Continuous instead of reverting your change. I think it's generally better to avoid branching/tagging, so I'd encourage maintainers to just accept the occasional revert now and then, but it's understandable if we won't all agree on this. On Mon, 2016-06-06 at 11:01 +0200, Milan Crha wrote: > - a soname version bump can break dependant projects. Such change > should not be reverted, the dependencies should be rebuilt. Right; incremental builds with jhbuild are not completely reliable, and a soname bump is generally not a valid reason to revert a commit. But if a clean build doesn't work, then we have a problem to be fixed. > - a soname version bump with an API change can break dependant > projects. Such change should not be reverted, the dependencies > should be rebuilt and adapted to the change. In this case, when the API change breaks something in core or gnome- apps, then the module and its dependencies really need to be updated in jhbuild at (approximately) the same time, so there's at most a small window of time where jhbuild is broken. Of course it can take time to port dependencies; in such cases, we can either (a) make the API change on a side branch, port dependencies on side branches, and merge the changes to master when all GNOME modules are ready; or (b) make the API change on master, but branch the module in the jhbuild moduleset so that other modules continue to build against the older version of the module with the API change. Really, you can do whatever you want, so long as you avoid breaking jhbuild or Continuous, please! > - a change which can build, but causes regressions in runtime. Such > change should be reverted, right? But wait, you care only about > the build, not about runtime. Having a software buildable and > having the same software usable are two very different things. Yeah, of course runtime bugs are problems, but not the problem we are trying to solve here. :) Hope that clarifies where I'm coming from. Michael ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Continuous Builds in GNOME
On 03/06/2016 17:26, Sriram Ramkrishna wrote: > Yes, but it highlights the value something like GitHub provides. What > about GitLab? (BTW I'm not advocating we move, just looking to see > alternatives) GitLab while open source, does not seem to open source > its CI work that I can tell. Still, maybe a deal can be made.. It's all open-source: https://about.gitlab.com/gitlab-ci/ Ciao, Alberto ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Continuous Builds in GNOME
Hey Milan, | The proposal about random reverts makes me feel that you want to flip | the positions. While I agree that the maintainers point of view is | broken, the position flip just means (for me): "the GNOME | infrastructure/jhbuild environment is the only good build environment | and if the maintainer breaks it, then the GNOME infrastructure team can | punish the maintainer if it needs to". Do you see how absurd it sounds? I see what you meant, but I think you are looking at it with "bad eyes". It is not punishment, is just that should be a requirement that the master branch is buildable on continuous and jhbuild, and the only way to do so it's reverting the breaking change. In the meantime the maintainer can do whatever he wants in a branch and wait until the he/she figures out the build problem before merging to master again. | The breakage should be dealt with in a cooperation manner, rather than | in a force manner. Of course, if the maintainer is not willing to | cooperate..., but I hope that's only a minority of the GNOME-hosted Indeed, and I think Emmanuelle acts in the right way, trying to reach the maintainer, and if he/she is not available, just reverting with a commit message explaining the reason or fixing it temporarily so everyone can still build the module. I think this is more about that bad feeling of someone else not involved in the module touching "our" project without even asking to us the maintainers, and I also have that feeling, however I think it's more important that everyone can build our modules. Otherwise we are going to make worse the already difficult experience of building GNOME. Cheers, Carlos Soriano ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Continuous Builds in GNOME
On Fri, 2016-06-03 at 08:28 -0500, Michael Catanzaro wrote: > I expect module maintainers to be understanding when reverts happen. > It's not the end of the world; you can land your change again as soon > as you figure out why it was broken. We can all live with a few > revert commits in our git history. Your module can still be your > fiefdom... just so long as it's still building with the rest of > GNOME. Hi, I'm not a fan of the random reverts from some 3rd party person whom doesn't know the "bigger picture" of the project changes which can lead to a win-win state, even the changes are that large that it's better (for whatever reason) to split them into several commits, which not necessarily follow in a short time. Such reverts would mean more work for the maintainers, which is always a pita. I hear here an argument that the maintainers claim that "it builds on their machine" and they do not care of the other build environments. Personally, I do not understand this statement, one (the maintainer) should be willing to support his project home, appreciate that the project can use the infrastructure and be tested in more places, which is always a good thing. The proposal about random reverts makes me feel that you want to flip the positions. While I agree that the maintainers point of view is broken, the position flip just means (for me): "the GNOME infrastructure/jhbuild environment is the only good build environment and if the maintainer breaks it, then the GNOME infrastructure team can punish the maintainer if it needs to". Do you see how absurd it sounds? The breakage should be dealt with in a cooperation manner, rather than in a force manner. Of course, if the maintainer is not willing to cooperate..., but I hope that's only a minority of the GNOME-hosted projects (I do not know any numbers here) and definitely the core project maintainers are all good, I believe, thus there might not be any issue. The "bad" projects, if not from the core part, can be skipped in the Continuous builds due to the lack of the interest from the maintainers side. There had been discussed also what can be reverted and what not. If you recall: - a soname version bump can break dependant projects. Such change should not be reverted, the dependencies should be rebuilt. - a soname version bump with an API change can break dependant projects. Such change should not be reverted, the dependencies should be rebuilt and adapted to the change. - a change which can build, but causes regressions in runtime. Such change should be reverted, right? But wait, you care only about the build, not about runtime. Having a software buildable and having the same software usable are two very different things. I suppose the GNOME build system runs some unit tests, if the module provides such, but such tests are testing the new code, not the regressions the change in the module can cause in another module which had been/will be built against it. Just my opinion on the matter. Bye, Milan ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Continuous Builds in GNOME
On Fri, Jun 3, 2016, 01:42 Emmanuele Bassi wrote: > > Right now, the easiest and cheapest option would literally be moving > the GNOME development infrastructure wholesale to GitHub, put > everything under Travis CI, and keep a separate machine somewhere that > cranks out GNOME Continuous images from the GitHub repositories. For > reasons that you may guess, this is not going to be a very good move. > Any other option involves replicating that set up on gnome.org > infrastructure, with all the issues that it entails. > We already have mirrors of all the Gnome repositories on GitHub, and I believe you could run Travis CI off of those without shifting the entire primary infrastructure to GitHub. Travis CI isn't free-as-in-speech, but as far as I know there is no lock-in. Maybe having such a setup as a temporary measure would 1) visibly demonstrate to module maintainers why it's useful and necessary, and therefore help to get people out of the "it builds on my machine" mindset; and 2) inspire people to replicate it on Gnome infrastructure out of free components. That task would probably seem less daunting if there was a familiar example to follow. Travis will hardly be ideal either though, because I believe we'd have a hard time getting dependencies of our bleeding edge modules onto their Ubuntu 12.04 and 14.04 VMs... Regards, Philip C > ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Continuous Builds in GNOME
On Fri, Jun 3, 2016 at 1:42 AM Emmanuele Bassi wrote: > Hi Sri; > > On 3 June 2016 at 02:53, Sriram Ramkrishna wrote: > > I found this discussion really fascinating and so I wanted to continue > it, > > separately from Emmanuele's thread so that issue is resolved without > > bifurcating the discussion. > > > > My thoughts are that we really shouldn't be looking at something and say > > 'well, we can't do it we don't have the resources' especially when there > is > > a clear return on investment. I think this would be an interesting > > challenge and we should find a way to figure out how we can attract the > > talent, machines and money to do this. After all, we have a foundation > > dedicated to supporting the development of GNOME. > > Can the Foundation hire a full time "build team" that: > > * keeps the build machines running > * keeps the build running > * works on the necessary tooling to integrate our Git repositories > activity with our mailing list, our bug tracking system, and possibly > our IRC channels > I don't know, right now, probably not. But we could attract some devops talent, and help support the initiatives and Andrea Veri and Patrick Uiterwijk have been doing to attractive them. We only a part time person for the moment and even that has given us a significant benefit. But as you allude to, even with the above we have a social problem, so before we can even do that we need to figure out how to get maintainers to participate into the build system. In this day and age, most developers accept the fact of using a try server and not merging code that doesn't build. > > The current situation can barely run on volunteers time, and I think > it's time to be pretty clear on what our requirements are — instead of > architecture astronauting our way out. > Sure. > > > Right now, the easiest and cheapest option would literally be moving > the GNOME development infrastructure wholesale to GitHub, put > everything under Travis CI, and keep a separate machine somewhere that > cranks out GNOME Continuous images from the GitHub repositories. For > reasons that you may guess, this is not going to be a very good move. > Any other option involves replicating that set up on gnome.org > infrastructure, with all the issues that it entails. > Yes, but it highlights the value something like GitHub provides. What about GitLab? (BTW I'm not advocating we move, just looking to see alternatives) GitLab while open source, does not seem to open source its CI work that I can tell. Still, maybe a deal can be made.. > > Incidentally, if you find somebody like that, good luck not getting > them poached by companies that want the exact same thing, and are > likely to pay more than the GNOME Foundation for it. > > Actually, I was going to use this argument to find me talented, but out of work systems administrators who for whatever reason is not able to find a job. I know one such right now, and I've worked with him for 20 years. It is completely within his skill set. If we can find someone who can actually implement it then we have at least some chance. > > Still, people consider Continuous somebody else's problem; if it > "breaks on Continuous" but not on a maintainer machine, then who > cares? > Then we should change where Continuous fits in the scheme of things by asking the release team to have weekly builds from Continuous. We should consider everything on the table. > > Even if we magically got the resources (build machines, at least one > person working on the infrastructure side, volunteer work to improve > the tooling), the attitude of "my module is my fiefdom, if a build > breaks *you* fix it" has to change. GNOME has long since become an > interconnected, complex system, and it requires a level of oversight > that does not map with "a loosely coupled set of components", with > each one of them that may or may not build; may or may not pass tests; > may or may not break other components; may or may not lead to a > bootable, usable system. > Well, we've done a lot of social changes, bringing in designers working with developers, all firsts in a Free Software project. It is clear we can make social changes when we need to. Since 2011, we've already started to think of ourselves as a product, and so the idea that some people think we are still a loosely coupled set of components is old way thinking and they need to 'git pull' on their thinking. > > So, more than a *technical* issue (as usual), this is a *social* > issue. People have to care about this stuff — and not just a couple of > people getting Continuous running. > > Well, I think to expand on that, all of us need to think about the big picture. We have an entire foundation set up to drive development. But I think there is a disconnect between development (development, translation, and documentation) and the foundation/money/engagement. People don't realize that everything is connected. Perception management, publi
Re: Continuous Builds in GNOME
On Fri, 2016-06-03 at 09:42 +0100, Emmanuele Bassi wrote: > Even if we magically got the resources (build machines, at least one > person working on the infrastructure side, volunteer work to improve > the tooling), the attitude of "my module is my fiefdom, if a build > breaks *you* fix it" has to change. GNOME has long since become an > interconnected, complex system, and it requires a level of oversight > that does not map with "a loosely coupled set of components", with > each one of them that may or may not build; may or may not pass > tests; > may or may not break other components; may or may not lead to a > bootable, usable system. Based on the last time we discussed this, I'm pretty sure this view is passionately-held by a very small minority of contributors. Yes it's a social issue, but also a technical one with bearing on our ability to onboard new contributors and the productivity of existing contributors. I support a policy that all contributors may revert a patch on master if it breaks Continuous, and I suspect the rest of release team would as well, as we're tired of dealing with jhbuild issues on release day. master should always be buildable. I expect module maintainers to be understanding when reverts happen. It's not the end of the world; you can land your change again as soon as you figure out why it was broken. We can all live with a few revert commits in our git history. Your module can still be your fiefdom... just so long as it's still building with the rest of GNOME. I would also like us to stop tagging modules in Continuous in general, instead revert commits that introduce build breakage in the modules themselves, and save tagging for very rare cases where reverts are not appropriate. Of course, sometimes we may need a transition in which master of some module is broken for a brief period while somebody is actively working on fixing it (in a matter of hours, not days). Such issues are reasonable if they're rare. But in general, everyone should feel empowered to keep our modules building. Michael ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Continuous Builds in GNOME
Hi Emanuelle, I just wanted to pitch into this discussion and say thanks to you (and everyone else working on this) for putting all the effort into fixing build issues and improving continuous integration. I have been a developer for different gnome components for a while, but I am not such a frequent contributor that I know all the internals of most core components. So when a module breaks during a jhbuild, then I am often quite lost and need a few hours of google, asking people at #gtk or #gnome as well as trial and error. Most of the time I just want to "quickly get a fairly recent build" and then "write a small patch" to fix something in one of the gnome modules. When that happens I may be able to set aside a few hours or half a day but not a whole week. However most times this half day is wasted battling with jhbuild and trying to get a working environment that not only builds glib but which actually comes as far as to build the module for which I wanted to write a patch in the first place. When I have already spend 4 hours and jhbuild is just building module 20 out of 90, and breaking on some unresolvable dependency, missing library, some obscure configure or make error or what ever, then this is usually the point were I just go and do something else. So again, all your effort its highly appreciated. I really hope that at some point in the future its possible to just type `jhbuild build` then go away to work on something else when then come back to find the build finished and login to the ready jhbuild session. If on top of that this would work not only on the most recent fedora "from last week" or so, but on the most recent Ubuntu _LTS_ or Debian _stable_, then that would be really awesome. And it would mean that I could spend more time testing gnome modules, fixing bugs and help to improve gnome rather then spending days trying to get all modules building. Thanks and Cheers Sebastian On 03/06/16 10:42, Emmanuele Bassi wrote: Still, people consider Continuous somebody else's problem; if it "breaks on Continuous" but not on a maintainer machine, then who cares? Even if we magically got the resources (build machines, at least one person working on the infrastructure side, volunteer work to improve the tooling), the attitude of "my module is my fiefdom, if a build breaks*you* fix it" has to change. ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: Continuous Builds in GNOME
Hi Sri; On 3 June 2016 at 02:53, Sriram Ramkrishna wrote: > I found this discussion really fascinating and so I wanted to continue it, > separately from Emmanuele's thread so that issue is resolved without > bifurcating the discussion. > > My thoughts are that we really shouldn't be looking at something and say > 'well, we can't do it we don't have the resources' especially when there is > a clear return on investment. I think this would be an interesting > challenge and we should find a way to figure out how we can attract the > talent, machines and money to do this. After all, we have a foundation > dedicated to supporting the development of GNOME. Can the Foundation hire a full time "build team" that: * keeps the build machines running * keeps the build running * works on the necessary tooling to integrate our Git repositories activity with our mailing list, our bug tracking system, and possibly our IRC channels The current situation can barely run on volunteers time, and I think it's time to be pretty clear on what our requirements are — instead of architecture astronauting our way out. We don't need machines; those are "easy" to get. We need actual people that would fit the bill of a "devops" team; we need people that can install and maintain a complex build system; that set up a Continuous Integration process on top of our existing Continuous Delivery one; and that can fill in the blank spots on the map between "this branch on this module failed to build" and "this is breaking the whole GNOME please fix it". Right now, the easiest and cheapest option would literally be moving the GNOME development infrastructure wholesale to GitHub, put everything under Travis CI, and keep a separate machine somewhere that cranks out GNOME Continuous images from the GitHub repositories. For reasons that you may guess, this is not going to be a very good move. Any other option involves replicating that set up on gnome.org infrastructure, with all the issues that it entails. Incidentally, if you find somebody like that, good luck not getting them poached by companies that want the exact same thing, and are likely to pay more than the GNOME Foundation for it. > We have an amazing infrastructure that we can attract possible very smart > people of whatever age to come and work on for the betterment of GNOME and > of course their own careers. Colin Walters presented Continuous to the GNOME community in 2012 — and it was already cranking out GNOME builds. Apart from a flurry of activity in the beginning, I have not seen much interest being attracted to the actual tools; for a long while, people didn't even care much about the builds — and to be fair, why would they? It works on *their* machine and maybe they can even release tarballs every once in a while, and if it breaks patches welcome, right? Well, patches started to arrive, and at the end we got a much better experience for newcomers, with jhbuild not constantly breaking. Still, people consider Continuous somebody else's problem; if it "breaks on Continuous" but not on a maintainer machine, then who cares? Even if we magically got the resources (build machines, at least one person working on the infrastructure side, volunteer work to improve the tooling), the attitude of "my module is my fiefdom, if a build breaks *you* fix it" has to change. GNOME has long since become an interconnected, complex system, and it requires a level of oversight that does not map with "a loosely coupled set of components", with each one of them that may or may not build; may or may not pass tests; may or may not break other components; may or may not lead to a bootable, usable system. So, more than a *technical* issue (as usual), this is a *social* issue. People have to care about this stuff — and not just a couple of people getting Continuous running. > I just hate when we say we don't have resources when we can't quantify what > we have and we aren't quantifying what we need. I mean, yes, it is > difficult and hard right now, and so we need to make strategic plans to fix > it especially if it is a man power problem. Our development, our > engagement, the board, everything is connected and so we need to look at it > from all angles. > > I really love the ideas of try servers, and you know they have this some how > in github, and man, we should try to find clever means to do the same thing. Sometimes there are no "clever" options, just the old dumb ones. Ciao, Emmanuele. > -- Forwarded message - > From: Emmanuele Bassi > Date: Tue, May 31, 2016 at 9:51 AM > Subject: Re: Enabling builddir != srcdir by default in jhbuild > To: Michael Catanzaro > Cc: Desktop Development List > > > Hi; > > I already pushed the default change to master, as that will only > affect new clones or updates. I'm also building locally the default > gnome moduleset — but I can safely say that the core platform builds > fine. I'm just worried about gnome-world, but for that I
Re: Continuous Builds in GNOME
I also think it is sometimes important to discuss about the big picture, even if it is just pipe dreams. On Fri, Jun 03, 2016 at 01:53:28AM +, Sriram Ramkrishna wrote: > I really love the ideas of try servers, and you know they have this some > how in github, and man, we should try to find clever means to do the same > thing. >From my naive point of view, a try server would be more complicated to implement and integrate into the gnome.org infrastructure, and it would be a big change for contributors and maintainers. When doing such changes, the biggest challenge comes from the people, not from the technical aspect of it. My (again, naive) proposal with the master/master-stable-next/master-stable branches [1] seems easier to implement to me and the advantage is that contributors and maintainers don't need to change how they work. Of course a try server would be nice too, but it can be done separately. [1] The more I think about it the more it becomes complicated, I know the Devil is in the details (or God is in the details. Whatever is in the details it is something important). -- Sébastien ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Continuous Builds in GNOME
I found this discussion really fascinating and so I wanted to continue it, separately from Emmanuele's thread so that issue is resolved without bifurcating the discussion. My thoughts are that we really shouldn't be looking at something and say 'well, we can't do it we don't have the resources' especially when there is a clear return on investment. I think this would be an interesting challenge and we should find a way to figure out how we can attract the talent, machines and money to do this. After all, we have a foundation dedicated to supporting the development of GNOME. A more interesting discussion would be a practical set of criteria needed to implement a reasonable version that we can evolve over time and then convert that into something that GNOME Foundation directors and engagement team members can work to figure what needs to be done. We have an amazing infrastructure that we can attract possible very smart people of whatever age to come and work on for the betterment of GNOME and of course their own careers. I just hate when we say we don't have resources when we can't quantify what we have and we aren't quantifying what we need. I mean, yes, it is difficult and hard right now, and so we need to make strategic plans to fix it especially if it is a man power problem. Our development, our engagement, the board, everything is connected and so we need to look at it from all angles. I really love the ideas of try servers, and you know they have this some how in github, and man, we should try to find clever means to do the same thing. sri -- Forwarded message - From: Emmanuele Bassi Date: Tue, May 31, 2016 at 9:51 AM Subject: Re: Enabling builddir != srcdir by default in jhbuild To: Michael Catanzaro Cc: Desktop Development List Hi; I already pushed the default change to master, as that will only affect new clones or updates. I'm also building locally the default gnome moduleset — but I can safely say that the core platform builds fine. I'm just worried about gnome-world, but for that I guess we'll have to wait until stuff breaks. Ciao, Emmanuele. On 31 May 2016 at 17:47, Michael Catanzaro wrote: > On Mon, 2016-05-30 at 23:44 +0100, Emmanuele Bassi wrote: >> So, it seems that the discussion died on these shores. >> >> In the meantime, GVfs is but the latest module that broke because >> people don't test under builddir != srcdir; I really, *really* don't >> want to deal with this kind of perfectly avoidable build breakages >> any >> more. >> >> Ciao, >> Emmanuele. > > Emmanuele, I think you can feel free to change the default in jhbuild > provided that everything in the apps and core suites still builds after > doing so. i.e. you need to make sure to add exceptions in the jhbuild > modulesets for all modules that need it. > > Just please wait a couple days first to see if there are any > substantial objections (which I do not expect). > > Michael -- https://www.bassi.io [@] ebassi [@gmail.com] ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
