Hi Matthias, I realize now that this was too much information at once (even for the involved reader as opposed to a fly-by reader).
So I'd like to thank you for your mind share. On Wed, 2017-04-26 at 16:39 -0400, Matthias Clasen wrote: > Tristan, > > again, it is impossible to reply to an email of this length. I can > only give a few general comments, beyond that, we really need to sit > down face-to-face and discuss this. I hope you are going to be at > Guadec ? I will certainly be around all week at GUADEC to meet with you and anyone who wants to discuss :) I am preparing a talk on this subject, but perhaps I should also try to organize something more hands on, maybe a BoF or such would be good. > My general comments: > > What you are describing here (and in your previous communications) > looks like a big, all-encompassing system, with lots of its own > terminology and a complete worldview of how things should be built. I > prefer a system that starts small and solves one problem initially, > and then maybe grows over time. I can see how it can come across this way, we are trying to break the trend of having a build system as something that is tied to any particular deployment/use case. As such, I needed to give consideration to a lot of use cases to be sure that this is something that fits, and is also an improvement over what exists. These considerations will be reflected in my communications and I can see how one might think this appears to be some kind of huge monolith which does everything. However this is exactly the opposite of what I'm trying to achieve, instead we are striving to "do one thing well" and are making an effort to ensure we're doing it the right way, for any use case. So, the core codebase itself should remain small over time, with really the sole purpose of being: "A format and engine for modeling and executing pipelines of elements which perform mutations on filesystem data within an isolated sandboxed environment" In time, I expect that an ecosystem of plugins and projects will grow around this, and use cases I had not even foreseen will come to light. This has already started to happen in some ways, as Jochen Breuer's commented on my blog here: https://blogs.gnome.org/tvb/2017/02/06/introducing-buildstream-2/ And as a result has started to work on a plugin which would allow importing distro packages and building on top of those bases: https://gitlab.com/BuildStream/buildstream/issues/10 > The system you describe seems to be all about centralization, and > about introducing a new language to describe what we build. That is > by-and-large what we already have in various incarnations that you > describe: jhbuild modulesets, the continuous manifest, flatpak > runtimes. I can get behind the idea of unifying these into a single > way of describing a multi-module build. > > But I've also seen things mentioned like 'conversion scripts for > flatpak'. And I think that is exactly the opposite of what we need > for application building. I may be mistaken, but I have a feeling you are getting the same impression which Christian had last month, which I tried to explain in this email: https://mail.gnome.org/archives/desktop-devel-list/2017-March/msg00003.html > If we want to convince 3rd party applications to use flatpak, we > can't treat it as an intermediate format that we generate from some > other source, and just use somewhere in a centralized build process. > We need to embrace it ourselves and use it, just like we expect 3rd > party applications to use it. So at the risk of being repetitive, I am completely behind application authors maintaining their own build metadata themselves, building flatpaks themselves and/or submitting build metadata to a "flathub" to have them built and published to users. Of course this makes sense, because the application authors themselves are usually best situated to know what should be in their bundling build metadata. So let me try to break down how I would see this work (I realize, already a long email): GNOME core modules and services (excluding Flatpak apps) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ These can be expressed in a single format/repository for all the interesting purposes: o Performing CI o Creating bootable GNOME images on top of some base system, mostly for developers o Release modulesets o Producing the GNOME Flatpak runtime and SDK This is of course, one centralization. Flatpak Applications ~~~~~~~~~~~~~~~~~~~~ Considering the benefits which the GNOME core modules and services get by representing build metadata for multiple purposes, a given application developer team may also benefit in similar ways. This is without centralizing everything into one big build blob, or using intermediate formats or anything like this. Now, this may be more of a personal goal, a bit more ambitious, maybe best punted to later on, but consider that for instance: many applications still today have OS specific subdirectories where build metadata in various formats are used to create application bundles for OSX and win32. Also, I'm not exactly rooting for "snap" here, but it may very well be that an application developer who wants to deploy a Flatpak bundle, may also want to deploy a "snap". Now, even if the arguments above for using BuildStream *instead* of flatpak-builder to build apps never bear fruit (maybe win32 bundles are so immensely different that there is little or no gain to representing those builds in the same tree ?), it would still be a net win to maintain one less build system (and this is where conversion scripts for me become interesting, as an easy migration path, not an "intermediate format"). I realize that Alex has already made a statement that flatpak-builder will continue to be supported and maintained, possibly indefinitely, but I wont lie about this: I really want to save developer hours. If it is possible that we can reuse the same body of work to build many things, and the result is a reduced maintenance burden on the Flatpak developers, I would much prefer to see valuable developer hours focused on how Flatpak actually works, than diverted into menial tasks like getting builds to work. I hope this response clears up some things for you. And thanks again for taking the time to reply and sharing your considerations, despite the huge size of my initial email :) Best Regards, -Tristan _______________________________________________ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list