Re: For projects switching to Meson *only*
On Thu, Apr 27, 2017 at 7:00 PM, Peter Huttererwrote: I will, but I'll keep the two parallel for at least a release or two. If you need me to add anything specific to keep continuous happy for the time being, let me know. Cheers, Peter Here's a request for maintainers who are supporting both meson and Autotools: please consider adding the meson build files to your release tarballs (add them to EXTRA_DIST) if you want people to actually use the meson build system. You don't have to, but it'd be nice. For people helping maintain the jhbuild modulesets, please do not switch modules to use meson until the meson build files have appeared in a release tarball, or you're confident they will appear in the next tarball. (Unless Autotools support has already been dropped, in which case maintainers, please follow the GNOME release cycle in making your next release!) This will reduce the amount of manual hacking required to produce release modulesets that actually build. I've temporarily switched GStreamer and Grilo back to using Autotools for this reason. Thanks, Michael ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: For projects switching to Meson *only*
On Thu, Apr 27, 2017 at 03:24:24PM +0100, Emmanuele Bassi wrote: > Replying to both Michael's and Iain at the same time, to reduce the > amount of email. > > On 27 April 2017 at 15:04, Michael Catanzarowrote: > > On Thu, Apr 27, 2017 at 4:11 AM, Iain Lane wrote: > >> > >> As it happens I interacted with this script (in gnome-software) > >> yesterday. I hadn't got a new enough jhbuild - the one I had was trying > >> to call ./configure instead of meson directly. > >> > >> The script AFAICS ignores unknown arguments. In particular, I was > >> interested in passing some --enable/--disable flags to select features > >> but I didn't find out how to do that short of explicitly extending it > >> with those particular options. If you expect distributors to be using > >> this, or if this is interesting for users of the build API, is having > >> some support for ./configure <-> meson_options integration a reasonable > >> request? > > Each module has its own set of options with their own name and > semantics; the build-api only defines a specific subset, so if you > want to have a `configure` script to paper over the Meson switch, you > also get to add the options you want to port over. > > Mostly because if I end up patching this script into Continuous, then > I get to be the one who decides which options get ported over and > which one don't; maintainers of a project are usually best suited at > that. > > For instance, the libinput maintainer has started dropping all > auto-detection checks and now the build relies on specifying all > options; a worthy goal, and I'd actually hope more modules followed > suit. If he also switched to Meson-only, I'd have to write a patch for > Continuous that manually ported over the build options as a > compatibility layer; I would do this only for the options Continuous > supports, though, and it would take me slightly more time than just > copying the file over. I will, but I'll keep the two parallel for at least a release or two. If you need me to add anything specific to keep continuous happy for the time being, let me know. Cheers, Peter > > >> Otherwise, and this is what happens now that I upgraded jhbuild, using > >> meson directly works fine. But if a goal of this is to smooth the > >> transition path and avoid a requirement for tooling to be updated, maybe > >> it would be useful. > > Of course. > > > This compatibility issue seems like a very good argument for shipping the > > "compatibility" script in Continuous patches rather than application > > repositories. > > As I said, this takes me (in the absolute simplest case) about 90 > seconds of my life, so I don't mind doing a patch. > > I'd be happier, though, if maintainers that are planning to drop > autotools also dropped me a line so that I don't wake up to a string > of failed builds and then have to figure out whether or not this was a > planned break, or just general CI failure. *Especially* if the > maintainer also fixes the jhbuild moduleset. > > So, I guess the real question is: communicate these changes > beforehand, instead of pushing to master and then going home without > looking at the explosion? > > Ciao, > Emmanuele. > > > -- > https://www.bassi.io > [@] ebassi [@gmail.com] > ___ > desktop-devel-list mailing list > desktop-devel-list@gnome.org > https://mail.gnome.org/mailman/listinfo/desktop-devel-list ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: For projects switching to Meson *only*
On Thu, Apr 27, 2017 at 9:24 AM, Emmanuele Bassiwrote: I'd be happier, though, if maintainers that are planning to drop autotools also dropped me a line so that I don't wake up to a string of failed builds and then have to figure out whether or not this was a planned break, or just general CI failure. *Especially* if the maintainer also fixes the jhbuild moduleset. So, I guess the real question is: communicate these changes beforehand, instead of pushing to master and then going home without looking at the explosion? Yes, and my apologies for failing to do this yesterday when I switched over Epiphany. I even left JHBuild broken. Maintainers, please don't do that. :) Michael ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: For projects switching to Meson *only*
Replying to both Michael's and Iain at the same time, to reduce the amount of email. On 27 April 2017 at 15:04, Michael Catanzarowrote: > On Thu, Apr 27, 2017 at 4:11 AM, Iain Lane wrote: >> >> As it happens I interacted with this script (in gnome-software) >> yesterday. I hadn't got a new enough jhbuild - the one I had was trying >> to call ./configure instead of meson directly. >> >> The script AFAICS ignores unknown arguments. In particular, I was >> interested in passing some --enable/--disable flags to select features >> but I didn't find out how to do that short of explicitly extending it >> with those particular options. If you expect distributors to be using >> this, or if this is interesting for users of the build API, is having >> some support for ./configure <-> meson_options integration a reasonable >> request? Each module has its own set of options with their own name and semantics; the build-api only defines a specific subset, so if you want to have a `configure` script to paper over the Meson switch, you also get to add the options you want to port over. Mostly because if I end up patching this script into Continuous, then I get to be the one who decides which options get ported over and which one don't; maintainers of a project are usually best suited at that. For instance, the libinput maintainer has started dropping all auto-detection checks and now the build relies on specifying all options; a worthy goal, and I'd actually hope more modules followed suit. If he also switched to Meson-only, I'd have to write a patch for Continuous that manually ported over the build options as a compatibility layer; I would do this only for the options Continuous supports, though, and it would take me slightly more time than just copying the file over. >> Otherwise, and this is what happens now that I upgraded jhbuild, using >> meson directly works fine. But if a goal of this is to smooth the >> transition path and avoid a requirement for tooling to be updated, maybe >> it would be useful. Of course. > This compatibility issue seems like a very good argument for shipping the > "compatibility" script in Continuous patches rather than application > repositories. As I said, this takes me (in the absolute simplest case) about 90 seconds of my life, so I don't mind doing a patch. I'd be happier, though, if maintainers that are planning to drop autotools also dropped me a line so that I don't wake up to a string of failed builds and then have to figure out whether or not this was a planned break, or just general CI failure. *Especially* if the maintainer also fixes the jhbuild moduleset. So, I guess the real question is: communicate these changes beforehand, instead of pushing to master and then going home without looking at the explosion? Ciao, Emmanuele. -- https://www.bassi.io [@] ebassi [@gmail.com] ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME Build situation and BuildStream
On Thu, 2017-04-27 at 14:41 +0200, Sébastien Wilmet wrote: > Hi Tristan, > [...] > With jhbuild, when we enter into a jhbuild shell we are still in the > same directory, usually inside the git repository. With builddir == > srcdir we have all the files that we can directly open with our > preferred text editor. When we open a new terminal tab, we are in the > same directory where we are able to 1) do git commands 2) building > (with > recursive make) 3) launching executables 4) editing files, etc. > > With BuildStream, will it be similar? Hi Sébastien, Good question :) There are some things which will inevitably be different. I think the most disruptive thing is that you will not have the experience of having a single, persistent filesystem tree where the things you've built "are". This is because BuildStream does not have a serial build model but rather will parallelize builds where possible; every build result is stored in a separate "artifact", and sandboxed environments are created on demand. So, first of all to talk about VMs, launching a full VM is the preferred way to: o Test how some software interacts in a full GNOME environment, usually the bleeding edge of development. o Work on modules like GNOME Shell, GDM, GNOME Session etc which is very difficult to isolate and work on in your host environment. That said, today BuildStream has a `bst shell` option to stage a given module's dependencies in a sandbox and run shell on demand. There are two semantics for this, first of all let's assume that you have a checkout of the GNOME build metadata (or "modulesets"), your current working directory is at the root of that checkout, and the module you want to hack on is called "foo". bst shell --scope build foo.bst ~~~ Stage all of the build time dependencies for the module `foo.bst`, and also stage the sources for building `foo.bst`, and drop you into a shell in the build directory. Useful for debugging failed builds (however, when a build fails you will be presented an option to shell into the actual failed build sandbox anyway). bst shell --scope run foo.bst ~ Stage all of the runtime dependencies for the module `foo.bst`, including the built `foo.bst` module itself, and drop you into a shell in that sandboxed environment. Useful for debugging applications to a certain degree, can run gdb and strace and similar things in here. This shell differs from the actual build sandbox because it allows some pass through of the host environment variables. This makes it possible to launch graphical applications like say, gedit. This will _only_ work well on systems which have a somewhat conforming environment, i.e. your host should be running dbus and you should have DBUS_SESSION_BUS_ADDRESS set in your environment, similarly you want to have DISPLAY in your environment. So essentially, launching graphical applications inside `bst shell --scope run foo.bst` should work only for the cases that it would have worked when using jhbuild, so no loss there really. Now that part is already working, and dont worry about speed; even if hundreds of "artifacts" need to be staged into a directory, this is lightning fast and uses hardlinks to get it done. But what you will be more interested in is your edit/compile/debug cycles, for that we have a hard blocker before BuildStream can be really efficient for the type of workflow you want; we're calling this "Workspaces"[0]. With workspaces, you will be able to use a directory of your choosing to build a specific module from (and you can have more than one "active workspace" at a time, so you might open a workspace to hack on glib, and another one to hack on GTK+, and have your local trees for both be effective for any builds). This is not done yet but here's an approximate mock shell of what I think the UX would look like: # First get a local copy of the modulesets host$ git clone host$ cd gnome-modulesets # Now lets create some workspaces host$ bst init-workspace glib.bst ~/glib host$ bst init-workspace gtk+.bst ~/gtk+ # Open your favorite text editor, and edit # files directly in ~/glib and/or ~/gtk+ # # Now build something, maybe we want to just test with gtk3-demo host$ bst build gtk+.bst # Lets test it host$ bst shell --scope run gtk+.bst # We're in the sandbox now sandbox$ gtk3-demo # Hmmm, why did it crash ? sandbox$ gdb gtk3-demo # Ah, I see what I did there... sandbox$ exit # Edit some files in ~/glib and/or ~/gtk+ and try again # host$ bst build gtk+.bst host$ bst shell --scope run gtk+.bst sandbox$ gtk3-demo sandbox$ exit # Ok that worked ! host$ cd ~/gtk+ host$ git commit -a -m "Its a patch !" # Do appropriate thing, maybe you push, maybe you # do `git format-patch` and post some patch # # At this point you may want to continuously leave # the
Re: For projects switching to Meson *only*
On Thu, Apr 27, 2017 at 4:11 AM, Iain Lanewrote: As it happens I interacted with this script (in gnome-software) yesterday. I hadn't got a new enough jhbuild - the one I had was trying to call ./configure instead of meson directly. The script AFAICS ignores unknown arguments. In particular, I was interested in passing some --enable/--disable flags to select features but I didn't find out how to do that short of explicitly extending it with those particular options. If you expect distributors to be using this, or if this is interesting for users of the build API, is having some support for ./configure <-> meson_options integration a reasonable request? Otherwise, and this is what happens now that I upgraded jhbuild, using meson directly works fine. But if a goal of this is to smooth the transition path and avoid a requirement for tooling to be updated, maybe it would be useful. This compatibility issue seems like a very good argument for shipping the "compatibility" script in Continuous patches rather than application repositories. Michael ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME Build situation and BuildStream
Hi Tristan, For application or library developers (libraries used by applications), I'm a bit struggling to see what the workflow will look like with BuildStream. I've described two examples of my current workflow in this mail: https://mail.gnome.org/archives/desktop-devel-list/2016-August/msg00047.html "builddir != srcdir in jhbuild breaks my workflow" See also: https://mail.gnome.org/archives/desktop-devel-list/2017-February/msg00018.html "Equivalent of recursive make with meson/ninja?" With BuildStream you're talking about launching a VM. It's quite a big change compared to how applications are launched with jhbuild. So, can you describe a little more how the workflow would look like for application developers using the terminal (not an IDE)? With jhbuild, when we enter into a jhbuild shell we are still in the same directory, usually inside the git repository. With builddir == srcdir we have all the files that we can directly open with our preferred text editor. When we open a new terminal tab, we are in the same directory where we are able to 1) do git commands 2) building (with recursive make) 3) launching executables 4) editing files, etc. With BuildStream, will it be similar? -- Sébastien ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: GNOME Build situation and BuildStream
Hi Matthias, I realize now that this was too much information at once (even for the involved reader as opposed to a fly-by reader). So I'd like to thank you for your mind share. On Wed, 2017-04-26 at 16:39 -0400, Matthias Clasen wrote: > Tristan, > > again, it is impossible to reply to an email of this length. I can > only give a few general comments, beyond that, we really need to sit > down face-to-face and discuss this. I hope you are going to be at > Guadec ? I will certainly be around all week at GUADEC to meet with you and anyone who wants to discuss :) I am preparing a talk on this subject, but perhaps I should also try to organize something more hands on, maybe a BoF or such would be good. > My general comments: > > What you are describing here (and in your previous communications) > looks like a big, all-encompassing system, with lots of its own > terminology and a complete worldview of how things should be built. I > prefer a system that starts small and solves one problem initially, > and then maybe grows over time. I can see how it can come across this way, we are trying to break the trend of having a build system as something that is tied to any particular deployment/use case. As such, I needed to give consideration to a lot of use cases to be sure that this is something that fits, and is also an improvement over what exists. These considerations will be reflected in my communications and I can see how one might think this appears to be some kind of huge monolith which does everything. However this is exactly the opposite of what I'm trying to achieve, instead we are striving to "do one thing well" and are making an effort to ensure we're doing it the right way, for any use case. So, the core codebase itself should remain small over time, with really the sole purpose of being: "A format and engine for modeling and executing pipelines of elements which perform mutations on filesystem data within an isolated sandboxed environment" In time, I expect that an ecosystem of plugins and projects will grow around this, and use cases I had not even foreseen will come to light. This has already started to happen in some ways, as Jochen Breuer's commented on my blog here: https://blogs.gnome.org/tvb/2017/02/06/introducing-buildstream-2/ And as a result has started to work on a plugin which would allow importing distro packages and building on top of those bases: https://gitlab.com/BuildStream/buildstream/issues/10 > The system you describe seems to be all about centralization, and > about introducing a new language to describe what we build. That is > by-and-large what we already have in various incarnations that you > describe: jhbuild modulesets, the continuous manifest, flatpak > runtimes. I can get behind the idea of unifying these into a single > way of describing a multi-module build. > > But I've also seen things mentioned like 'conversion scripts for > flatpak'. And I think that is exactly the opposite of what we need > for application building. I may be mistaken, but I have a feeling you are getting the same impression which Christian had last month, which I tried to explain in this email: https://mail.gnome.org/archives/desktop-devel-list/2017-March/msg3.html > If we want to convince 3rd party applications to use flatpak, we > can't treat it as an intermediate format that we generate from some > other source, and just use somewhere in a centralized build process. > We need to embrace it ourselves and use it, just like we expect 3rd > party applications to use it. So at the risk of being repetitive, I am completely behind application authors maintaining their own build metadata themselves, building flatpaks themselves and/or submitting build metadata to a "flathub" to have them built and published to users. Of course this makes sense, because the application authors themselves are usually best situated to know what should be in their bundling build metadata. So let me try to break down how I would see this work (I realize, already a long email): GNOME core modules and services (excluding Flatpak apps) These can be expressed in a single format/repository for all the interesting purposes: o Performing CI o Creating bootable GNOME images on top of some base system, mostly for developers o Release modulesets o Producing the GNOME Flatpak runtime and SDK This is of course, one centralization. Flatpak Applications Considering the benefits which the GNOME core modules and services get by representing build metadata for multiple purposes, a given application developer team may also benefit in similar ways. This is without centralizing everything into one big build blob, or using intermediate formats or anything like this. Now, this may be more of a personal goal, a bit more ambitious, maybe best punted to later on, but
Re: For projects switching to Meson *only*
Hiya, Apologies for any ignorance about the scope and intent of the build API. On Thu, Apr 27, 2017 at 09:45:04AM +0100, Emmanuele Bassi wrote: > Hi everyone; > A simple script is available here[2], and it has nice properties, like > being able to work with a simple: > > ./configure > make > sudo make install > > which makes distributors and integrators happy. > […] > [2]: https://github.com/ebassi/graphene/blob/master/configure As it happens I interacted with this script (in gnome-software) yesterday. I hadn't got a new enough jhbuild - the one I had was trying to call ./configure instead of meson directly. The script AFAICS ignores unknown arguments. In particular, I was interested in passing some --enable/--disable flags to select features but I didn't find out how to do that short of explicitly extending it with those particular options. If you expect distributors to be using this, or if this is interesting for users of the build API, is having some support for ./configure <-> meson_options integration a reasonable request? Otherwise, and this is what happens now that I upgraded jhbuild, using meson directly works fine. But if a goal of this is to smooth the transition path and avoid a requirement for tooling to be updated, maybe it would be useful. Cheers, -- Iain Lane [ i...@orangesquash.org.uk ] Debian Developer [ la...@debian.org ] Ubuntu Developer [ la...@ubuntu.com ] signature.asc Description: PGP signature ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: For projects switching to Meson *only*
The issue is not writing a simple patch: it takes me about 90 seconds of my life to copy the file, create a commit, format-patch it, and add the patch to gnome-continuous. The whole point would be to avoid adding a patch to Continuous. Additionally, conforming to the build-api is a nice transitional tool for distributors that are still trying to catch up to Meson. Ciao, Emmanuele. On 27 April 2017 at 09:48, Carlos Sorianowrote: > Hello Emmanuele, > > Would be fine if the maintainer does the patch for continuous instead of > doing the build-API? > > Carlos Soriano > > Original Message > Subject: For projects switching to Meson *only* > Local Time: 27 April 2017 10:45 AM > UTC Time: 27 April 2017 08:45 > From: eba...@gmail.com > To: Desktop Development List > > Hi everyone; > > since maintainers have started switching to Meson for the development > cycle (awesome!) I'd like to ask for a favour: if you're dropping > autotools entirely, could you please add a `configure` wrapper script > that conforms to the build-api[1] that GNOME Continuous uses? > > A simple script is available here[2], and it has nice properties, like > being able to work with a simple: > > ./configure > make > sudo make install > > which makes distributors and integrators happy. > > Before you ask: yes, I'm looking at adding Meson support to > Continuous, as part of a larger rework of the manifest file to adapt > to the Flatpak builder manifest format; that will take time, though, > so in the interim it'd be great if we didn't have to ship a patch for > every project, when the alternative is simply dropping a script into > the project's top-level. > > Ciao, > Emmanuele. > > [1]: https://github.com/cgwalters/build-api > [2]: https://github.com/ebassi/graphene/blob/master/configure > > -- > https://www.bassi.io > [@] ebassi [@gmail.com] > ___ > desktop-devel-list mailing list > desktop-devel-list@gnome.org > https://mail.gnome.org/mailman/listinfo/desktop-devel-list > > -- https://www.bassi.io [@] ebassi [@gmail.com] ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
Re: For projects switching to Meson *only*
Hello Emmanuele, Would be fine if the maintainer does the patch for continuous instead of doing the build-API? Carlos Soriano Original Message Subject: For projects switching to Meson *only* Local Time: 27 April 2017 10:45 AM UTC Time: 27 April 2017 08:45 From: eba...@gmail.com To: Desktop Development ListHi everyone; since maintainers have started switching to Meson for the development cycle (awesome!) I'd like to ask for a favour: if you're dropping autotools entirely, could you please add a `configure` wrapper script that conforms to the build-api[1] that GNOME Continuous uses? A simple script is available here[2], and it has nice properties, like being able to work with a simple: ./configure make sudo make install which makes distributors and integrators happy. Before you ask: yes, I'm looking at adding Meson support to Continuous, as part of a larger rework of the manifest file to adapt to the Flatpak builder manifest format; that will take time, though, so in the interim it'd be great if we didn't have to ship a patch for every project, when the alternative is simply dropping a script into the project's top-level. Ciao, Emmanuele. [1]: https://github.com/cgwalters/build-api [2]: https://github.com/ebassi/graphene/blob/master/configure -- https://www.bassi.io [@] ebassi [@gmail.com] ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list
For projects switching to Meson *only*
Hi everyone; since maintainers have started switching to Meson for the development cycle (awesome!) I'd like to ask for a favour: if you're dropping autotools entirely, could you please add a `configure` wrapper script that conforms to the build-api[1] that GNOME Continuous uses? A simple script is available here[2], and it has nice properties, like being able to work with a simple: ./configure make sudo make install which makes distributors and integrators happy. Before you ask: yes, I'm looking at adding Meson support to Continuous, as part of a larger rework of the manifest file to adapt to the Flatpak builder manifest format; that will take time, though, so in the interim it'd be great if we didn't have to ship a patch for every project, when the alternative is simply dropping a script into the project's top-level. Ciao, Emmanuele. [1]: https://github.com/cgwalters/build-api [2]: https://github.com/ebassi/graphene/blob/master/configure -- https://www.bassi.io [@] ebassi [@gmail.com] ___ desktop-devel-list mailing list desktop-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/desktop-devel-list