Bug#1030223: gobject-introspection mini-policy: separate GIR XML from -dev package to make cross-compilation possible?
On Wed, 01 Feb 2023 at 10:39:30 +, Simon McVittie wrote:
> I think this would require changes to dependent packages if they make
> use of the GIR XML (because build-depending on libflatpak-dev would
> no longer be enough, and it would also be necessary to build-depend on
> gir1.2-flatpak-1.0-dev or similar). If so, this would have to
> be done on a package-by-package basis, with dependent packages changed
> before their dependency. A migration path would be to add
> Provides: gir1.2-flatpak-1.0-dev (= ${binary:Version}) on libflatpak-dev.
The dh_girepository in gobject-introspection (>= 1.78.1-3~) generates a
${gir:Provides} substvar for any package that contains public GIR XML,
partially automating this. For example, libflatpak-dev_1.14.5-1+b1 has
Provides: gir1.2-flatpak-1.0-dev (= 1.14.5-1+b1).
To make this work, a sourceful change is still necessary, to add
Depends: ${gir:Depends}
Provides: ${gir:Provides}
to every package that ships GIR XML in the public directories:
/usr/share/gir-1.0/*.gir or /usr/lib/${DEB_HOST_MULTIARCH}/gir-1.0/*.gir.
We also want packages that ship public typelibs
(/usr/lib/${DEB_HOST_MULTIARCH}/girepository-1.0/*.typelib or legacy
/usr/lib/girepository-1.0/*.typelib) to have Depends: ${gir:Depends}.
We also want packages that ship public typelibs that do not match the
package's name to have Provides: ${gir:Provides}. For example,
gir1.2-flatpak-1.0 doesn't need this, because the only typelib it contains
is Flatpak-1.0.gir; but gir1.2-gtk-4.0 *does* need this, because in
addition to Gtk-4.0.typelib, it also contains Gdk-4.0.typelib and others.
The dh_girepository in gobject-introspection (>= 1.78.1-3~) also emits
warnings suggesting that (for example) src:libadwaita-1 should have a
Build-Depends on gir1.2-gtk-4.0-dev, if the Build-Depends isn't already
present and the -dev package already has the necessary Provides. This
will let us get those Build-Depends into place gradually.
It is not fully clear to me how much of this is applicable to private
typelibs or GIR XML, shipped in private locations (for example by mutter,
gnome-shell, muffin, cinnamon and cinnamon-screensaver). There is some
discussion of this on #1057391.
> A side benefit of this would be that we could separate out the GIR XML
> that is provided by src:gobject-introspection (#859013) into one or more
> similar gir1.2-*-dev packages.
I intend to do this next time gobject-introspection goes through NEW, or
as part of the first upload of GLib 2.79.x, whichever is sooner.
(GLib 2.79.x is going to take over responsibility for GLib-2.0.typelib etc.
and GLib-2.0.gir etc. from gobject-introspection.)
On Mon, 17 Apr 2023 at 21:54:23 +0200, Helmut Grohne wrote:
> Let me propose another alternative. Rather than
> have gir1.2-flatpak-1.0-dev be a real binary package, we can provide it
> as a virtual one from libflatpak-dev. The mini-policy would then have to
> be updated such that using the gir files requires depending on a
> gir*-dev package and that a dependency on the regular lib*-dev package
> is insufficient. When enabling the nogir profile, the virtual package is
> removed thus no interface is broken. This variant still breaks
> reproducibility of course.
I would prefer not to be 100% relying on this, because I'd prefer for
build-profiles to be "safe" (no functional changes to package content,
and ideally fully reproducible); but it would be a useful stopgap to
avoid having the NEW queue block forward progress.
> In your second mail, you propose storing pre-built gir files in debian/.
I think I mentioned when talking to Helmut in person that I wasn't
proposing storing pre-built GIR files, only the intermediate "dump" files
that are one of the inputs to generating a GIR file. The workflow goes
like this:
- gcc compiles C headers/source into a shared library
- g-ir-scanner produces GIR XML by:
- parsing C headers/source
- generating a temporary file with a list of GObject types learned
from the C headers/source
- compiling a "dumper" binary linked to the shared library
- running the "dumper" binary with the list of GObject types as input
- parsing the output of the "dumper" binary, which includes things
that in general can only be known at runtime, like all of a GObject
type's signals, properties and ancestors
- g-ir-compiler compiles GIR XML into a binary typelib
The GIR XML is a combination of information parsed directly from the C
headers and source (I'd estimate 80%), and information from the output
of the "dumper" binary (I'd estimate 20%).
What I was proposing storing pre-built in debian/ is the input to the
"dumper" binary (list of GObject types), and the corresponding output
(each type's list of signals and so on). This is vaguely analogous
to storing a .symbols file, but not a prebuilt shared or static library.
> I dislike this proposal, because it poses continuous extra effort for
> maintainers. We already struggle with symbols files and
Bug#1030223: gobject-introspection mini-policy: separate GIR XML from -dev package to make cross-compilation possible?
Hi Simon,
On Wed, Feb 01, 2023 at 10:39:30AM +, Simon McVittie wrote:
> Ideally, when cross-compiling Flatpak, we'd prefer to build it with
> a build profile like cross or nogir that has the effect of building it with
> ./configure --disable-introspection, which disables the generation of the
> files marked [*]. However, that would make this build profile cause a
> functional change to the contents of libflatpak-dev, which is strongly
> discouraged.
I agree in principle. The problem with changing the contents of
libflatpak-dev is twofold. For one thing, a binary package name (such as
libflatpak-dev) implicitly defines an interface and a contract between
cooperating packages. The present interface includes the gir files and
having them vanish from libflatpak-dev on specifying a nogir profile
would break the interface contract. The other reason for the
discouragement is being able to validate profiles using reproducible
builds.
> If we separated the GIR XML into a new binary package, perhaps named
> gir1.2-flatpak-1.0-dev or something like that, then we would be able to
> turn off both gir1.2-flatpak-1.0 and gir1.2-flatpak-1.0-dev under the
> nogir build profile (similar to how the noudeb, nocil and noinsttest
> profiles work), making cross-compilation possible (as long as you don't
> care about GObject-Introspection-based language bindings).
Such separation solves both the interface contract and the
reproducibility aspect. Let me propose another alternative. Rather than
have gir1.2-flatpak-1.0-dev be a real binary package, we can provide it
as a virtual one from libflatpak-dev. The mini-policy would then have to
be updated such that using the gir files requires depending on a
gir*-dev package and that a dependency on the regular lib*-dev package
is insufficient. When enabling the nogir profile, the virtual package is
removed thus no interface is broken. This variant still breaks
reproducibility of course.
> I think this would require changes to dependent packages if they make
> use of the GIR XML (because build-depending on libflatpak-dev would
> no longer be enough, and it would also be necessary to build-depend on
> gir1.2-flatpak-1.0-dev or similar). If so, this would have to
> be done on a package-by-package basis, with dependent packages changed
> before their dependency. A migration path would be to add
> Provides: gir1.2-flatpak-1.0-dev (= ${binary:Version}) on libflatpak-dev.
In essence, I am saying that this migration path could be the final
result thus eliminating the NEW trip, but not eliminating the need to
update dependent packages. Note that dpkg resolves profiles in binary
package relations at build time, so you can write:
Provides: gir1.2-flatpak-1.0-dev (= ${binary:Version})
> A side benefit of this would be that we could separate out the GIR XML
> that is provided by src:gobject-introspection (#859013) into one or more
> similar gir1.2-*-dev packages.
Of course, a separation by real binary packages would not be prohibited.
In your second mail, you propose storing pre-built gir files in debian/.
I dislike this proposal, because it poses continuous extra effort for
maintainers. We already struggle with symbols files and shouldn't add
another variant of them. I understand that this looks attractive from a
cross builder's point of view, but I think cross building should bear
the effort here, not gnome package maintainers. I hope you can agree
with that.
Given this, would you agree that nogir would make sense right now as a
non-reproducible profile? It still requires updating all reverse
dependencies, yeah, but we can do so in parallel. For the migration
period, we would have missing dependencies in the presence of nogir
profiles, but these would solely affect users of the nogir profile,
which would be the ones to report and fix such missing dependencies.
Regular users (and buildds in particular) would never experience issues
due to such missing dependencies.
Helmut
Bug#1030223: gobject-introspection mini-policy: separate GIR XML from -dev package to make cross-compilation possible?
On Wed, 01 Feb 2023 at 10:39:30 +, Simon McVittie wrote: > GObject-Introspection gets some of its information by parsing the source > code, and that's fine. However, it gets other information by introspecting > the just-built libflatpak.so.0 (hence its name): linking a small executable > to libflatpak.so.0, and running it. When doing "true" cross-compiling > (without the ability to run host executables via qemu-user or similar), > this is not possible. Alternatively, we might be able to do something like this: - commit a copy of the output of the dump executable in debian/ - optionally have architecture-specific output, but as far as I know, the common case is the same for every architecture (like .symbols files) - during a native build: - if the cached copy in debian/ does not match the real result, fail with an error (ideally showing the differences) - or if the cached copy in debian/ is missing, tell the maintainer what to add (in the form of a diff, like .symbols files) - during a cross build, use the cached copy (and assume it's right) That would mean we can have GIR, and maybe even typelibs?, from cross-builds too - as long as someone has done a native build of the same upstream version at least once (on any architecture for the common case where the GIR is the same on all architectures, or on the matching architecture in the uncommon case where it's different per-architecture). smcv
Bug#1030223: gobject-introspection mini-policy: separate GIR XML from -dev package to make cross-compilation possible?
Package: gobject-introspection
Version: 1.74.0-3
Severity: wishlist
File: /usr/share/doc/gobject-introspection/policy.txt.gz
Tags: trixie sid
At the moment, a typical introspected library (I'm using libflatpak as my
example) has this layout, where the files marked [*] are generated by
GObject-Introspection:
libflatpak-dev
/usr/include/flatpak
/usr/lib/*/libflatpak.so
/usr/lib/*/pkgconfig/flatpak.pc
/usr/share/gir-1.0/Flatpak-1.0.gir[*]
libflatpak0
/usr/lib/*/libflatpak.so.0.*
/usr/lib/*/libflatpak.so.0
gir1.2-flatpak-1.0
/usr/lib/*/girepository-1.0/Flatpak-1.0.typelib [*]
GObject-Introspection gets some of its information by parsing the source
code, and that's fine. However, it gets other information by introspecting
the just-built libflatpak.so.0 (hence its name): linking a small executable
to libflatpak.so.0, and running it. When doing "true" cross-compiling
(without the ability to run host executables via qemu-user or similar),
this is not possible.
Ideally, when cross-compiling Flatpak, we'd prefer to build it with
a build profile like cross or nogir that has the effect of building it with
./configure --disable-introspection, which disables the generation of the
files marked [*]. However, that would make this build profile cause a
functional change to the contents of libflatpak-dev, which is strongly
discouraged.
If we separated the GIR XML into a new binary package, perhaps named
gir1.2-flatpak-1.0-dev or something like that, then we would be able to
turn off both gir1.2-flatpak-1.0 and gir1.2-flatpak-1.0-dev under the
nogir build profile (similar to how the noudeb, nocil and noinsttest
profiles work), making cross-compilation possible (as long as you don't
care about GObject-Introspection-based language bindings).
I think this would require changes to dependent packages if they make
use of the GIR XML (because build-depending on libflatpak-dev would
no longer be enough, and it would also be necessary to build-depend on
gir1.2-flatpak-1.0-dev or similar). If so, this would have to
be done on a package-by-package basis, with dependent packages changed
before their dependency. A migration path would be to add
Provides: gir1.2-flatpak-1.0-dev (= ${binary:Version}) on libflatpak-dev.
A side benefit of this would be that we could separate out the GIR XML
that is provided by src:gobject-introspection (#859013) into one or more
similar gir1.2-*-dev packages.
smcv
