On Thu, 11 Jan 2024 at 14:15:50 +0100, Helmut Grohne wrote:
> On Thu, Jan 11, 2024 at 12:08:53PM +0000, Simon McVittie wrote:
> > The ${GNU_TYPE}-g-ir-compiler wrapper script (which happens to be written
> > in Python, the same as the upstream g-ir-compiler) explicitly tells
> > g-ir-compiler to run the "dumper" binary under qemu-user if it detects
> > that the Python architecture is not one that can run the host architecture.
>
> Do I understand correctly that cross building to i386 on amd64 would
> cause this wrapper to run the i386 binary in qemu?
Until today: no, but only because there was a hard-coded special case
for the common i386-on-amd64. Cross-building to armhf on arm64, or to
powerpc on ppc64, *would* have used qemu automatically, even if not
actually necessary.
In the version I hope to upload today: no, because I've added auto-detection
of whether we can execute host binaries as you suggested.
> I agree with the approach taken, but I think g-ir-compiler could be more
> clever. Rather than assume that the host architecture is not runnable
> when it differs from the build architecture, could it detect that? A
> simple way would be invoking arch-test ${DEB_HOST_ARCH}, but it can as
> well compile and run trivial program (as autoconf does all the time).
I'm testing an implementation of that. arch-test needs specific
porting for each new architecture because of how it's written,
so I'm not intending to use that directly, but it's easy to add a
precompiled arch-test-like binary of the host architecture to the
gobject-introspection binary package, and have the wrapper script try
to invoke it and see what happens.
> > I would tend to think that qemu dependencies in Build-Depends are
> > appropriate if and only if it's the source package that is making the
> > choice to invoke qemu.
>
> The argument is reasonable. Your way of looking at it also lowers
> maintenance cost as we don't have to modify tons of B-D.
Right - if we decide that qemu is not so good for some pair of (real,
emulated) architectures, and actually we'd prefer to use some other
user-space emulator like FEX or box86 for a particular pair, I don't
want to have to make new sourceful changes in all 238 source packages[1]
that produce public GIR/typelibs, plus however many packages produce
private GIR/typelibs. It seems like it would be better to only change
src:gobject-introspection and O(1) other packages.
We will want to make sourceful changes in all of those 238 packages
eventually, to replace libgirepository1.0-dev (which cannot be M-A:same
without breaking some dependent packages) with a cross-friendly
alternative, but I only want to do that once per source package in
most cases.
For some of those packages (the ones where GIR is optional, like
src:flatpak, but not the ones where GIR is required functionality, like
src:gnome-shell) we will eventually also want to add support for <!nogir>,
and maybe split out gir1.2-NAMESPACE-VERSION-dev into its own binary
package so that the nogir build profile can be a reproducible/"safe"
one - but, again, that should be something we can do once per source
package, not something that we have to repeat every time an implementation
detail changes.
[1] grep-dctrl -FPackage-List -sPackage -e --pattern='gir1\.2-' \
/var/lib/apt/lists/deb.debian.org_debian_dists_sid_main_source_Sources \
| sort -u | wc -l
> I am wondering
> about a middle-ground of having a package can-run-arch being M-A:same
> and having a maintainer script that validates the property. Then you
> could Depends: qemu-user | can-run-arch (expressing the preference for
> qemu-user) while any builder could still --add-depends=can-run-arch to
> opt out of qemu.
If the cross-toolchain team implements such a thing, it would be fine to
add it as an alternative dependency in a later version. I'd prefer not to
do that while it's still hypothetical, because until there's a concrete
implementation we'd have no way to test it.
Another option (which could perhaps be combined with this) would be for
the cross-toolchain team to define an interface to "the preferred way to
run executables from architecture A if they can't be run directly", and
then gobject-introspection could try that in preference to qemu. Meson
calls this an "EXE wrapper", which seems like as good a name as any other.
Here's a straw-man design, assuming for the sake of concrete examples that
the build architecture is amd64 and the host architecture is riscv64:
- gobject-introspection:riscv64 Depends: cross-exe-wrapper | can-run-arch
- cross-exe-wrapper:riscv64 is M-A:same and Depends on
cross-exe-wrapper-riscv64-linux-gnu
- cross-exe-wrapper-TUPLE is M-A:foreign (or perhaps a virtual
package provided by cross-exe-wrapper-bin, which is M-A:foreign)
- For the trivial case, cross-exe-wrapper-TUPLE:ARCH, where TUPLE and
ARCH match, contains a /usr/bin/TUPLE-exe-wrapper which just runs its
arguments as-is
(like a trivial shell script that just does an 'exec "$@"')
- There could be simple special cases for some pairs, like
cross-exe-wrapper-i686-linux-gnu:amd64 might contain a
/usr/bin/i686-linux-gnu-exe-wrapper which invokes linux32
- in general cross-exe-wrapper-TUPLE:ARCH, where ARCH is supported by qemu
and TUPLE and ARCH do not match, might contain a symlink
/usr/bin/TUPLE-exe-wrapper -> qemu-ARCH and depend on qemu-user
(or it might be a script that auto-detects whether a wrapper is needed
or does something more complicated, that's an implementation detail)
- Document that the first argument to /usr/bin/TUPLE-exe-wrapper should
be a TUPLE binary, normally by its absolute path; if it starts with
"-" then the result is undefined; and it needs to pass all subsequent
arguments to the binary
(for some emulators this might mean it needs to invoke
'exec /usr/bin/emulator -- "$@"')
For the gobject-introspection use-case it would be possible to skip the
cross-exe-wrapper package name and make g-i depend directly on
cross-exe-wrapper-${local:DEB-HOST-GNU-TYPE}, but I think we would need
the intermediate package name anyway if you want to be able to add
Build-Depends: cross-exe-wrapper <cross>, for use-cases where the upstream
build system will invoke TUPLE-exe-wrapper itself.
> I wonder though whether we should make such "do not depend" rules
> explicitly checkable somehow. There is more of that in the archive.
A Lintian check could make sense, perhaps? It could have a table of
package names that should not be directly (build-)depended on, each with
its allowed exceptions if any, for example:
gobject-introspection-%-endian src:gobject-introspection
gobject-introspection-bin src:gobject-introspection
libmutter% src:budgie-desktop src:gnome-remote-desktop src:gnome-shell
src:mutter
liburweb0 src:urweb
lighttpd-modules-% src:lighttpd
perl-modules-% src:perl
python-dev-is-% src:what-is-python
python-is-% src:what-is-python
python3-minimal python3
python3.%-minimal python3.%
(mostly based on an aptitude search for; ~d"(do|should|must) not depend")
smcv
