On Sun, 14 May 2023 at 23:37:34 +0200, Josh Triplett wrote:
> People build things on Debian that are not Debian packages. People
> compile binaries on Debian, and expect them to work on any system that
> has sufficiently new libraries.
*raises hand*
Hello, I represent an example of those people. With my $work hat on,
I'm involved in maintaining a family of Debian derivatives (the Steam
Runtime) that is used to run Steam games on arbitrary distributions,
including but not limited to Debian. Some of these binaries are built
on a Debian derivative, but run on an arbitrary other distribution,
using a RPATH[1] to find their non-glibc dependencies.
At the moment those binaries are built in ancient environments (based
on Ubuntu 12.04 and Debian 8), but as newer versions of glibc become
ubiquitous, we'll want to start relying on newer versions of Debian
(which will still be very old versions *at the time*, but I'm sure that
by 2030 or 2040 we'll want to be using versions of Debian that, in 2023,
are not released yet). In this use-case, yes we do need to be using the
interoperable ELF interpreter path.
We were able to use (Ubuntu and) Debian for this *because* Debian is
a relatively "ordinary" distribution that tends to follow cross-distro
standards. The major counterexample is multiarch library paths, but
multiarch library paths have some compelling technical advantages, and
because there's no ambiguity about which architecture uses which directory,
they're actually better for interop in some ways than /usr/lib (which
is ambiguous, because the Red Hat family of distros expects to find 32-bit
libraries there, but the Arch family expects 64-bit libraries, and it's
not possible to construct a tree where both get what they want).
I've spent a lot of the last 5 years working on putting Steam games in
containers so that they'll work more reliably on all distros, including
Debian, with less reliance on weird library search paths (although we
still have to use binaries built on an ancient Debian derivative with a
non-trivial RPATH to bootstrap the container environment). Because of
constraints around recent GPUs needing current graphics drivers even
if running on an otherwise ancient library stack, Steam's container
runtime has to construct a hybrid environment where glibc is either the
one from the host or the one from the container runtime environment,
whichever is newer; and while doing that, we have experienced some
broken situations that were caused by distributions diverging from the
interoperable ELF interpreter. One concrete example is that Arch Linux
uses the interoperable ELF interpreter for *almost* all executables,
but uses a different ELF interpreter for executables from the glibc
source package, for whatever reason; we were able to work around this,
but for a while it caused Arch to fail to launch these containers where
Debian/Fedora/etc. could.
This is not something that any of the distributions involved is
going to formally support, so the overall system consists of various
unsupported-but-works-in-practice things happening; but there is
absolutely no chance of Valve/Steam shipping separate binaries for
each distro (there are too many distros for that, even if you don't
count source-based distros like Gentoo that have an almost unlimited
number of concrete instantiations as binaries). However loudly we
might insist that our small subset-of-a-subset is the one they should
be targeting, what we're going to get is binaries that work on "mostly
interoperable" distros. As a result, if we want games on Linux (and
anything else requiring binary interop) to continue to be a thing,
pragmatically we should aim to be one of those "mostly interoperable"
distros, and encourage our friends and/or rivals in other distros to
do the same. Otherwise, the most likely outcome is for developers to go
back to an attitude of "I'm not going to support Linux, too many random
differences" and releasing for Windows only, and we all lose out.
As a result of all this, I would strongly prefer our compiler to
default to hard-coding the same interoperable ELF interpreter that
glibc upstream and the various major distros have agreed on (for example
/lib64/ld-linux-x86-64.so.2 on amd64), and I would also prefer it if we
can use that interoperable path in all the binaries we ship, including
src:glibc and the rest of the transitively Essential set.
One way that I like to think about this sort of thing is that we have a
finite number of "weird Debianism" tokens available, and we should aim
to spend them on the things that give us the best cost/benefit ratio.
We've never considered changing the ELF interpreter to be one of those,
to the extent of having a /lib64 solely for its benefit (on amd64)
even though by policy we don't generally use lib64.
(Incidentally, Steam's container runtime is always a merged-/usr
environment, to provide an environment that maximizes the probability
of any given script or binary working correctly; but it also goes to
considerable lengths to work on both merged- and unmerged-/usr host
systems. I think that's the right way round: it's conservative with what
it requires, but liberal with what it can run.)
smcv
[1] at the moment, genuinely a RPATH and not the modern RUNPATH for
annoying technical reasons
