On Wed, Jun 29, 2016 at 3:03 AM, Daniel P. Berrange <berra...@redhat.com> wrote: > For those who aren't familiar, QEMU actually provides two completely > different sets of emulators > > - system emulators - they emulate a full virtual machine and thus run > a full guest OS. > - user emulators - they emulate the Linux userspace ABI letting you > run non-native arch executables directly. > > The user emulators are what I'm concerned with in this mail, so ignore > the system emulators. > > Currently all the user emulators are provided in the "qemu-user" RPM > which also includes files in /usr/lib/binfmt.d to register each emulator > binary as a binary format handler for its respective architecture. > > This is ok if you have a non-native arch binary that's statically linked > and you just want to run it from context of your main OS root filesystem. > Running dynamic linked binaries won't fly because if say running an arm > binary on x86_64 host, it'll look for /lib/libc.so and find the i386 one, > instead of the arm one. You can't set LD_LIBRARY_PATH to override this > as the env var will apply to both qemu-arm (an x86_64 binary) and the > binary it is trying to run (an arm binary). > > More typical though is that you have a directory containing an fullish > install tree of a non-native architecture and you just want to chroot > into that. When doing such a chroot, the qemu-$ARCH emulator must be > present inside the chroot too. ie the x86_64 build of /usr/bin/qemu-arm > must be present inside at /my/chroot/for/fedora-arm/usr/bin/qemu-arm. > So again you have the potential problem of clashing libc.so in /usr/lib > It is a shame Fedora doesn't have full multi-arch support, instead of > merely multi-lib to avoid these clashing lib dirs across architecture > RPMs. > > The recommended way to deal with this for the qemu user emulator binaries > to be statically linked, so when copied inside the non-native arch chroot, > they never need to resolve any native arch libraries. Fedora's qemu user > binaries are all dynamic linked right now. > > Debian handles this by having several packages [1] > > - qemu-user - the dynamic linked qemu user binaries > - qemu-binfmt - binfmt rules registering the dynamic linked binaries > - qemu-user-static - the static linked qemu user binaries *and* binfmt > rules to register them. The static binaries all > have -static suffix on their name > > NB, this means qemu-binfmt and qemu-user-static are mutually exclusive > since they both provide the same binfmt files. You can however have both > qemu-user and qemu-user-static installed as their binary names won't > clash, and in this case the static ones will be registered as binfmts > > This nice thing about this multiple package approach is that when you > copied the x86_64 build of the "qemu-arm-static" binary into your arm > chroot, you still then have the possibility of installing the arm build > of the "qemu-arm" binary inside that chroot without filename clash. > > An alternative simpler approach would be to just have one package, > qemu-user, which contains the static binaries and never ship any > dynamic linked qemu user binaries. This is slightly more restrictive > though, as explained in the previous paragraph, so I'd like to avoid > doing that. > > > I'd like to make using non-native arch chroots simple with Fedora without > people needing to manually build their own static QEMU binaries, or download > static binaries provided by another distro[2]. So I'm suggesting to make a > change to Fedora qemu packages to essentially copy the way Debian has done > things. Specifically I will > > - Pull the binfmt registration files out of qemu-user and into a > new qemu-binfmt package which depends on qemu-user. > > - Add static builds of qemu user emulators to a new qemu-user-static > package, along with binfmt registration files > > The static build of QEMU user emulators is moderately light on > dependancies, only requiring glib2-static, pcre-static, zlib-static > and glibc-static packages. > > The change to introduce a qemu-binfmt package has small upgrade > implications since anyone with qemu-user installed today, will loose > the binary format rules unless they manually install qemu-binfmt. I > think the number of people affected is probably quite small, and some > of them may well wish to use qemu-user-static instead anyway. > > Obviously this would only be done in rawhide, not any existing stable > releases of Fedora. > > Nothing will change about the rest of QEMU packaging - ie all system > emulators will continue to use dynamic linking > > Regards, > Daniel >
Amazingly, I've been pre-empted here. I was going to ask about this for Fedora, since we've got interest in this for Mageia as part of potentially enabling cross-arch building with mock in an easy fashion. Mageia tries to keep its QEMU package in sync with Fedora, so I'd love to see this happen so that it becomes easier to maintain the qemu-user-static stuff. Right now, we only have a couple of the arches built, but one of our guys interested in this might be willing to help out on this side of the fence with it (Per Øyvind Karlsen). I've CC'd him to this email. -- 真実はいつも一つ！/ Always, there's only one truth! -- devel mailing list devel@lists.fedoraproject.org https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org
