On Thu, 15 Oct 2009 21:23:04 +0100 Colin Watson <cjwat...@ubuntu.com> wrote:
> Package: dpkg-dev > Version: 1.15.4 > Severity: normal > User: ubuntu-de...@lists.ubuntu.com > Usertags: origin-ubuntu karmic > > The PKG_CONFIG_LIBDIR override introduced for cross-compilation > purposes due to #439979 (so courtesy-copying Neil as the reporter of > that bug, although ISTR he's trying to reduce his involvement with > Emdebian at the moment) has a somewhat awkward consequence. I'm not > sure I have a good solution for this right now, but would like to > have a bug open for discussion. Day-to-day cross-building for Emdebian has stalled, not just because of my reduced involvement but also because the cross-building system has critical problems and the solution means migrating to a fully operational multiarch - after Squeeze. The system used for Emdebian Crush 1.0 (based on Lenny) had severe limitations (principally in correct handling of build-dependencies) and can take us no further. http://lists.debian.org/debian-embedded/2009/08/msg00005.html How will multiarch affect pkg-config? I'm not sure that the detail has been decided yet, so CC:'ing debian-embedded for info. FTR there are other cases that are similar - pango and gtk both build tools that are expected to be run during the build and these tools were simply not built (or at least not executed) in the cross-builds, using Emdebian-specific patches where necessary, causing a loss of functionality. I haven't been able to solve that problem (but there were workarounds for Crush 1.0 that relied on copying the data files from other systems instead of generating new ones. Not ideal.) > It's true that setting PKG_CONFIG_LIBDIR when cross-compiling makes > it a lot easier to build the majority of packages that use pkg-config > in simple ways. You ask pkg-config for libraries, it automatically > looks them up for the host architecture, and your job's done. PKG_CONFIG_LIBDIR was not the complete solution but it did/does do a good job for most packages that use pkg-config. There were still workarounds needed for some packages (using path mangling with gccross) and dbus was one of these packages. Emdebian used a field in the debian/xcontrol file (X-Build-Cross-Libtool=yes) that changed the PATH so that the cross-build always used the right compiler. In effect, this prevented dbus from building or executing the binding-tool using the build compiler and only allowed the host compiler to be "seen". http://www.emdebian.org/emdebian/intro.html#gccross i.e. this is a known bug that I haven't been able to solve without a truly horrible hack (much worse than Colin's suggestion) and I fully agree that it is a corner case of the more general solution. Nevertheless, dbus seemed to work at runtime in Emdebian Crush 1.0 - whether that was by design or just because the limited package selection in Emdebian Crush 1.0 simply didn't ask dbus to do much, I wouldn't like to say. Sadly, with the current state of cross-building for Emdebian, it is hard to test fixes until multiarch is at least partially in place. > However, consider the case where you need to build a binary for use > during the build that needs to link against external > build-architecture libraries that should be discovered using > pkg-config. An example of this is dbus-glib, which builds a binary > (dbus-binding-tool) both for use during its build and to ship in one > of its .debs. This binary uses the dbus-1 and gobject-2.0 pkg-config > modules, linking against the corresponding libraries. In the > cross-compilation case, we clearly need to build this binary twice - > but now we need to persuade pkg-config to give us build libraries! > > I've approached this in a hacky fashion in my experiments by using > things like this in configure.ac: > > DBUS_CFLAGS_FOR_BUILD=`unset PKG_CONFIG_LIBDIR; pkg-config --cflags > dbus-1` > > The problem with this is that it's horribly Debian-specific However, the use of PKG_CONFIG_LIBDIR is also Debian-specific (specifically cross-building in Debian specific). Other teams cross-building dbus are likely to be using qemu or other staging methods where at least some of the build is actually achieved on the embedded device or something approaching compatibility with the device. I'm not sure that fixing this upstream would actually make any difference to those who cross-build the upstream dbus using non-Debian methods or staging methods involving things like qemu or builds-on-the-device. The only team trying to cross-build Debian packages using dpkg-dev and not using emulators or requiring access to the device until after the installation tarball has been created, is Emdebian. This is principally because Emdebian seeks to create a binary distribution (i.e. Debian-like) where the packages and the installation image can be prepared independently. (Using staging tends to require that you build all your packages yourself and then build one specific installation from that one build, making all build errors fatal for the installation testing and fatally undermining reproducibility testing of the final packages. Debugging embedded stuff is hard enough without every user having different builds of all packages.) >, and it > would really be easier for everyone if this kind of change (which I am > able to implement in Autoconf/Automake without resorting to > packaging-level changes, although it's not all that pretty) could go > upstream. A Debian-specific fix would seem appropriate to me. There are a lot of corner cases right now; dbus, pango and gtk are ostensibly similar cases, gcc and perl are much harder ones to solve (but gcc is a higher priority because not every cross-built system needs a GUI or dbus but everything needs libgcc1, sadly, whilst Crush 1.0 did without perl entirely - reasonably well too). Other corner cases may well crop up during the adoption of multiarch. Once there is a working implementation, we can see about spreading multiarch to various upstreams as a complete task instead of the historical patchwork of incremental hacks. > The standard upstream approach to this kind of thing is > documented in /usr/share/doc/pkg-config/README: > > Users can define the PKG_CONFIG environment variable to point at the > right one, or if they cross-compile and have a correctly named > pkg-config (eg. arm-linux-pkg-config) in their PATH that will be used > in preference. > > So, really what it seems that we ought to do is require > $host-pkg-config on all cross-building systems just like we require > $host-gcc etc. and then *not* set PKG_CONFIG_LIBDIR; maybe as a > fallback we could set PKG_CONFIG_LIBDIR in the event that > $host-pkg-config is not on the path. With that done, it would be > possible to use $host-pkg-config and pkg-config depending on whether > cross or native builds are needed, the way it was clearly intended to > work upstream. I can't remember now quite why this didn't originally work (it may simply have been that Emdebian didn't try this level of change) but with all the changes needed for the adoption of multiarch, I think it is time to ditch all workarounds (like dpkg-cross mangling of .pc files and PKG_CONFIG_LIBDIR in dpkg and gccross in emdebian-tools) and attempt to implement a new, generalised, solution. True, this may result in a new set of corner cases and a new set of workarounds, but I think it's a net gain. It should be possible to implement /usr/bin/arm-linux-gnueabi-pkg-config as a shell script provided by a package that is made into a dependency of the cross-building environment (ala binutils-multiarch). Indeed, the new package could simply divert /usr/bin/pkg-config and use dpkg-architecture values to detect which environment to set in the wrapper with $host-pkg-config being a symlink - possibly even generated in the postinst or at runtime so that new triplets can be supported cleanly. The binary pkg-config-multiarch package could be built from the dpkg-cross source package to make it easier to keep it in step with supported triplets or it could be part of dpkg-dev from the start. (dpkg-cross will still need to be merged into dpkg-dev eventually, the question is how much of it is to be thrown away first. Most of it, hopefully.) > I'm not sure how to get there from here, though. Perhaps get > cross-targeted pkg-config packages onto buildd.emdebian.org and allow > a while for people to install them? I think this needs to be tied in with the other changes being worked on for the .pc files themselves in order to achieve multiarch support. It is achievable - it would also provide a possible home for changes to pkg-config that could make it unnecessary to modify the .pc files in the first place. It's not just autotools either, CMake needs similar changes: The main changes involve setting PKG_CONFIG_LIBDIR within CMake an setting the include directories to locate the cross libraries installed by dpkg-cross e.g. #Make pkg-config look in the right place SET(ENV{PKG_CONFIG_LIBDIR} ${CMAKE_FIND_ROOT_PATH}/lib/pkgconfig/) ELSE (CMAKE_CROSSCOMPILING) dpkg-cross (1) Those changes could also be made in the multiarch pkg-config wrapper. A little test and unpacking: # diff -u usr/lib/pkgconfig/dbus-glib-1.pc usr/arm-linux-gnueabi/lib/pkgconfig/dbus-glib-1.pc --- usr/lib/pkgconfig/dbus-glib-1.pc 2009-09-29 23:39:48.000000000 +0000 +++ usr/arm-linux-gnueabi/lib/pkgconfig/dbus-glib-1.pc 2009-10-16 08:17:10.000000000 +0000 @@ -1,4 +1,4 @@ -prefix=/usr +prefix=/usr/arm-linux-gnueabi exec_prefix=${prefix} libdir=${exec_prefix}/lib includedir=${prefix}/include Multiarch support already requires that this behaviour is modified. It may be useful to implement /usr/bin/arm-linux-gnueabi-pkg-config as a wrapper for /usr/bin/pkg-config that ensures that prefixes and libdir settings are correctly set *only* within that wrapper. The wrapper could have support for particular options / environment variables that support overriding the detection of the build and host arguments. > A further wrinkle is that $host-pkg-config will indeed be > automatically used for anyone who's using the PKG_CHECK_MODULES > Autoconf macro (which I suspect is the majority, but certainly won't > be everyone), but not for anyone who's just calling pkg-config > directly. A wrapper that knows about dpkg-architecture should be able to handle that. > Should we just say that if they're doing that then they > need to compare build and host architectures directly, the way > they're probably already doing for gcc and friends? Would this be a > serious regression for Emdebian that would need to be tackled in > advance? Emdebian Crush (the cross-built distribution) is already in a serious regression but this provides an opportunity for major changes whilst everything is "up for grabs". Fixing this will not cause any regression in Emdebian that is not already present - the implementation of multiarch is the ideal time to do "TheRightThing". -- Neil Williams ============= http://www.data-freedom.org/ http://www.linux.codehelp.co.uk/ http://e-mail.is-not-s.ms/
pgpzll42Auv1z.pgp
Description: PGP signature
