Easy crossbuilding of package sets/images
One output from the 11.09 release was a reasonably painless way of cross-building whole images against an archive, which also forms the basis for an auto-crossbuilder. There is a HOWTO (for building linaro-nano images) here: https://wiki.linaro.org/Platform/DevPlatform/CrossCompile/CrossBuildNano (It uses armel as the example, but should work the same for armhf) Summary of process -- Essentially you can generate a cross-building chroot in one command (using multistrap), chroot into that (using schroot), and give the builder (xbuilder-simple) a list of packages to build - either a pre-generated one for an image, or your own. It chunters through and builds them all (using xdeb), and leaving build-logs for each package. Then at the end (from outside the chroot) you cross-generate an image from the debs (using multistrap, but the pile could be input for a different tool if you prefer). -- To make this work usefully against a stable (natty) baseline, updated versions of both tools and packages are in two PPAs at: https://launchpad.net/~linaro-foundations/+archive/cross-build-tools and https://launchpad.net/~linaro-foundations/+archive/cross-alip Caveats --- This is currently a technology demonstration in so far as some of the packages needed for a nano image don't successfully cross-build, so you can't actually currently cross-build all of it, but that should be fixed quite soon. Anyone who wants to help with that is very welcome. I've filed current status and remaining bugs here: https://wiki.linaro.org/Platform/DevPlatform/CrossCompile/LinaroNanoStatus The autobuilder can actually run just as well using pdebuild-cross as xdeb to do the build-work, but that's not been tested for a while and is not covered in the HOWTO. The existing xbuilder is pretty stupid (that's why it's called xbuilder-simple) and it does not yet fully ensure a clean build environment every time (when using xdeb, it should if using pbuilder-cross), but it does enough to work reasonably well in practice. A more rigorous environment reset is on the list of improvements RSN. Ongoing work The current focus is on getting the remaining packages cross-building so that the whole process works to completeion without cheating by bringing in pre-built packages from the existing archive (That's a useful way to proceed if you want to use this tech today - just adjust the multistrap config to include the base natty armel/armhf archive too) Once this is completed I'll be setting up a continuously-running autobuilder so that the cross-buildability or otherwise of packages can be more easily discovered and more people can get involved in fixing up packages so that cross-building of larger images becomes realistic. It should be a simple matter of three multistrap config file changes to use this on an oneiric base, but it broke in a painfull mess of bind-mounts of deleted /run dirs when I tried it last week, so I'd hold off that for a little while, and stick with the nice natty-based examples. I'll add destructions to the HOWTO when I've got it going. I hope some of you find this useful, or at least interesting. I could really use some help on fixing up breakage in assorted packages, and this HOWTO makes it easy to set up a consistent build environment, so do give it a try and get stuck in if you'd like to see this working fully sooner rather than later. Reports of success, breakage, miserable failure, or how this almost-but-doesn't-quite address your use case are always welcome of course. Wookey -- Principal hats: Linaro, Emdebian, Wookware, Balloonboard, ARM http://wookware.org/ ___ linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
Re: Easy crossbuilding of package sets/images
On Wed, Oct 12, 2011 at 1:23 AM, Wookey woo...@wookware.org wrote: One output from the 11.09 release was a reasonably painless way of cross-building whole images against an archive, which also forms the basis for an auto-crossbuilder. There is a HOWTO (for building linaro-nano images) here: https://wiki.linaro.org/Platform/DevPlatform/CrossCompile/CrossBuildNano (It uses armel as the example, but should work the same for armhf) Summary of process -- Essentially you can generate a cross-building chroot in one command (using multistrap), chroot into that (using schroot), and give the builder (xbuilder-simple) a list of packages to build - either a pre-generated one for an image, or your own. It chunters through and builds them all (using xdeb), and leaving build-logs for each package. Then at the end (from outside the chroot) you cross-generate an image from the debs (using multistrap, but the pile could be input for a different tool if you prefer). -- To make this work usefully against a stable (natty) baseline, updated versions of both tools and packages are in two PPAs at: https://launchpad.net/~linaro-foundations/+archive/cross-build-tools and https://launchpad.net/~linaro-foundations/+archive/cross-alip Caveats --- This is currently a technology demonstration in so far as some of the packages needed for a nano image don't successfully cross-build, so you can't actually currently cross-build all of it, but that should be fixed quite soon. Anyone who wants to help with that is very welcome. I've filed current status and remaining bugs here: https://wiki.linaro.org/Platform/DevPlatform/CrossCompile/LinaroNanoStatus The autobuilder can actually run just as well using pdebuild-cross as xdeb to do the build-work, but that's not been tested for a while and is not covered in the HOWTO. The existing xbuilder is pretty stupid (that's why it's called xbuilder-simple) and it does not yet fully ensure a clean build environment every time (when using xdeb, it should if using pbuilder-cross), but it does enough to work reasonably well in practice. A more rigorous environment reset is on the list of improvements RSN. Ongoing work The current focus is on getting the remaining packages cross-building so that the whole process works to completeion without cheating by bringing in pre-built packages from the existing archive (That's a useful way to proceed if you want to use this tech today - just adjust the multistrap config to include the base natty armel/armhf archive too) Once this is completed I'll be setting up a continuously-running autobuilder so that the cross-buildability or otherwise of packages can be more easily discovered and more people can get involved in fixing up packages so that cross-building of larger images becomes realistic. Nice. So with the proper setup I could inject a different GCC and use this as a compiler testsuite? Perhaps with NEON and -O3 on by default? I know we don't support it, but could it build an ARMv5 subset of Ubuntu? -- Michael ___ linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
Re: Easy crossbuilding of package sets/images
+++ Michael Hope [2011-10-12 09:17 +1300]: On Wed, Oct 12, 2011 at 1:23 AM, Wookey woo...@wookware.org wrote: One output from the 11.09 release was a reasonably painless way of cross-building whole images against an archive, which also forms the basis for an auto-crossbuilder. snip Ongoing work The current focus is on getting the remaining packages cross-building so that the whole process works to completeion without cheating by bringing in pre-built packages from the existing archive (That's a useful way to proceed if you want to use this tech today - just adjust the multistrap config to include the base natty armel/armhf archive too) Once this is completed I'll be setting up a continuously-running autobuilder so that the cross-buildability or otherwise of packages can be more easily discovered and more people can get involved in fixing up packages so that cross-building of larger images becomes realistic. Nice. So with the proper setup I could inject a different GCC and use this as a compiler testsuite? Perhaps with NEON and -O3 on by default? By 'this' do you mean the forthcoming autobuilder set-up or the existing 'build me a package-set' tool? The 'build me a package set' stuff is essentially a local instantiation of the autobuilder. The public autobuilder will be using the standard distro-supplied toolchain, but of course you can set up whatever you like in your local instance. It's certainly pretty straighforward to build the build-env chroot with a new toolchain by just changing the multistrap config: [UbuntuToolchains] packages=g++-arm-linux-gnueabi libc-dev-armel-cross linux-libc-dev-armel-cross source=http://archive.ubuntu.com/ubuntu suite=natty components=main universe keyring=ubuntu-keyring to point at your repo of packaged toolchains. If your toolchains aren't packaged then it's a bit fiddlier, but you drop that stanza and specify a setupscript to untar a toolchain into place and do any other jiggery-pokery required. setupscript=/path/to/setup.sh You can only get NEON and O3 on by default by one of: a) building your toolchain to default that way b) nobbling the specs file after instalation c) putting in a wrapper to ensure those are set Nothing about this really makes that any harder/easier than it already was, except that adding it to the setupscript might ensure it didn't get forgotten. I know we don't support it, but could it build an ARMv5 subset of Ubuntu? Well, 'yes', and 'no', depending on where you start from. If you already have an armv5 archive to build against then 'yes, it's trivial', but otherwise: 'no, not really' - there is a piece missing for that: You need to be able to build everything from scratch using only stuff just built, which requires bootstrapping to break build-dependency loops. You can do it for subsets that have no build-dep loops, but this package-builder/autobuilder tech is explicitly designed to build packages against an archive. So that means existing armel/armhf debian or ubuntu repos. If you want to do a recursive build from scratch then xdeb alone is your best tool, but if there are build-dep loops in what you've asked for it'll bail. Adding bootstrapabilty to the archive is next once we have usefully functional cross-building without bootstrapping. I hope these sorts of questions are answered on the 'Crossbuilding quickstart' wiki page. Please poke me if not to add your case: https://wiki.linaro.org/Platform/DevPlatform/CrossCompile/CrossbuildingQuickStart Wookey -- Principal hats: Linaro, Emdebian, Wookware, Balloonboard, ARM http://wookware.org/ ___ linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
