Re: Which kernels to include on ISOs? (Was: Re: Netboot Xen images for amd64)
On Thu, 2009-06-04 at 01:10 +0200, Frans Pop wrote: [...snipped a bunch of good points which I won't contest...] Personally I would say that full CD and DVD are not even very interesting for Xen installs: their content is desktop oriented, so why download a lot of shite you're not going to be using anyway? Netinst and businesscard are much more relevant, but as explained earlier those have space restrictions. But there is one image that might exactly fit the bill: the i386/amd64/ppc multi-arch netinst CD. Current size (Lenny): 488MB. For that it does not matter if it would grow a bit, and it's targeted exactly at your users: (semi-)professional sysadmins. That sounds ideal, thanks for the suggestion. Am I right in thinking that a netinst CD includes the base system but not any of the other stuff? (as opposed to just including the installer itself) Only problem is that implementing adding Xen to just that image will require a fair few changes in debian-cd. In configuration, but I think also in code (you'll need to introduce a concept of variants within arches for D-I tasks). It'll not be trivial to implement that cleanly, though it should certainly be possible. I'm afraid that I have no real affinity with Xen (the current discussion on lkml rather amuses me TBH), so I don't think I'd work on that [2]. That's fine I'm willing to have a stab at it, I've been exploring the debian-cd code already since I thought I might have to go a-hacking and I've done some (small) d-i stuff in the past. Do you have any specific guidance regarding the direction you'd like to see me take with this or shall I just dive in and see what I come up with? Initially from the debian-cd end I'm thinking along the lines of adding -DMULTIARCH=1 (where appropriate) to the preprocessor when generating the package lists. Thanks, Ian. -- Ian Campbell Remember, in 2039, MOUSSE PASTA will be available ONLY by prescription!! signature.asc Description: This is a digitally signed message part
Re: Which kernels to include on ISOs? (Was: Re: Netboot Xen images for amd64)
On Thursday 04 June 2009, Ian Campbell wrote: On Thu, 2009-06-04 at 01:10 +0200, Frans Pop wrote: [...snipped a bunch of good points which I won't contest...] :-) But there is one image that might exactly fit the bill: the i386/amd64/ppc multi-arch netinst CD. Current size (Lenny): 488MB. For that it does not matter if it would grow a bit, and it's targeted exactly at your users: (semi-)professional sysadmins. That sounds ideal, thanks for the suggestion. Am I right in thinking that a netinst CD includes the base system but not any of the other stuff? (as opposed to just including the installer itself) Correct. But if you implement it correctly, building a Xen businesscard image (multi-arch or not) should automatically be supported too. Only problem is that implementing adding Xen to just that image will require a fair few changes in debian-cd. In configuration, but I think also in code (you'll need to introduce a concept of variants within arches for D-I tasks). It'll not be trivial to implement that cleanly, though it should certainly be possible. Do you have any specific guidance regarding the direction you'd like to see me take with this or shall I just dive in and see what I come up with? Initially from the debian-cd end I'm thinking along the lines of adding -DMULTIARCH=1 (where appropriate) to the preprocessor when generating the package lists. That sounds wrong. The condition is not am I building multiarch?. It should be please build a multi-arch image, but add Xen variations. I.e, building a normal m-a image should still be possible and should be default. The challenge is to find a way to specify that _optionally_ the bits needed for Xen should be added on top of the normal image. There are three things you need to accomplish: 1) include the kernel udebs for 686-bigmem on the image 2) include the 686-bigmem kernel-image deb (+ maybe headers packages?) 3) support it in build scripts on farbror (d-cd buildd) and possibly the easybuild script (recommended for testing!) For 1) you need to somehow ensure the 686-bigmem udebs do *not get excluded* for only (!) the m-a netinst CD (see tools/generate_di_list and data/squeeze/exclude-udebs-i386), but not for other images. Similar challenge for 2). Kernel packages get included through the script ools/generate_di+k_list, which currently does not allow for variations within an architecture. Ignore the tasks directory: that is only used for full CD and DVD images, not for netinst and businesscard. For 3) you can find the current build scripts used on farbror at [1]. The please add Xen support option should be set in the cronjob.daily script in the bit starting with: for arch in $ARCHES; do echo Building $arch: if [ $arch = multi-arch ] ; then echo i386/amd64/ppc sid netinst [...] echo i386/amd64/ppc squeeze netinst IMO it should be something like adding 'VARIANT=xen', which should automatically only result in changes for arches that support the variant (the m-a CD also includes powerpc, which does not support Xen, and thus should build without any differences with or without the VARIANT option passed). Bonus points if it can be implemented in such a way that 'VARIANT=foo,bar,baz' would be supported too (but that might be tricky with the exclude-udeb files; maybe those need a different implementation anyway?). Such a variant option could be included in the default CONF.sh (commented out by default) and could also be added in the easybuild.sh script (either commented out by default or activated with a new parameter). All the above is subject to comments from Steve McIntyre. Please check your planned solution with him (by mailing d-cd list). Cheers, FJP [1] http://svn.debian.org/wsvn/debian-cd/setup/#_setup_ -- To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Which kernels to include on ISOs? (Was: Re: Netboot Xen images for amd64)
On Thursday 04 June 2009, Frans Pop wrote: There are three things you need to accomplish: 1) include the kernel udebs for 686-bigmem on the image 2) include the 686-bigmem kernel-image deb (+ maybe headers packages?) 3) support it in build scripts on farbror (d-cd buildd) and possibly the easybuild script (recommended for testing!) Ah, and you need to actually include the correct D-I initrd and kernel on the CD of course. And you'll need to modify the syslinux boot menu so that an install Xen option gets added. Both of those mean changes in the boot-x86 and x86-desktop.sh scripts in tools/boot/sqeeze/ [1]. A regular install should remain the default of course. Cheers, FJP [1] I committed some fairly heavy changes in both D-I and d-cd early this week for that. You may want to look at both the old and new version as the old version shows how you could manipulate the syslinux config to add the options in d-cd. I don't think the base syslinux config in d-i should need to be changed for this. -- To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Which kernels to include on ISOs? (Was: Re: Netboot Xen images for amd64)
On Thu, 2009-06-04 at 10:42 +0200, Frans Pop wrote: On Thursday 04 June 2009, Frans Pop wrote: There are three things you need to accomplish: 1) include the kernel udebs for 686-bigmem on the image 2) include the 686-bigmem kernel-image deb (+ maybe headers packages?) 3) support it in build scripts on farbror (d-cd buildd) and possibly the easybuild script (recommended for testing!) Ah, and you need to actually include the correct D-I initrd and kernel on the CD of course. Thank you for the pointers and advice, I think I've got enough info to make a start. And you'll need to modify the syslinux boot menu so that an install Xen option gets added. Xen doesn't boot via syslinux so I don't think that will be necessary. I was thinking I would expand the existing xm-debian.cfg (currently included in the netboot-xen d-i output) to include the configurability to boot from a CD device too. Thanks again! Ian -- Ian Campbell Current Noise: My Dying Bride - A Doomed Lover Your love life will be... interesting. -- To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: Which kernels to include on ISOs? (Was: Re: Netboot Xen images for amd64)
On Friday 22 May 2009, Ian Campbell wrote: On Sat, 2009-04-18 at 12:49 +0200, Frans Pop wrote: Also note that adding the 686-bigmem kernel to CD images still has a rather high impact (as you'd need to add both the kernel udeb _and_ the regular kernel-image deb), which is rather undesirable for both the netinst CD and the regular CD1. In the first case because of the size increase it would cause [1], in the second case because it would push packages important for the desktop task off CD1. Even the DVD1 image is getting tight as we currently support installation of _all_ desktop environments from it. The margin for businesscard images depends mainly on the capacity of actual businesscard sized media. I've been thinking about this some more and I wonder if 486 + 686 is still the best option for DVD1 -- as opposed to 486 + 686-bigmem. Yes, I'm very sure it is. IMHO the set of machines which benefit from a 686 kernel but are unable to run a 686-bigmem kernel is already small and getting smaller. I reckon those machine would be fine with a 486 kernel anyway, the 686 optimisations don't buy you that much and SMP-but-non-PAE machine are an even smaller set (if such a thing even exists, I'm not sure). I'm afraid I quite strongly disagree with you on this. 1) 686-bigmem has a significant performance penalty for systems that don't need it (otherwise it would be a non-issue as the option would just be enabled in the generic -686 kernel). 2) I guestimate that 90% of systems that really need a -686 kernel have less memory than the limit supported by the generic -686 kernel. Random sample: I have three desktops and a laptop running 686, and none of them comes even close to that limit. 3) IIRC there are fairly significant differences between -486 and -686 performance on normal Pentiums and AMD boxes. 4) According to Linus, anybody who fits huge amounts of memory in normal Pentium systems is insane as the lowmem/highmem distinction will always continue to hurt you. 5) I would think that the set of machines you're aiming at is mostly 64-bit capable (the 32-bit segment can hardly be said to be growing). In that case they really should be running either the amd64 arch, or, if they really want a 32-bit userland, i386, but with the -amd64 kernel [0]! I expect that last would support Xen as well. IMO your argumentation is strongly colored by your own goals here. New machines these days already have 1-2G as a pretty basic minimum and And are 64-bit capable and thus shouldn't be using -686 at all! I predict that when squeeze arrives getting on for 4G will be a common default. A PAE kernel starts to become necessary around 3.5G anyway due to the PCI hole and even for machines with 3.5G of RAM you get things like NX support thrown in. The 686 kernel is still just an aptitude run away. *shrug* so is the -bigmem kernel [1]. A more important issue for me is that we should make sure to install the correct *generic* kernel for regular users and leave specialized kernels to experts. And now something slightly more constructive. Personally I would say that full CD and DVD are not even very interesting for Xen installs: their content is desktop oriented, so why download a lot of shite you're not going to be using anyway? Netinst and businesscard are much more relevant, but as explained earlier those have space restrictions. But there is one image that might exactly fit the bill: the i386/amd64/ppc multi-arch netinst CD. Current size (Lenny): 488MB. For that it does not matter if it would grow a bit, and it's targeted exactly at your users: (semi-)professional sysadmins. Only problem is that implementing adding Xen to just that image will require a fair few changes in debian-cd. In configuration, but I think also in code (you'll need to introduce a concept of variants within arches for D-I tasks). It'll not be trivial to implement that cleanly, though it should certainly be possible. I'm afraid that I have no real affinity with Xen (the current discussion on lkml rather amuses me TBH), so I don't think I'd work on that [2]. Cheers, FJP [0] Supporting automatic selection of the -amd64 flavor in D-I for i386 has never really been discussed. It would be interesting, but again the space limitations would have to be carefully considered. [1] Yes, I know, that doesn't work if you want to install Xen. [2] Though I would consider doing it for a suitable bounty. -- To UNSUBSCRIBE, email to debian-cd-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Which kernels to include on ISOs? (Was: Re: Netboot Xen images for amd64)
[adding debian-cd. The context is trying to figure out how to support installation from CD/DVD into a Xen guest -- this requires a PAE (==686-bigmem) kernel for 32 bit to run under Xen, both for the installer itself and the final target kernel, hence the is a need to either fit a 686-bigmem-di udeb + regular 686-bigmem deb onto an existing CD/DVD or swap out the 686 kernel for 686-bigmem on the DVD or create a new CD1 variant just for Xen or some other option I hasn't occurred to be yet ] On Sat, 2009-04-18 at 12:49 +0200, Frans Pop wrote: Also note that adding the 686-bigmem kernel to CD images still has a rather high impact (as you'd need to add both the kernel udeb _and_ the regular kernel-image deb), which is rather undesirable for both the netinst CD and the regular CD1. In the first case because of the size increase it would cause [1], in the second case because it would push packages important for the desktop task off CD1. Even the DVD1 image is getting tight as we currently support installation of _all_ desktop environments from it. The margin for businesscard images depends mainly on the capacity of actual businesscard sized media. I've been thinking about this some more and I wonder if 486 + 686 is still the best option for DVD1 -- as opposed to 486 + 686-bigmem. IMHO the set of machines which benefit from a 686 kernel but are unable to run a 686-bigmem kernel is already small and getting smaller. I reckon those machine would be fine with a 486 kernel anyway, the 686 optimisations don't buy you that much and SMP-but-non-PAE machine are an even smaller set (if such a thing even exists, I'm not sure). New machines these days already have 1-2G as a pretty basic minimum and I predict that when squeeze arrives getting on for 4G will be a common default. A PAE kernel starts to become necessary around 3.5G anyway due to the PCI hole and even for machines with 3.5G of RAM you get things like NX support thrown in. The 686 kernel is still just an aptitude run away. Ian. -- Ian Campbell In West Union, Ohio, No married man can go flying without his spouse along at any time, unless he has been married for more than 12 months. signature.asc Description: This is a digitally signed message part
