Re: proper syslinux settings for ext4 boot, dm-crypt + btrfs subvol root?
On Sun, Feb 12, 2017 at 5:11 PM, Chris Murphy wrote: > On Fri, Feb 10, 2017 at 5:05 PM, John Hendy wrote: >> On Fri, Feb 10, 2017 at 5:49 PM, Hugo Mills wrote: >>> On Fri, Feb 10, 2017 at 11:40:41AM -0600, John Hendy wrote: Greetings, >> >> >>>Just as an aside -- it doesn't affect you here, but you might want >>> to be aware -- syslinux doesn't support multi-device btrfs. I've been >>> bitten by that before, and it's a consideration for you should you >>> wish to add another device to this FS. Also, I haven't tried it, but I >>> suspect that it may not support compression in btrfs either. >> >> Aside appreciated! I don't plan to add a second device, so that's alright. > > Nonfactor in your case because your boot volume is ext4 not Btrfs, so > none of the Btrfs limitations apply to the bootloader. > > As a relative noob, when a mailing list brings something to my attention I want to make sure I understand what's being said. I said in my very original post that /boot was ext4... but it was brought up that syslinux doesn't support multi-device and compression, so I wanted to know compression on *what*? >> >> I've run into the statement on compression as well... does this mean >> the device it's *on* or the device I want to mount as root? > > rootflags applies mount options to the root filesystem only. > >> >> I have to assume the former, and ran across this in my preparation, >> which is why I used ext4. Given that I've been able to specify >> rootflags=compress=lzo in my APPEND line, this is also in /etc/fstab >> and it booted... I'm using this as evidence that it must support >> compression on a root btrfs device? Any reason to think otherwise? > > I don't understand the question. What is "it" when you ask "it must support"? I was just thinking aloud as an extension of the above. Since I specified compress=lzo in rootflags, on the / mount options in fstab, and I was able to get a successful boot with these options... presumably syslinux (*it*) supports that setup (namely: compression on btrfs root). John > > > -- > Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: proper syslinux settings for ext4 boot, dm-crypt + btrfs subvol root?
On Sun, Feb 12, 2017 at 5:07 PM, Chris Murphy wrote: > On Fri, Feb 10, 2017 at 4:34 PM, John Hendy wrote: > >> >> I did a couple of things after I sent this. For one, I ditched >> arch/root. Now I just have arch. That let me get away from the call to >> a nested subvol. >> >> That on it's own didn't work, but I started playing with the >> rootflags. I haven't tried one by one, but re-specifying my mount >> options seems to work. So instead of just rootflags=subvol=arch, I >> have: >> >> rootflags=compress=lzo,subvol=arch,ssd,discard >> >> Is there a reason that would make a difference? > > The only change that should matter is subvol path change, and I'm not > even convinced of that. But I don't even know where exactly the > failure is. It sounds like the failure happens right away and you get > back to the syslinux menu; that suggests to me the kernel isn't even > loaded let alone executed, and the boot parameters don't matter until > the kernel is executed. > > Indeed, and see my most recent reply. Unfortunately, I think the issue might have been paring down extra options (I had some libata.force stuff), including the loading of intel-ucode at the same time I was adding explicit rootflags options. Thus, compress=lzo stood out as a notable change, but I think I overlooked the real culprit which was intel-ucode. So, as you said, I think the issue was it not even getting past the bootloader and not what I initially thought. John > > -- > Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: proper syslinux settings for ext4 boot, dm-crypt + btrfs subvol root?
On Fri, Feb 10, 2017 at 5:05 PM, John Hendy wrote: > On Fri, Feb 10, 2017 at 5:49 PM, Hugo Mills wrote: >> On Fri, Feb 10, 2017 at 11:40:41AM -0600, John Hendy wrote: >>> Greetings, > > >>Just as an aside -- it doesn't affect you here, but you might want >> to be aware -- syslinux doesn't support multi-device btrfs. I've been >> bitten by that before, and it's a consideration for you should you >> wish to add another device to this FS. Also, I haven't tried it, but I >> suspect that it may not support compression in btrfs either. > > Aside appreciated! I don't plan to add a second device, so that's alright. Nonfactor in your case because your boot volume is ext4 not Btrfs, so none of the Btrfs limitations apply to the bootloader. > > I've run into the statement on compression as well... does this mean > the device it's *on* or the device I want to mount as root? rootflags applies mount options to the root filesystem only. > > I have to assume the former, and ran across this in my preparation, > which is why I used ext4. Given that I've been able to specify > rootflags=compress=lzo in my APPEND line, this is also in /etc/fstab > and it booted... I'm using this as evidence that it must support > compression on a root btrfs device? Any reason to think otherwise? I don't understand the question. What is "it" when you ask "it must support"? -- Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: proper syslinux settings for ext4 boot, dm-crypt + btrfs subvol root?
On Fri, Feb 10, 2017 at 4:34 PM, John Hendy wrote: > > I did a couple of things after I sent this. For one, I ditched > arch/root. Now I just have arch. That let me get away from the call to > a nested subvol. > > That on it's own didn't work, but I started playing with the > rootflags. I haven't tried one by one, but re-specifying my mount > options seems to work. So instead of just rootflags=subvol=arch, I > have: > > rootflags=compress=lzo,subvol=arch,ssd,discard > > Is there a reason that would make a difference? The only change that should matter is subvol path change, and I'm not even convinced of that. But I don't even know where exactly the failure is. It sounds like the failure happens right away and you get back to the syslinux menu; that suggests to me the kernel isn't even loaded let alone executed, and the boot parameters don't matter until the kernel is executed. > I started wondering > about the compression immediately and I admit I'm not savvy on how > that works. Not a factor for two reasons: a. Your boot volume is not Btrfs, so the fact syslinux (actually extlinux in this case) doesn't support compression doesn't matter. Its job is just to load and execute the kernel and initrams. b. The file system metadata contains information per extent whether it's compressed and with which algorithm. So Btrfs doesn't need the compression hint as a boot parameter either. This mount option is only needed to tell the kernel to write new extents with compression. -- Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: proper syslinux settings for ext4 boot, dm-crypt + btrfs subvol root?
On Fri, Feb 10, 2017 at 11:40 AM, John Hendy wrote: > Greetings, > > > I'm trying to utilize btrfs subvols to allow me to boot separate > distros without having to create so many partitions. I'm on Arch Linux > and not finding the wiki super helpful with specifics on dm-crypt/luks > and btrfs. [snip] > Here's the setup I'm looking for once booted, with the /mnt/btf becoming / > > $ mount > /dev/mapper/btr on /mnt/btr type btrfs > (rw,relatime,ssd,space_cache,subvolid=259,subvol=/arch/root) > /dev/mapper/btr on /mnt/btr/home type btrfs > (rw,relatime,ssd,space_cache,subvolid=260,subvol=/arch/home) > /dev/sdb1 on /mnt/btr/boot type ext4 (rw,relatime,data=ordered) > > My functioning syslinux.cfg: > > LABEL arch-ssd > MENU LABEL arch-ssd-uuid > LINUX ../vmlinuz-linux > APPEND luks.uuid=7101e83b-31c0-4cdf-bc07-678e00e19c32 > root=UUID=eb20c219-0df8-4051-bad2-39d57aed7b59 luks.allow-discards rw > INITRD ../intel-ucode.img,../initramfs-linux.img > > I tried this: > > LABEL arch-btrfs > MENU LABEL arch-btrfs-uuid > LINUX ../vmlinuz-linux > APPEND luks.uuid=bcacb6d5-2874-4652-a25d-88b0bf3bbce8 > root=UUID=e1284231-c264-4944-807d-5fcb1832ce47 > rootflags=subvol=/arch/root luks.allow-discards rw > INITRD ../intel-ucode.img,../initramfs-linux.img > > Current behavior is I successfully get a syslinux boot menu (at least > after I disabled the 64bit default option), but selecting the above > entry just refreshes the menu and I'm stuck there with an automatic > countdown from 5sec and then back to 5sec when it hits 0. It's like it > just keeps pointing at the /dev/sdb MBR or can't find something and > thus tries again? Closing the loop on this. It took a while but after a couple of install option tries, I got ubuntu planing nicely with syslinux so I figured the arch process would be piece of cake. I had everything setup and then went to boot and got the same behavior... I tabbed to get out the loop behavior and thought "ah! I wonder if I installed the intel-ucode package?" So, previously, I thought it was specifying the compress=lzo option, but now I think that was just one of many steps I did when fiddling with the syslinux entry above. I'm guessing another was removing the loading of intel-ucode. I don't want to go through all the steps again to be sure, but at least wanted this updated with my experience in case someone else stumbles on the issue. Thus far: - I have a single /boot partition (sdb1) with syslinux installed - I can boot either arch or ubuntu from separate btrfs subvols on the same encrypted partition (sdb2) - thus far I just have an arch and ubuntu top level subvol with nested home subvol under each - I *don't* think the issue was loading a nested subvol for root [1] - this is awesome and so much cooler than having to guess what size to make everything ahead of time Thanks for the quick responses all and like most issues the problem was almost certainly me! John [1] this post suggests loading __active/rootvol - https://www.brunoparmentier.be/blog/how-to-install-arch-linux-on-an-encrypted-btrfs-partition.html [snip] > Best regards, > John -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: proper syslinux settings for ext4 boot, dm-crypt + btrfs subvol root?
On Fri, Feb 10, 2017 at 06:05:57PM -0600, John Hendy wrote: > On Fri, Feb 10, 2017 at 5:49 PM, Hugo Mills wrote: > > On Fri, Feb 10, 2017 at 11:40:41AM -0600, John Hendy wrote: > >> Greetings, > > > >Just as an aside -- it doesn't affect you here, but you might want > > to be aware -- syslinux doesn't support multi-device btrfs. I've been > > bitten by that before, and it's a consideration for you should you > > wish to add another device to this FS. Also, I haven't tried it, but I > > suspect that it may not support compression in btrfs either. > > Aside appreciated! I don't plan to add a second device, so that's alright. > > I've run into the statement on compression as well... does this mean > the device it's *on* or the device I want to mount as root? The FS that the kernel and initramfs live on. The bootloader is responsible for finding the two files containing the kernel and the initramfs on the FS and loading/running them. (It may also need to find some components if itself on there; GRUB looks for config files, for example). Once the kernel and initramfs have started, it's up to the kernel and the early userspace environment to deal with, and it's not the bootloader's problem. > I have to assume the former, and ran across this in my preparation, > which is why I used ext4. Given that I've been able to specify > rootflags=compress=lzo in my APPEND line, this is also in /etc/fstab > and it booted... I'm using this as evidence that it must support > compression on a root btrfs device? Any reason to think otherwise? > Their wiki appears to talk about just the boot device: > - http://www.syslinux.org/wiki/index.php?title=Filesystem#Btrfs rootflags=... is ignored by the bootloader and is passed to the initramfs environment. > Even with ext4, my current arch install is < May 2016 which is when > mkfs.ext4 auto-implemented some sort of new 64bit feature so that > caught me off guard. Initially syslinux just spit out a "can't find > ldlinux.sys" or similar. > > Given I'm early enough here to abort the plan without much fuss, this > sort of aside is quite nice to consider if there's something > inherently wrong with this! So far I have a booting arch and I'm using > debootstrap to manually install Ubuntu since it's installer has ~zero > customization options to prep your mount points if you've already got > them ready... The main problem with multi-boot environments on btrfs is that most distribution installers don't know about btrfs subvols, and won't install on them. debootstrap is about the only way I know of that will cope with it. (Although I suspect that some of the from-source distributions like Gentoo will manage it given enough persuasion). A secondary problem is that every distribution assumes that it's the only thing that's managing /boot and the corresponding bootloader, kernel and initramfs. This has always been problematic -- it was a problem when I was multi-booting back in 1999 -- and isn't restricted to btrfs-based environments. I don't have a good solution for managing it, I'm afraid. Hugo. -- Hugo Mills | For months now, we have been making triumphant hugo@... carfax.org.uk | retreats before a demoralised enemy who is advancing http://carfax.org.uk/ | in utter disorder. PGP: E2AB1DE4 | Eric Frank Russell, Wasp signature.asc Description: Digital signature
Re: proper syslinux settings for ext4 boot, dm-crypt + btrfs subvol root?
On Fri, Feb 10, 2017 at 5:49 PM, Hugo Mills wrote: > On Fri, Feb 10, 2017 at 11:40:41AM -0600, John Hendy wrote: >> Greetings, >Just as an aside -- it doesn't affect you here, but you might want > to be aware -- syslinux doesn't support multi-device btrfs. I've been > bitten by that before, and it's a consideration for you should you > wish to add another device to this FS. Also, I haven't tried it, but I > suspect that it may not support compression in btrfs either. Aside appreciated! I don't plan to add a second device, so that's alright. I've run into the statement on compression as well... does this mean the device it's *on* or the device I want to mount as root? I have to assume the former, and ran across this in my preparation, which is why I used ext4. Given that I've been able to specify rootflags=compress=lzo in my APPEND line, this is also in /etc/fstab and it booted... I'm using this as evidence that it must support compression on a root btrfs device? Any reason to think otherwise? Their wiki appears to talk about just the boot device: - http://www.syslinux.org/wiki/index.php?title=Filesystem#Btrfs Even with ext4, my current arch install is < May 2016 which is when mkfs.ext4 auto-implemented some sort of new 64bit feature so that caught me off guard. Initially syslinux just spit out a "can't find ldlinux.sys" or similar. Given I'm early enough here to abort the plan without much fuss, this sort of aside is quite nice to consider if there's something inherently wrong with this! So far I have a booting arch and I'm using debootstrap to manually install Ubuntu since it's installer has ~zero customization options to prep your mount points if you've already got them ready... Thanks, John > >Hugo. > > -- > Hugo Mills | All mushrooms are edible, but some are only edible > hugo@... carfax.org.uk | once. > http://carfax.org.uk/ | > PGP: E2AB1DE4 | -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: proper syslinux settings for ext4 boot, dm-crypt + btrfs subvol root?
On Fri, Feb 10, 2017 at 11:40:41AM -0600, John Hendy wrote: > Greetings, > > > I'm trying to utilize btrfs subvols to allow me to boot separate > distros without having to create so many partitions. I'm on Arch Linux > and not finding the wiki super helpful with specifics on dm-crypt/luks > and btrfs. > > Requested info: > > $ uname -a > Linux arch_zbook 4.9.8-1-ARCH #1 SMP PREEMPT Mon Feb 6 12:59:40 CET > 2017 x86_64 GNU/Linux > > $ btrfs --version > btrfs-progs v4.9 > > $ sudo btrfs fi show > Label: 'bigD' uuid: e1284231-c264-4944-807d-5fcb1832ce47 > Total devices 1 FS bytes used 1.27GiB > devid1 size 232.38GiB used 3.02GiB path /dev/mapper/btr [snip] > My functioning syslinux.cfg: [snip] > Current behavior is I successfully get a syslinux boot menu (at least > after I disabled the 64bit default option), but selecting the above > entry just refreshes the menu and I'm stuck there with an automatic > countdown from 5sec and then back to 5sec when it hits 0. It's like it > just keeps pointing at the /dev/sdb MBR or can't find something and > thus tries again? Just as an aside -- it doesn't affect you here, but you might want to be aware -- syslinux doesn't support multi-device btrfs. I've been bitten by that before, and it's a consideration for you should you wish to add another device to this FS. Also, I haven't tried it, but I suspect that it may not support compression in btrfs either. Hugo. -- Hugo Mills | All mushrooms are edible, but some are only edible hugo@... carfax.org.uk | once. http://carfax.org.uk/ | PGP: E2AB1DE4 | signature.asc Description: Digital signature
Re: proper syslinux settings for ext4 boot, dm-crypt + btrfs subvol root?
On Fri, Feb 10, 2017 at 2:03 PM, Chris Murphy wrote: > This is not a Btrfs problem yet. Syslinux isn't even getting the > kernel and initramfs loaded, otherwise you'd get a failure during > boot. But instead you're at the bootloader still. > > One thing to realize is that syslinux only finds kernels and initramfs > on the local file system. So assuming this syslinux is on /dev/sda1: > UUID="37441f68... ext2 volume; then it can only load kernels and > initramfs from that file system. So I'm not sure what the purpose of > /dev/sdb1: UUID="e046823f ext4 is for, but if you use it for the boot > volume, you have to use extlinux install extlinux there *and* you have > to make sure the MBR has sdb1's active bit set *and* you have to make > sure the firmware boot manager is pointed to /dev/sdb. For sure. The whole install is on /dev/sdb - sdb1 is ext4 /boot partition - sdb1 was mounted to /path/to/boot when I installed Arch - I did extlinux --install on sdb1 - boot flag set on sdb1 - syslinux mbr dd'd to /dev/sdb I think that part is ok. > > But I don't think that's necessary, mainly you just need a really big > ext4 volume to hold all the distro kernels in one volume so that > syslinux has access to them. If you want separate boot volumes for > each distro then you have no choice but to use GRUB which can find and > load kernel/initramfs on any drive, any volume, with pretty much any > file system. This is encouraging as I was particularly hoping to *avoid* separate boot partitions. > > So the question is why won't that boot entry load the already working > kernel and initramfs? I don't have a good explanation for that. What > should happen if anything in APPEND is wrong, is you still get kernel > and initramfs loading, but then end up at a dracut shell. Or maybe it > crashes earlier due to some incompatibility by mixing kernel/initramfs > with the wrong rootfs (different kernel versions for example since all > the modules are version specific and those will be on Btrfs not on > boot). > I did a couple of things after I sent this. For one, I ditched arch/root. Now I just have arch. That let me get away from the call to a nested subvol. That on it's own didn't work, but I started playing with the rootflags. I haven't tried one by one, but re-specifying my mount options seems to work. So instead of just rootflags=subvol=arch, I have: rootflags=compress=lzo,subvol=arch,ssd,discard Is there a reason that would make a difference? I started wondering about the compression immediately and I admit I'm not savvy on how that works. When I installed things were mounted with compress=lzo. Would this mean syslinux was maybe trying to mount it uncompressed and thus files are jumbled (like trying to run cat on a .zip)? Just wild imagining here. Short story is I can now boot! I just want to figure out what happened and update the Arch wiki as I haven't seen much definitive material on this. > I'd say you need some syslinux debug option set to find out what it's > tripping up on, because there's not enough information without that. > Maybe there's something in APPEND that it doesn't like, but I don't > think syslinux parses that line, it just adds it to the kernel boot > param line handed off to the kernel. But it's not even getting that > far. > > The UUID e1284231- is the correct one to use for Btrfs. > > I have not tried a double nested subvolume. I think you're better off > with directories at the top level for each distro, and then use > subvolumes in those directories for root and home. Note that the > Ubuntu and opensuse GUI installers will make this impossible, they > will not install to a user defined subvolume, they insist on creating > things at the top level. Fedora's installer will let you do this > except for rootfs, it requires that it be a new subvolume which you > can name but the UI won't let you stick it in a directory, or create > nested subvolumes. You could troubleshoot your existing APPEND by > using rootflags=subvolid= and specifying the ID of the arch/root > subvolume. But I'm suspicious whether that's a problem. You could try > removing the leading / because the path to the subvolume is really > subvol=arch/root. Hmmm. As a separate question, I noticed that if I did: mount -o subvol=arch/root /mnt/btr ## note no leading slash and then looked at `genfstab -p -U /mnt/btr`, I got both of these in the options: subvol=arch/root, subvol=/arch/root So that tripped me up big time. In blogs/forums I don't tend to see folks specifying a leading slash, but the system seemed to use it? Thanks for the assistance! John > > That's all I'm thinking of at the moment. > > Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: proper syslinux settings for ext4 boot, dm-crypt + btrfs subvol root?
This is not a Btrfs problem yet. Syslinux isn't even getting the kernel and initramfs loaded, otherwise you'd get a failure during boot. But instead you're at the bootloader still. One thing to realize is that syslinux only finds kernels and initramfs on the local file system. So assuming this syslinux is on /dev/sda1: UUID="37441f68... ext2 volume; then it can only load kernels and initramfs from that file system. So I'm not sure what the purpose of /dev/sdb1: UUID="e046823f ext4 is for, but if you use it for the boot volume, you have to use extlinux install extlinux there *and* you have to make sure the MBR has sdb1's active bit set *and* you have to make sure the firmware boot manager is pointed to /dev/sdb. But I don't think that's necessary, mainly you just need a really big ext4 volume to hold all the distro kernels in one volume so that syslinux has access to them. If you want separate boot volumes for each distro then you have no choice but to use GRUB which can find and load kernel/initramfs on any drive, any volume, with pretty much any file system. So the question is why won't that boot entry load the already working kernel and initramfs? I don't have a good explanation for that. What should happen if anything in APPEND is wrong, is you still get kernel and initramfs loading, but then end up at a dracut shell. Or maybe it crashes earlier due to some incompatibility by mixing kernel/initramfs with the wrong rootfs (different kernel versions for example since all the modules are version specific and those will be on Btrfs not on boot). I'd say you need some syslinux debug option set to find out what it's tripping up on, because there's not enough information without that. Maybe there's something in APPEND that it doesn't like, but I don't think syslinux parses that line, it just adds it to the kernel boot param line handed off to the kernel. But it's not even getting that far. The UUID e1284231- is the correct one to use for Btrfs. I have not tried a double nested subvolume. I think you're better off with directories at the top level for each distro, and then use subvolumes in those directories for root and home. Note that the Ubuntu and opensuse GUI installers will make this impossible, they will not install to a user defined subvolume, they insist on creating things at the top level. Fedora's installer will let you do this except for rootfs, it requires that it be a new subvolume which you can name but the UI won't let you stick it in a directory, or create nested subvolumes. You could troubleshoot your existing APPEND by using rootflags=subvolid= and specifying the ID of the arch/root subvolume. But I'm suspicious whether that's a problem. You could try removing the leading / because the path to the subvolume is really subvol=arch/root. Hmmm. That's all I'm thinking of at the moment. Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
