Re: Mixing partitioned and non-partitioned discs in a RAID?
Am Tue, 13 Sep 2016 04:07:37 + (UTC) schrieb Duncan <1i5t5.dun...@cox.net>: > Kai Krakow posted on Tue, 13 Sep 2016 00:21:10 +0200 as excerpted: > > > Am Sun, 21 Aug 2016 02:19:33 + (UTC) > > schrieb Duncan <1i5t5.dun...@cox.net>: > > > >> Chris Murphy posted on Sat, 20 Aug 2016 18:36:21 -0600 as > >> excerpted: > [...] > >> > >> Depends on the distro. On gentoo, you set it up the way you want > >> of course, but the recommendation has always been /boot, and now > >> the ESP, not mounted by default. > >> > >> But that would be /expected/ on gentoo, since being able to > >> configure it the way you want is the whole /point/ of running > >> gentoo in the first place. Sort of like arch, only much more so. > > > > systemd systems (I'm booting Gentoo with systemd) should auto-mount > > ESP to /boot on access, and auto-unmount after a short timeout. So > > the solution to this problem is already wired into systemd if you > > use (a) proper GPT setup (with correct GUIDs) and (b) do not > > mention /boot in fstab. > > No "automagic" on-access mounting here. The kernel options aren't > turned on for it, neither do I want them on (the systemd ebuild > checks for and recommends but does not force them, and systemd > functions fine without them except it doesn't automount on access, > which is pretty much the point). What gets mounted on boot is what I > have in fstab (plus a few things like cgroups that systemd handles > entirely on its own). After that, nothing else is mounted unless I > mount it, or unmounted, unless I umount it or systemd does it as part > of shutdown, etc. https://www.freedesktop.org/software/systemd/man/systemd-gpt-auto-generator.html FD> Mount and automount units for the EFI System Partition (ESP), FD> mounting it to /boot, are generated on EFI systems where the boot FD> loader communicates the used ESP to the operating system. Since FD> this generator creates an automount unit, the mount will only be FD> activated on-demand, when accessed. On systems where /boot is an FD> explicitly configured mount (for example, listed in fstab(5)) or FD> where the /boot mount point is non-empty, no mount units are FD> generated. So there are a few points when this does not happen. It does not blindly auto-mount ESP. > However, good point about systemd, since it should more or less > standardize handling across mainline distros over the longer term, > except of course where admins/distros specifically configure it > otherwise, as I've done here. I also explicitly put it in fstab - so the generator doesn't work for me. -- Regards, Kai Replies to list-only preferred. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Mixing partitioned and non-partitioned discs in a RAID?
On Mon, Sep 12, 2016 at 4:21 PM, Kai Krakow wrote: > Am Sun, 21 Aug 2016 02:19:33 + (UTC) > schrieb Duncan <1i5t5.dun...@cox.net>: > >> Chris Murphy posted on Sat, 20 Aug 2016 18:36:21 -0600 as excerpted: >> >> > FAT leaves a lot to be desired but it's pretty universally >> > supported and almost trivial to repair *if* the volume is >> > repairable in the first place. The much bigger issue with ESP on >> > Linux is this neurotic tendency of distros to persistently mount >> > shit that does not need to be mounted. Like the ESP, and even the >> > dedicated boot volume. They only need to be mounted when being >> > updated and then should be umounted. And worse the convention is to >> > do nested mount with /boot and then /boot/efi for the ESP so it's >> > twice as bad a practice. By virtue of mounting the ESP the dirty >> > bit is set, so any crash means it must be fsck'd and if that >> > doesn't work, it's game over for that volume. Fragile setup. >> >> Depends on the distro. On gentoo, you set it up the way you want of >> course, but the recommendation has always been /boot, and now the >> ESP, not mounted by default. >> >> But that would be /expected/ on gentoo, since being able to configure >> it the way you want is the whole /point/ of running gentoo in the >> first place. Sort of like arch, only much more so. > > systemd systems (I'm booting Gentoo with systemd) should auto-mount ESP > to /boot on access, and auto-unmount after a short timeout. So the > solution to this problem is already wired into systemd if you use (a) > proper GPT setup (with correct GUIDs) and (b) do not mention /boot in > fstab. What distro uses this layout by default? It's certainly incompatible with multiboot where either Windows or macOS exist first. It also necessitates an excessively large ESP to store all possible kernels and initrds for every OS, making it all the harder to keep sync'd in any case where even a simple raid1 setup is desired. The ESP is a file system that doesn't support permissions, acls, or security labels either. So I'm not really a fan of this, or even nested /boot/efi either for that matter. They should be mounted in /run only on demand and only by the process that has permission to modify each. -- Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Mixing partitioned and non-partitioned discs in a RAID?
Kai Krakow posted on Tue, 13 Sep 2016 00:21:10 +0200 as excerpted: > Am Sun, 21 Aug 2016 02:19:33 + (UTC) > schrieb Duncan <1i5t5.dun...@cox.net>: > >> Chris Murphy posted on Sat, 20 Aug 2016 18:36:21 -0600 as excerpted: >> >> > FAT leaves a lot to be desired but it's pretty universally supported >> > and almost trivial to repair *if* the volume is repairable in the >> > first place. The much bigger issue with ESP on Linux is this neurotic >> > tendency of distros to persistently mount shit that does not need to >> > be mounted. Like the ESP, and even the dedicated boot volume. They >> > only need to be mounted when being updated and then should be >> > umounted. And worse the convention is to do nested mount with /boot >> > and then /boot/efi for the ESP so it's twice as bad a practice. By >> > virtue of mounting the ESP the dirty bit is set, so any crash means >> > it must be fsck'd and if that doesn't work, it's game over for that >> > volume. Fragile setup. >> >> Depends on the distro. On gentoo, you set it up the way you want of >> course, but the recommendation has always been /boot, and now the ESP, >> not mounted by default. >> >> But that would be /expected/ on gentoo, since being able to configure >> it the way you want is the whole /point/ of running gentoo in the first >> place. Sort of like arch, only much more so. > > systemd systems (I'm booting Gentoo with systemd) should auto-mount ESP > to /boot on access, and auto-unmount after a short timeout. So the > solution to this problem is already wired into systemd if you use (a) > proper GPT setup (with correct GUIDs) and (b) do not mention /boot in > fstab. No "automagic" on-access mounting here. The kernel options aren't turned on for it, neither do I want them on (the systemd ebuild checks for and recommends but does not force them, and systemd functions fine without them except it doesn't automount on access, which is pretty much the point). What gets mounted on boot is what I have in fstab (plus a few things like cgroups that systemd handles entirely on its own). After that, nothing else is mounted unless I mount it, or unmounted, unless I umount it or systemd does it as part of shutdown, etc. However, good point about systemd, since it should more or less standardize handling across mainline distros over the longer term, except of course where admins/distros specifically configure it otherwise, as I've done here. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Mixing partitioned and non-partitioned discs in a RAID?
Am Sun, 21 Aug 2016 02:19:33 + (UTC) schrieb Duncan <1i5t5.dun...@cox.net>: > Chris Murphy posted on Sat, 20 Aug 2016 18:36:21 -0600 as excerpted: > > > FAT leaves a lot to be desired but it's pretty universally > > supported and almost trivial to repair *if* the volume is > > repairable in the first place. The much bigger issue with ESP on > > Linux is this neurotic tendency of distros to persistently mount > > shit that does not need to be mounted. Like the ESP, and even the > > dedicated boot volume. They only need to be mounted when being > > updated and then should be umounted. And worse the convention is to > > do nested mount with /boot and then /boot/efi for the ESP so it's > > twice as bad a practice. By virtue of mounting the ESP the dirty > > bit is set, so any crash means it must be fsck'd and if that > > doesn't work, it's game over for that volume. Fragile setup. > > Depends on the distro. On gentoo, you set it up the way you want of > course, but the recommendation has always been /boot, and now the > ESP, not mounted by default. > > But that would be /expected/ on gentoo, since being able to configure > it the way you want is the whole /point/ of running gentoo in the > first place. Sort of like arch, only much more so. systemd systems (I'm booting Gentoo with systemd) should auto-mount ESP to /boot on access, and auto-unmount after a short timeout. So the solution to this problem is already wired into systemd if you use (a) proper GPT setup (with correct GUIDs) and (b) do not mention /boot in fstab. -- Regards, Kai Replies to list-only preferred. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Mixing partitioned and non-partitioned discs in a RAID?
Thanks for the responses all. I'm on Gentoo as well for what that matters. I think what i'm going to do (if i don't abandon EFI entirely) is just to get a small (16gb), $25 m.2 since my board supports that, and use that as my boot drive. Then mkfs other all discs to be partitionless and away i go. Not sure what i'll use the extra 15g on that m.2 for. I learned a lot from your long response Chris. Thank you. On Sat, Aug 20, 2016 at 8:19 PM, Duncan <1i5t5.dun...@cox.net> wrote: > Chris Murphy posted on Sat, 20 Aug 2016 18:36:21 -0600 as excerpted: > >> FAT leaves a lot to be desired but it's pretty universally supported and >> almost trivial to repair *if* the volume is repairable in the first >> place. The much bigger issue with ESP on Linux is this neurotic tendency >> of distros to persistently mount shit that does not need to be mounted. >> Like the ESP, and even the dedicated boot volume. They only need to be >> mounted when being updated and then should be umounted. And worse the >> convention is to do nested mount with /boot and then /boot/efi for the >> ESP so it's twice as bad a practice. By virtue of mounting the ESP the >> dirty bit is set, so any crash means it must be fsck'd and if that >> doesn't work, it's game over for that volume. Fragile setup. > > Depends on the distro. On gentoo, you set it up the way you want of > course, but the recommendation has always been /boot, and now the ESP, > not mounted by default. > > But that would be /expected/ on gentoo, since being able to configure it > the way you want is the whole /point/ of running gentoo in the first > place. Sort of like arch, only much more so. > > Meanwhile, I'm kinda partial to booting in legacy mode in grub2, with > grub2 installed to the legacy-BIOS partion on multiple gpt-partitioned > devices, which each one having its own /boot as well, setup so the > firmware boot device selector can choose which device and thus which > grub2 installation on that device, I boot from. That gives me as many > backup grub2 installations on the legacy BIOS partitions, pointing at as > many backup /boots on the same device by default, as I care to setup. Of > course what's extra nice about that is that you can switch to the /boot > on another device direct from the grub2 prompt, emergency mode if it > can't load the /boot off that device, if you like. > > -- > Duncan - List replies preferred. No HTML msgs. > "Every nonfree program has a lord, a master -- > and if you use the program, he is your master." Richard Stallman > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Bearcat M. Şándor Feline Soul Systems LLC Voice: 872.CAT.SOUL (872.228.7685) Fax: 406.235.7070 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Mixing partitioned and non-partitioned discs in a RAID?
Chris Murphy posted on Sat, 20 Aug 2016 18:36:21 -0600 as excerpted: > FAT leaves a lot to be desired but it's pretty universally supported and > almost trivial to repair *if* the volume is repairable in the first > place. The much bigger issue with ESP on Linux is this neurotic tendency > of distros to persistently mount shit that does not need to be mounted. > Like the ESP, and even the dedicated boot volume. They only need to be > mounted when being updated and then should be umounted. And worse the > convention is to do nested mount with /boot and then /boot/efi for the > ESP so it's twice as bad a practice. By virtue of mounting the ESP the > dirty bit is set, so any crash means it must be fsck'd and if that > doesn't work, it's game over for that volume. Fragile setup. Depends on the distro. On gentoo, you set it up the way you want of course, but the recommendation has always been /boot, and now the ESP, not mounted by default. But that would be /expected/ on gentoo, since being able to configure it the way you want is the whole /point/ of running gentoo in the first place. Sort of like arch, only much more so. Meanwhile, I'm kinda partial to booting in legacy mode in grub2, with grub2 installed to the legacy-BIOS partion on multiple gpt-partitioned devices, which each one having its own /boot as well, setup so the firmware boot device selector can choose which device and thus which grub2 installation on that device, I boot from. That gives me as many backup grub2 installations on the legacy BIOS partitions, pointing at as many backup /boots on the same device by default, as I care to setup. Of course what's extra nice about that is that you can switch to the /boot on another device direct from the grub2 prompt, emergency mode if it can't load the /boot off that device, if you like. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Mixing partitioned and non-partitioned discs in a RAID?
On Sat, Aug 20, 2016 at 1:38 PM, Alberto Bursi wrote: > > > On 08/20/2016 05:21 PM, Chris Murphy wrote: >> >> On Fri, Aug 19, 2016 at 10:00 PM, Bearcat Şándor >> wrote: >>> >>> I have a btrfs non-raid file system that i'd like to convert to >>> raid10. This single device has my efi boot partion on it, so it's >>> partitioned into sda1 and sda2. I have 3 other discs (sdc-sde) that >>> i'd like to make partition-less systems and then add them to the first >>> disc (sda) using the steps in the conversion section of the wiki >>> >>> (https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices#Conversion). >>> >>> My concern in that i'll be mixing a partitioned and partition-less >>> system. Will that work or will i hose myself? >> >> That will work, but you'll likely be in a hosed situation later should >> you ever lose a drive so I don't recommend this layout. >> >> a. You have the ESP on only one drive, so it's still a single point of >> failure. If that drive dies you can't boot. There are various ways >> around that but they all pretty much suck right now from an end user >> perspective. Basically redundant boot on UEFI with Linux right now is >> pretty much shit. You have to be a bootloader expert to set it up, >> maintain it with software updates, and fix it should things go bad. > > The usual solution to this is keeping the ESP on a separate (small) flash > drive. It's a good idea. It'd be nice if installers offered to put the ESP on non-boot USB (so it's not overwriting the installer media). > I still vehemently hate the fact that they couldn't provide like 2 MB of > space on an onboard NOR flash chip to store a bootloader. Seriously, rEFInd > with filesystem drivers to boot off btrfs and ext and whatever needs like > 1.5 MB. Needs to be bigger. The secure boot stuff takes up more room than this, about 6-8MiB depending on the distro. And then more room needed for dual booting let alone triple, which of course no manufacturer really cares about. > But no, they need to make a stupid system that is hardcoded to a disk, and > to a crappy filesystem. Ahh well the onboard flash could just appear as a small SATA or sdcard or USB mass storage devices, whatever's easiest to implement. FAT leaves a lot to be desired but it's pretty universally supported and almost trivial to repair *if* the volume is repairable in the first place. The much bigger issue with ESP on Linux is this neurotic tendency of distros to persistently mount shit that does not need to be mounted. Like the ESP, and even the dedicated boot volume. They only need to be mounted when being updated and then should be umounted. And worse the convention is to do nested mount with /boot and then /boot/efi for the ESP so it's twice as bad a practice. By virtue of mounting the ESP the dirty bit is set, so any crash means it must be fsck'd and if that doesn't work, it's game over for that volume. Fragile setup. -- Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Mixing partitioned and non-partitioned discs in a RAID?
On Sat, Aug 20, 2016 at 9:21 AM, Chris Murphy wrote: >If you > deleted this udev rule, you run the risk of some boots sometimes > having a slightly delayed drive becoming available and the volume > wrongly being mounted degraded when it's not necessary. Yuck. Revision time! If you were to delete this udev rule, it's possible to end up with an unnecessary degraded mount, simply due to a drive spinning up slower or otherwise having delayed readiness. Add on: Silently always mounting degraded in order to anticipate needing unattended degraded mounts will almost certainly lead to data loss. Btrfs just isn't ready for this yet. It has no concept of device faulty state still although patches for this are available for testing (not merged). So yeah, buyer beware. If you don't care about uptime and unattended operation, just self-healing during normal operation, then it's OK but you have to remain vigilant, and aware of Btrfs limitations, including it's lazy repair of formerly missing devices that reappear. It only fixes passively on reads. To completely fix it, you have to do a scrub, and you must do it before another drive goes missing or you can lose the whole file system. Btrfs has no partial resync like what's offered with mdadm arrays with a write-intent bitmap. -- Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Mixing partitioned and non-partitioned discs in a RAID?
On Fri, Aug 19, 2016 at 10:00 PM, Bearcat Şándor wrote: > I have a btrfs non-raid file system that i'd like to convert to > raid10. This single device has my efi boot partion on it, so it's > partitioned into sda1 and sda2. I have 3 other discs (sdc-sde) that > i'd like to make partition-less systems and then add them to the first > disc (sda) using the steps in the conversion section of the wiki > (https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices#Conversion). > > My concern in that i'll be mixing a partitioned and partition-less > system. Will that work or will i hose myself? That will work, but you'll likely be in a hosed situation later should you ever lose a drive so I don't recommend this layout. a. You have the ESP on only one drive, so it's still a single point of failure. If that drive dies you can't boot. There are various ways around that but they all pretty much suck right now from an end user perspective. Basically redundant boot on UEFI with Linux right now is pretty much shit. You have to be a bootloader expert to set it up, maintain it with software updates, and fix it should things go bad. b. Current udev and btrfs device ready code means degraded boot isn't possible, because any missing device means the volume isn't ready, so systemd won't even try to mount it (even with -o degraded). If you deleted this udev rule, you run the risk of some boots sometimes having a slightly delayed drive becoming available and the volume wrongly being mounted degraded when it's not necessary. And since Btrfs does not automatically fix up delayed devices later, you run the risk of data loss later on; or worse, corruption of the entire file system is possible if more than one device becomes stale at the same time. Between a. and b. it requires so much attention that it's just not worth it at all. If you're really doing this to avoid downtime and the ability to reboot degraded, the only two fairly easy and reliable ways to do this on UEFI is firmware or hardware RAID. Firmware RAID permits assembly of the RAID in firmware, so even the ESP can be mirrored, and then there's a handoff during boot to the md driver during normal operation and is managed by mdadm. -- Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Mixing partitioned and non-partitioned discs in a RAID?
Bearcat Şándor posted on Fri, 19 Aug 2016 22:00:17 -0600 as excerpted: > I have a btrfs non-raid file system that i'd like to convert to raid10. > This single device has my efi boot partion on it, so it's partitioned > into sda1 and sda2. I have 3 other discs (sdc-sde) that i'd like to make > partition-less systems and then add them to the first disc (sda) using > the steps in the conversion section of the wiki > (https://btrfs.wiki.kernel.org/index.php/ Using_Btrfs_with_Multiple_Devices#Conversion). > > My concern in that i'll be mixing a partitioned and partition-less > system. Will that work or will i hose myself? It should work fine as far as btrfs is concerned. Since you mentioned raid10, however, it's worth noting a difference between btrfs raid10 and conventional raid10 that takes some people by surprise, tho I don't know whether it'll be a factor for you or not. Btrfs raid is per-chunk, not per-device, and different chunks can be assigned differently. What that tends to mean in practice is that while with conventional raid10 you can lose a second device without loss of data as long as it's from the other raid1 underneath the raid0, with btrfs raid10, losing a second device is likely to lose data, because with the per-chunk assignment, it's relatively likely that the two bad devices will contain both copies of the data for at least some chunks. So with btrfs raid10, don't expect to be able to lose more than one device without data-loss, no matter /which/ devices they are. One way around that, that still retains the checksum error correction features of btrfs raid1/10, is to put btrfs raid1 on top of a pair of md/ dmraid0s. That's normally considered a raid01 instead of a raid10 and is discouraged as it makes rebuild more difficult, but btrfs' checksummed data error detection and with raid1/10, correction from the remaining good copy, makes it worth doing anyway. And that way you can for instance put one of the underlying raid0s in one data cabinet and the other in another, and lose the connection to everything in the one, without loss of data as raid0 on the other leg of the btrfs raid1 will still be operating. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Mixing partitioned and non-partitioned discs in a RAID?
20.08.2016 07:00, Bearcat Şándor пишет: > I have a btrfs non-raid file system that i'd like to convert to > raid10. This single device has my efi boot partion on it, so it's > partitioned into sda1 and sda2. I have 3 other discs (sdc-sde) that > i'd like to make partition-less systems and then add them to the first > disc (sda) using the steps in the conversion section of the wiki > (https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices#Conversion). > > My concern in that i'll be mixing a partitioned and partition-less > system. Will that work or will i hose myself? > Assuming all disks are of equal size, you will lose space on non-partitioned disks which is at least equal to your EFI boot partition. So it does not offer any advantages; in general the more uniform configuration, the better. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
