Re: Using mount -o bind vs mount -o subvol=vol
On Mon, May 05, 2014 at 02:12:30AM +, Duncan wrote: Marc MERLIN posted on Sat, 03 May 2014 17:47:32 -0700 as excerpted: Just as an FYI, like (likely) most subscribers, I do prefer Cc on replies. Without that, I'm much less likely to see your message timely, or at all if I'm behind on Email. TL;DR: Put simply, with certain sometimes major exceptions, IMO subvolumes are /mostly/ a solution looking for a problem. In the /general/ case, I don't see the point and personally STRONGLY prefer multiple independent partitions for their much stronger data safety and mounting/backup flexibility. That's why I use independent partitions, here. I'm a partitions guy, but now that I have subvolumes which can be snapshotted/backed up independently, I'm much happier with a single shared pool. Look at a btrfs pool like an LVM pool, except more flexible. To each their own I guess. 1) Multiple subvolumes on a common filesystem share the filesystem tree- and super-structure. If something happens to that filesystem, you had all your data eggs in that one basket and the bottom just dropped out of it! If you can't recover, kiss **ALL** those data eggs goodbye! Backups :) (and having your booting filesystem on a different pool from you data pool). 3) Filesystem size and time to complete whole-filesystem operations such as balance, scrub and check are directly related; the larger the filesystem, the longer such operations take. There are reports here of balances taking days on multi-terabyte filesystems, and double-digit hours isn't unusual at all. True, but if I have a 10TB array, I'm not going to cut it into 10 1TB arrays just for that. Now ask yourself, how likely are you to routinely run a scrub or balance as preventive maintenance if you know it's going to take the entire day to finish? Here, the times are literally so trivial can and do run a full filesystem rebalance to time it and make this point and maintenance such as scrub or balance simply ceases to be an issue. It runs nightly from cron on my laptop. 1TB filesystem on SSD, no sweat. 4) Many distros are using btrfs subvolumes on a single btrfs storage pool the way they formerly used LVM volume groups, as a common storage pool allowing them the flexibility to (re)allocate space to whatever lvm volume or btrfs subvolume needs it. Yep. OTOH, for users and distros with a pretty good idea of what their allocations are going to look like, generally due to the experience they've gained over the years, that extra flexibility isn't a big benefit You and me yes, most other people no. And to be honest, I've been doing this for 20 years, and my guesses are not always right 10 years later on a machine that's still running :) (of which I have several) 6) Subvolumes be used to control snapshotting since snapshots stop at subvolume boundaries. In the presence of point #5 storage pools, and given the reality of btrfs NOCOW attribute behavior when mixed with snapshots, subvolumes become an important tool for limiting snapshot coverage area, in particular, for demarcing areas that should NOT be snapshotted when the filesystem or parent subvolume is snapshotted, due for instance to the horrible interaction between large heavy-internal- rewrite files and COW, which means they should be set NOCOW, coupled with the horrible interaction between NOCOW on such files and snapshotting. Yep. Similarly, subvolumes and their boundaries can be used to set borders for frequency or timing of snapshotting, say snapshotting the general root/system tree before updates, while snapshotting /home hourly. Yep. Point #6 is, I'd argue, one of the few legitimate use-cases for subvolumes as opposed to independent filesystems, and it actually loses relevancy if #4 is subsumed to point #1 and #3, already. However, given the reality of popular distro btrfs layouts and usage, #4 is in practice overruling all the others in many distro-default btrfs deployments today, and #6 then becomes relevant. subvolumes are also used as units of backup for btrfs send. So my vote would be, for example (modified slightly for posting from my own mounts): mount /dev/sda5 / mount /dev/sda4 /var/log mount /dev/sda6 /home On my laptop: /dev/mapper/cryptroot on / type btrfs (rw,noatime,compress=lzo,ssd,discard,space_cache) /dev/mapper/cryptroot on /usr type btrfs (rw,noatime,compress=lzo,ssd,discard,space_cache) /dev/mapper/cryptroot on /var type btrfs (rw,noatime,compress=lzo,ssd,discard,space_cache) /dev/mapper/cryptroot on /home type btrfs (rw,noatime,compress=lzo,ssd,discard,space_cache) /dev/mapper/cryptroot on /tmp type btrfs (rw,noexec,noatime,compress=lzo,ssd,discard,space_cache) /dev/mapper/cryptroot on /var/local/nobckd2 type btrfs (rw,noatime,compress=lzo,ssd,discard,space_cache) /dev/mapper/disk2 on /var/local/space type btrfs (rw,noatime,compress=lzo,discard,space_cache) /dev/mapper/cryptroot
Re: Using mount -o bind vs mount -o subvol=vol
Marc MERLIN posted on Wed, 07 May 2014 03:55:51 -0700 as excerpted: subvolumes are also used as units of backup for btrfs send. Hmm, yes. Thanks. I don't use send/receive here so forgot about that. So my vote would be, for example (modified slightly for posting from my own mounts): mount /dev/sda5 / mount /dev/sda4 /var/log mount /dev/sda6 /home On my laptop: [snip] FWIW, those were examples. I actually have more. But to each their own :) Indeed. -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Using mount -o bind vs mount -o subvol=vol
Brendan Hide posted on Mon, 05 May 2014 08:55:55 +0200 as excerpted: You are 100% right, though. The scale is very small. By negligible, the penalty is at most a few CPU cycles. When compared to the wait time on a spindle, it really doesn't matter much. The analogy I've used before is that of taking a trip (which the data effectively is, between the device and the CPU). We've booked a 10-day cruise and are now debating what we plan on taking to and from the boarding dock. Will taking the local bus with a couple of transfers, or a taxi that will take us there in one trip but there's road construction and thus a detour, or a helicopter to fly directly, get us back from the cruise faster? Obviously, taking the helicopter (at least for the return leg) will get us back a bit faster, but we're talking perhaps a couple hours difference at the end of a 10 day cruise! =:^) -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Using mount -o bind vs mount -o subvol=vol
Marc MERLIN posted on Sat, 03 May 2014 17:47:32 -0700 as excerpted: Is there any functional difference between mount -o subvol=usr /dev/sda1 /usr and mount /dev/sda1 /mnt/btrfs_pool mount -o bind /mnt/btrfs_pool/usr /usr ? Brendan answered the primary aspect of this well so I won't deal with that. However, I've some additional (somewhat controversial) opinion/comments on the topic of subvolumes in general. TL;DR: Put simply, with certain sometimes major exceptions, IMO subvolumes are /mostly/ a solution looking for a problem. In the /general/ case, I don't see the point and personally STRONGLY prefer multiple independent partitions for their much stronger data safety and mounting/backup flexibility. That's why I use independent partitions, here. Relevant points to consider: Subvolume negatives, independent partition positives: 1) Multiple subvolumes on a common filesystem share the filesystem tree- and super-structure. If something happens to that filesystem, you had all your data eggs in that one basket and the bottom just dropped out of it! If you can't recover, kiss **ALL** those data eggs goodbye! That's the important one; the one that would prevent me sleeping well if that's the solution I had chosen to use. But there's a number of others, more practical in the binary it's not an unrecoverable failure case. 2) Presently, btrfs is rather limited in the opposing mount options it can apply to subvolumes on the same overall filesystem. Mounting just one subvolume nodatacow, for instance, without mounting all mounted subvolumes of the filesystem nodatacow isn't yet possible, tho the filesystem design allows for it and the feature is roadmapped to appear sometime in the future. This means that at present, the subvolumes solution severely limits your mount options flexibility, altho that problem should go away to a large degree at some rather handwavily defined point in the future. 3) Filesystem size and time to complete whole-filesystem operations such as balance, scrub and check are directly related; the larger the filesystem, the longer such operations take. There are reports here of balances taking days on multi-terabyte filesystems, and double-digit hours isn't unusual at all. Of course SSDs are generally smaller and (much) faster, but still, a filesystem the size of a quarter or a half-gig SSD could easily take an hour or more to balance or scrub, and that can still be a big deal. Contrast that with the /trivial/ balance/scrub times I see on my partitioned btrfs-on-ssd setup here, some of them under a minute, even the big btrfs of 24 GiB (gentoo packages/sources/ccache filesystem) taking under three minutes (just under 7 second per GiB). At those times the return is fast enough I normally run the thing in foreground and wait for it to return in real-time; times trivial enough I can actually do a full filesystem rebalance in ordered to time it to make this point on a post! =:^) Of course the other aspect of that is that I can for instance fsck my dedicated multimedia filesystem without it interfering with running X and my ordinary work on /home. If it's all the same filesystem and I have to fsck from the initramfs or a rescue disk... Now ask yourself, how likely are you to routinely run a scrub or balance as preventive maintenance if you know it's going to take the entire day to finish? Here, the times are literally so trivial can and do run a full filesystem rebalance to time it and make this point and maintenance such as scrub or balance simply ceases to be an issue. I actually learned this point back on mdraid, before I switched to btrfs. When I first setup mdraid, I had only three raids, primary/ working, secondary/first-backup, and the raid0 for stuff like package cache that I could simply redownload if necessary. But if a device dropped (as it occasionally did after a resume from hibernate, due to hardware taking too long to wake up and the kernel giving up on it), the rebuild would take HOURS! Later on, after a few layout changes, I had many more raids and kept some of them (like the one containing my distro package cache) deactivated unless I actually needed to use them (if I was actually doing an update). Since a good portion of the many more but smaller raids were offline most of the time, if a device dropped, I had far fewer and smaller raids to rebuild, and was typically back up and running in under a half hour. Filesystem maintenance time DOES make a difference! Subvolume positives, independent partition negatives: 4) Many distros are using btrfs subvolumes on a single btrfs storage pool the way they formerly used LVM volume groups, as a common storage pool allowing them the flexibility to (re)allocate space to whatever lvm volume or btrfs subvolume needs it. This is a killer feature from the viewpoint of many distros and users as the flexibility means no more hassle with guessing incorrectly
Re: Using mount -o bind vs mount -o subvol=vol
On 05/05/14 06:36, Roman Mamedov wrote: On Mon, 05 May 2014 06:13:30 +0200 Brendan Hide bren...@swiftspirit.co.za wrote: 1) There will be a *very* small performance penalty (negligible, really) Oh, really, it's slower to mount the device directly? Not that I really care, but that's unexpected. Um ... the penalty is if you're mounting indirectly. ;) I feel that's on about the same scale as giving your files shorter filenames, so that they open faster. Or have you looked at the actual kernel code with regard to how it's handled, or maybe even have any benchmarks, other than a general thought of it's indirect, so it probably must be slower? My apologies - not everyone here is a native English-speaker. You are 100% right, though. The scale is very small. By negligible, the penalty is at most a few CPU cycles. When compared to the wait time on a spindle, it really doesn't matter much. -- __ Brendan Hide http://swiftspirit.co.za/ http://www.webafrica.co.za/?AFF1E97 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Using mount -o bind vs mount -o subvol=vol
On 05/03/2014 08:47 PM, Marc MERLIN wrote: Is there any functional difference between mount -o subvol=usr /dev/sda1 /usr and mount /dev/sda1 /mnt/btrfs_pool mount -o bind /mnt/btrfs_pool/usr /usr The internal implementation of mount -o subvol is a bind mount. The only real difference is having /mnt/btrfs_pool hanging around. -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Using mount -o bind vs mount -o subvol=vol
On 2014/05/04 02:47 AM, Marc MERLIN wrote: Is there any functional difference between mount -o subvol=usr /dev/sda1 /usr and mount /dev/sda1 /mnt/btrfs_pool mount -o bind /mnt/btrfs_pool/usr /usr ? Thanks, Marc There are two issues with this. 1) There will be a *very* small performance penalty (negligible, really) 2) Old snapshots and other supposedly-hidden subvolumes will be accessible under /mnt/btrfs_pool. This is a minor security concern (which of course may not concern you, depending on your use-case). There are a few similar minor security concerns - the recently-highlighted issue with old snapshots is the potential that old vulnerable binaries within a snapshot are still accessible and/or executable. -- __ Brendan Hide http://swiftspirit.co.za/ http://www.webafrica.co.za/?AFF1E97 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Using mount -o bind vs mount -o subvol=vol
On 2014/05/05 02:56 AM, Marc MERLIN wrote: On Sun, May 04, 2014 at 09:07:55AM +0200, Brendan Hide wrote: On 2014/05/04 02:47 AM, Marc MERLIN wrote: Is there any functional difference between mount -o subvol=usr /dev/sda1 /usr and mount /dev/sda1 /mnt/btrfs_pool mount -o bind /mnt/btrfs_pool/usr /usr ? Thanks, Marc There are two issues with this. 1) There will be a *very* small performance penalty (negligible, really) Oh, really, it's slower to mount the device directly? Not that I really care, but that's unexpected. Um ... the penalty is if you're mounting indirectly. ;) 2) Old snapshots and other supposedly-hidden subvolumes will be accessible under /mnt/btrfs_pool. This is a minor security concern (which of course may not concern you, depending on your use-case). There are a few similar minor security concerns - the recently-highlighted issue with old snapshots is the potential that old vulnerable binaries within a snapshot are still accessible and/or executable. That's a fair point. I can of course make that mountpoint 0700, but it's a valid concern in some cases (not for me though). So thanks for confirming my understanding, it sounds like both are valid and if you're already mounting the main pool like I am, that's the easiest way. Thanks, Marc All good. :) -- __ Brendan Hide http://swiftspirit.co.za/ http://www.webafrica.co.za/?AFF1E97 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Using mount -o bind vs mount -o subvol=vol
On Mon, 05 May 2014 06:13:30 +0200 Brendan Hide bren...@swiftspirit.co.za wrote: 1) There will be a *very* small performance penalty (negligible, really) Oh, really, it's slower to mount the device directly? Not that I really care, but that's unexpected. Um ... the penalty is if you're mounting indirectly. ;) I feel that's on about the same scale as giving your files shorter filenames, so that they open faster. Or have you looked at the actual kernel code with regard to how it's handled, or maybe even have any benchmarks, other than a general thought of it's indirect, so it probably must be slower? -- With respect, Roman signature.asc Description: PGP signature
Re: Using mount -o bind vs mount -o subvol=vol
On Mon, May 05, 2014 at 06:13:30AM +0200, Brendan Hide wrote: Oh, really, it's slower to mount the device directly? Not that I really care, but that's unexpected. Um ... the penalty is if you're mounting indirectly. ;) I'd be willing to believe that more then :) (but indeed, if slowdown there is, it must be pretty irrelevant in the big picture. Cheers, Marc -- A mouse is a device used to point at the xterm you want to type in - A.S.R. Microsoft is to operating systems what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | PGP 1024R/763BE901 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Using mount -o bind vs mount -o subvol=vol
Is there any functional difference between mount -o subvol=usr /dev/sda1 /usr and mount /dev/sda1 /mnt/btrfs_pool mount -o bind /mnt/btrfs_pool/usr /usr ? Thanks, Marc -- A mouse is a device used to point at the xterm you want to type in - A.S.R. Microsoft is to operating systems what McDonalds is to gourmet cooking Home page: http://marc.merlins.org/ | PGP 1024R/763BE901 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
