Re: attacking btrfs filesystems via UUID collisions? (was: Subvolume UUID, data corruption?)
Christoph Anton Mitterer posted on Wed, 09 Dec 2015 06:07:38 +0100 as excerpted: > Well as I've said, getting that in via USB may be only one way. > We're already so far that GNOME automount devices when plugged... Ugh. ... And many know that's the sort of thing that made MS so much of a security headache, and want no part of it! FWIW, of course gentoo allows far more configurability in this regard than many distros, but no automount here, and while I don't do gnome because I like my system configurable and they'd just as soon it be their way or the highway (echoes of proprietaryware attitude there if you ask me, but I'm very glad gnome's available for them to work on as otherwise they'd be troubling kde and etc to go the same way), I do have a much more limited than usual kde installed, without stuff like the device notifier plasmoid or underlying infrastructure like udisks, as the only things I want mounted are the things I've either configured to be mounted via fstab, or the thing's I've manually mounted. (FWIW, the semantic- desktop crap is opted out at build-time too, so it's not even there to turn off at runtime, the best most distros allow for those not interested in that stuff. It meant dumping a few apps and some missing features in others, but I don't have indexing taking gigs of space and major IO bandwidth at the most inconvenient times (any time!) for nothing I'm going to make use of, either!) > You said it's basically not fixable in btrfs: > It's absolutely clear that I'm no btrfs expert (or even developer), but > my poor man approach which I think I've written before doesn't seem so > impossible, does it? > 1) Don't simply "activate" btrfs devices that are found but rather: > 2) Check if there are other devices of the same fs UUID + device ID, > or more generally said: check if there are any collisions > 3) If there are, and some of them are already active, > continue to use them, don't activate the newly appeared ones > 4) If there are, and none of them are already active, refuse to > activate *any* of them unless the user manually instructs to do so > via device= like options. The underlying issue pretty much isn't fixable, but as Qu has suggested on that subthread, there's ameliorations that can be done, basically in line with your suggestions above, and you've indicated that you'd consider that fixed, tho neither he nor I consider it "fixed", only hidden to some extent. Anyway, he's a dev and actively involved now, while I'm not a dev, so he can take it from there. =:^) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: attacking btrfs filesystems via UUID collisions? (was: Subvolume UUID, data corruption?)
On Sun, 2015-12-06 at 04:06 +, Duncan wrote: > There's actually a number of USB-based hardware and software vulns > out > there, from the under $10 common-component-capacitor-based charge- > and-zap > (charges off the 5V USB line, zaps the port with several hundred > volts > reverse-polarity, if the machine survives the first pulse and > continues > supplying 5V power, repeat...), to the ones that act like USB-based > input > devices and "type" in whatever commands, to simple USB-boot to a > forensic > distro and let you inspect attached hardware (which is where the > encrypted > storage comes in, they've got everything that's not encrypted), > to the plain old fashioned boot-sector viruses that quickly jump to > everything else on the system that's not boot-sector protected and/or > secure-boot locked, to... Well this is all well known - at least to security folks ;) - but to be quite honest: Not an excuse for allowing even more attack surface, in this case via the filesystem. One will *always* find a weaker element in the security chain, and could always argue with that not to fixe one's own issues. "Well, there's no need to fix that possible collision-data-leakage- issue in btrfs[0]! Why? Well an attacker could still simply abduct the bank manager, torture him for hours until he gives any secret with pleasure" ;-) > Which is why most people in the know say if you have unsupervised > physical > access, you effectively own the machine and everything on it, at > least > that's not encrypted. Sorry, I wouldn't say so. Ultimately you're of course right, which is why my fully-dm-crypted notebook is never left alone when it runs (cold boot or USB firmware attacks)... but in practise things are a bit different I think. Take the ATM example. Or take real world life in big computing centres. Fact is, many people have usually access, from the actual main personell, over electricians to the cleaning personnel. Whacking a device or attacking it via USB firmware tricks, is of course possible for them, but it's much more likely to be noted (making noise, taking time and so on),... so there is no need to give another attack surface by this. > If you haven't been keeping up, you really have some reading to > do. If > you're plugging in untrusted USB devices, seriously, a thumb drive > with a > few duplicated btrfs UUIDs is the least of your worries! Well as I've said, getting that in via USB may be only one way. We're already so far that GNOME automount devices when plugged... who says the the next step isn't that this happens remotely in some form, e.g. btrfs-image on dropbox, automounted by nautilus. Okay, that may be a bit constructed, but it should demonstrate that there could be plenty of ways for that to happen, which we don't even think of (and usually these are the worst in security). You said it's basically not fixable in btrfs: It's absolutely clear that I'm no btrfs expert (or even developer), but my poor man approach which I think I've written before doesn't seem so impossible, does it? 1) Don't simply "activate" btrfs devices that are found but rather: 2) Check if there are other devices of the same fs UUID + device ID, or more generally said: check if there are any collisions 3) If there are, and some of them are already active, continue to use them, don't activate the newly appeared ones 4) If there are, and none of them are already active, refuse to activate *any* of them unless the user manually instructs to do so via device= like options. > BTW, this is documented (in someone simpler "do not do XX" form) on > the > wiki, gotchas page. > > https://btrfs.wiki.kernel.org/index.php/Gotchas#Block-level_copies_of > _devices I know, but it doesn't really tell all possibly consequences, and again, it's unlikely that the end-user (even if possibly heavily affected by it) will stumble over that. Cheer, Chris. [0] Assuming there is actually one, I haven't really verified that and base it solely one what people told that basically arbitrary corruptions may happen on both devices. smime.p7s Description: S/MIME cryptographic signature
Re: attacking btrfs filesystems via UUID collisions? (was: Subvolume UUID, data corruption?)
On Sat, 2015-12-05 at 13:19 +, Duncan wrote: > The problem with btrfs is that because (unlike traditional > filesystems) > it's multi-device, it needs some way to identify what devices belong > to a > particular filesystem. Sure, but that applies to lvm, or MD as well... and I wouldn't know of any random corruption issues there. > And UUID is, by definition and expansion, Universally Unique ID. Nitpicking doesn't help here,... reality is they're not,.. either by people doing stuff like dd, other forms of clones, LVM, etc. ... or as I've described maliciously. > Btrfs > simply depends on it being what it says on the the tin, universally > unique, to ID the components of the filesystem and assemble them > correctly. Admittedly, I'm not an expert to the internals of btrfs, but it seems other multi-device containers can handle UUID duplicates fine, or at least so that you don't get any data corruption (or leaks). This is a showstopper - maybe not under lab conditions but surely under real world scenarios. I'm actually quite surprised that no-one else didn't complain about that before, given how long btrfs exists. > Besides dd, etc, LVM snapshots are another case where this goes > screwy. > If the UUID isn't UUID, do a btrfs device scan (which udev normally > does > by default these days) so the duplicate UUID is detected, and btrfs > *WILL* eventually start trying to write to all the "newly added" > devices > that scan found, identified by their Universally Unique IDs, aka > UUIDs. > It's not a matter of if, but when. Well.. as I said... quite scary, with respect to both, accidental and malicious cases of duplicate UUIDs. > And the UUID is embedded so deeply within the filesystem and its > operations, as an inextricable part of the metadata (thus avoiding > the > problem reiserfs had where a reiserfs stored in a loopback file on a > reiserfs, would screw up reiserfsck, on btrfs, the loopback file > would > have a different UUID and thus couldn't be mixed up), that changing > the > UUID is not the simple operation of changing a few bytes in the > superblock > that it is on other filesystems, which is why there's now a tool to > go > thru all those metadata entries and change it. I don't think that this design is per se bad and prevents the kernel to handle such situations gracefully. I would expect that in addition to the fs UUID, it needs a form of device ID... so why not simply ignoring any new device for which there already is a matching fs UUID and device ID, unless the respective tool (mount, btrfs, etc.) is explicitly told so via some device=/dev/sda,/dev/sdb option. If that means that less things work out of the box (in the sense of "auto-assembly") well than this is simply necessary. data security and consistency is definitely much more important than any fancy auto-magic. > So an aware btrfs admin simply takes pains to avoid triggering a > btrfs > device scan at the wrong time, and to immediately hide their LVM > snapshots, immediately unplug their directly dd-ed devices, etc, and > thus > doesn't have to deal with the filesystem corruption that'd be a when > not > if, if they didn't take such precautions with their dupped UUIDs that > actually aren't as UUID as the name suggests... a) People shouldn't need to do days of study to be able to use btrfs securely. Of course it's more advanced and not everything can be simplified in a way so that users don't need to know anything (e.g. all the well-known effects of CoW)... but when the point is reached where security and data integrity is threatened, there's definitely a hard border that mustn't be crossed. b) Given how complex software is, I doubt that it's easily possible, even for the aware admin, to really prevent all situations that can lead to such situations. Not to talk about about any attack-scenarios. > And as your followup suggests in a security context, they consider > masking out their UUIDs before posting them, as well, tho most kernel > hackers generally consider unsupervised physical access to be game- > over, > security-wise. Do they? I rather thought many of them had a rather practical and real- world-situations-based POV. > (After all, in that case there's often little or nothing > preventing a reboot to that USB stick, if desired, or simply yanking > the > devices and duping them or plugging them in elsewhere, if the BIOS is > password protected, with the only thing standing in the way at that > point > being possible device encryption.) There's hardware which would, when it detects physicals intrusion (like yanking) lock up itself (securely clearing the memory, disconnecting itself from other nodes, which may be compromised as well, when the filesystem on the attacked node would go crazy. You have things like ATMs, which are physically usually quite well secured, but which do have rather easily accessible maintenance ports. All of us have seen such embedded devices rebooting
Re: attacking btrfs filesystems via UUID collisions? (was: Subvolume UUID, data corruption?)
On Sat, 2015-12-05 at 12:01 +, Hugo Mills wrote: > On Sat, Dec 05, 2015 at 04:28:24AM +0100, Christoph Anton Mitterer > wrote: > > On Fri, 2015-12-04 at 13:07 +, Hugo Mills wrote: > > > I don't think it'll cause problems. > > Is there any guaranteed behaviour when btrfs encounters two > > filesystems > > (i.e. not talking about the subvols now) with the same UUID? > > Nothing guaranteed, but the likelihood is that things will go > badly > wrong, in the sense of corrupt filesystems. Phew... well sorry, but I think that's really something that makes btrfs not productively usable until fixed. > Except that that's exactly the mechanism that btrfs uses to handle > multi-device filesystems, so you've just broken anything with more > than one device in the FS. Don't other containers (e.g. LVM) do something similar, and yet they don't fail badly in case e.g. multipl PVs with the same UUID appear, AFAIC. And shouldn't there be some kind of device UUID, which differs different parts of the same btrfs (with the same fs UUID) but on different devices?! > If you inspect the devid on each device as well, and refuse > duplicates of those, you've just broken any multipathing > configurations. Well, how many people are actually doing this? A minority. So then it would be simply necessary that multipathing doesn't work out of the box and one need to specifically tell the kernel to consider a device with the same btrfs UUID as not a clone but another path to the same device. In any cases, rare feature like multipathing cannot justify the possibility of data corruption. That situtation as it is now is IMHO completely unacceptable. > Even if you can handle that, if you have two copies of dev1, and > two copies of dev2, how do you guarantee that the "right" pair of > dev1 > and dev2 is selected? (e.g. if you have them as network devices, and > the device enumeration order is unstable on each boot). Not sure what you mean now: The multipathing case? Then, as I've said, such situations would simply require to manually set things up and explicitly tell the kernel that the devices foo and bar are to be used (despite their dup UUID). If you mean what happens when I have e.g. two clones of a 2-device btrfs, as in fsdev1 fsdev2 fsdev1_clone fsdev2_clone Then as I've said before... if one pair of them is already mounted (i.e. when the *_clone appear), than it's likely that these belong actually together and the kernel should continue to use them and ignore any other. If all appear before any is mounted, then either is should refuse to mount/use any of them, or it should require to manually specify which devices to be used (i.e. via /dev/sda or so). Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature
Re: attacking btrfs filesystems via UUID collisions? (was: Subvolume UUID, data corruption?)
Christoph Anton Mitterer posted on Sun, 06 Dec 2015 02:51:20 +0100 as excerpted: > You have things like ATMs, which are physically usually quite well > secured, but which do have rather easily accessible maintenance ports. > All of us have seen such embedded devices rebooting themselves, where > you see kernel messages. > That's the point where an attacker could easily get the btrfs UUID: > [0.00] Command line: BOOT_IMAGE=/vmlinuz-4.2.0-1-amd64 > root=UUID=bd1ea5a0-9bba-11e5-82fa-502690aa641f > > If you can attack such devices already by just having access to a USB > port... then holly sh**... There's actually a number of USB-based hardware and software vulns out there, from the under $10 common-component-capacitor-based charge-and-zap (charges off the 5V USB line, zaps the port with several hundred volts reverse-polarity, if the machine survives the first pulse and continues supplying 5V power, repeat...), to the ones that act like USB-based input devices and "type" in whatever commands, to simple USB-boot to a forensic distro and let you inspect attached hardware (which is where the encrypted storage comes in, they've got everything that's not encrypted), to the plain old fashioned boot-sector viruses that quickly jump to everything else on the system that's not boot-sector protected and/or secure-boot locked, to... Which is why most people in the know say if you have unsupervised physical access, you effectively own the machine and everything on it, at least that's not encrypted. There's a reason some places hot-glue the USB ports. If you're plugging anything untrusted into them... and that's a well known social engineering hack as well, simply drop a few thumb drives in the target parking lot and wait to see who picks them up and plugs them in, so they can call home... Pen-testers do it. NSA does it. It's said a form of that is how they bridged the air-gap to the Iranian centrifuges... If you haven't been keeping up, you really have some reading to do. If you're plugging in untrusted USB devices, seriously, a thumb drive with a few duplicated btrfs UUIDs is the least of your worries! >> The only real alternative if you don't like it is using a different >> filesystem. > As I've said, I don't have a problem with UUIDs... I just can't quite > believe that btrfs and the userland cannot be modified so that it > handles such cases gracefully. As I implied, UUIDs usage is so deeply embedded, fixing btrfs to not work that way is pretty much impossible. You'd be pretty much starting from scratch and using some of the same ideas; it wouldn't be btrfs any longer. > If not, than, to be quite honest, that would be really a major > showstopper for many usage areas. Consider the show stopped, then. > And I'm not talking about ATMs (or any other embedded devices where > people may have non-supervides access - e.g. TVs in a mall, > entertainment systems in planes) but also the normal desktops/laptops > where colleagues, fellow students, etc. may want to play some "prank". As I said, if you're plugging in or allowing to be plugged in untrusted USB devices, show's over, they're already playing pretty much any prank they want, including zapping the hardware. USB's now less trusted than a raw Internet hookup with all services exposed. The only controlling factor now is the physical presence limitation, and if you're plugging in devices you get for instance as "gifts just for trying us out" or whatever, that someone mails to you... worse than running MS and mindlessly running any exe someone sends you. BTW, this is documented (in someone simpler "do not do XX" form) on the wiki, gotchas page. https://btrfs.wiki.kernel.org/index.php/Gotchas#Block-level_copies_of_devices -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: attacking btrfs filesystems via UUID collisions? (was: Subvolume UUID, data corruption?)
Thinking a bit more I that, I came to the conclusion that it's actually security relevant that btrfs deals gracefully with filesystems having the same UUID: Getting to know someone else's filesystem's UUID may be more easily possible than one may think. It's usually not considered secret and for example included in debug reports (e.g. several Debian packages do this). The only thing an attacker then needs to do is somehow making another filesystem with the UUID available in his victims system. Simplest way is via a USB stick when he has local access. Thanks to some stupid desktop environments, chances aren't to bad that the system will even auto mount the stick. If btrfs doesn't handle this gracefully the attacker may damage or destroy the original filesystem, or if things get awkwardly corrupted (and data is written to the fake btrfs) even get data out of such a system (despite any screen locks or dm-crypt). Cheers Chris. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
