On Sat, Jan 19, 2008 at 09:59:50PM +0000, Rodolfo Medina <[EMAIL PROTECTED]> was heard to say: > on Fri, Jan 18, 2008 at 12:16:28PM -0500, Douglas A. Tutty wrote: > Daniel Burrows <[EMAIL PROTECTED]> writes: > > > So does umount. (I see you said that below, but I wanted to underline > > it again; it looks like Rudolfo overlooked it) After "umount" > > completes, it's safe to remove the stick.
I apologize for misspelling your name -- I checked twice and still did it! > I didn't overlook this, but: > > 1) in my experience, some time ago I used to extract the pendrive after > `umount' completes and the device often got corrupted. When I asked this > list for help to recover it, someone suggested to apply the `sync' option > to the mount option: since then I put that option in my fstab, until this > thread advised not to do so; I've been using Linux for 10 years, and I've always, always, always heard that "umount" had a side effect of flushing dirty buffers to disk and syncing the filesystem. But I did a little research before I replied, and I came up with some surprising information. First, neither the manpage of umount(8) nor the manpage of mount/umount/umount2(2) mention flushing buffers; neither does the libc documentation on these functions. Linux system-level documentation is usually fairly thorough, so I'd expect an important detail like this to be mentioned if it were true. On the other hand, this is such a fundamental thing, that I could believe the manpage author forgot to mention it, the same way that the unlink(2) manpage doesn't mention that future calls to open(2) on the unlinked name will fail. So I hit Google. As I said, I'd picked up a long time ago the notion that umount(8) was how to make it safe to yank removable media, and this belief is widely spread in the community. (e.g., Google for "umount removable corruption" and you'll find this advice being purveyed in many locations) So I figured that a little Googling would quickly turn up people squawking about their expectations being violated if it were true that umount(8) was not flushing. I didn't find much; most of the , but I did find a very strong claim by some guy named "Christian Neumair" that umount(8) does *not* flush write buffers, but that eject(1) does and should be used on USB media: http://blogs.gnome.org/cneumair/2006/02/11/ioctl-fsync-how-to-flush-block-device-buffers/#comment-94 An Ubuntu user named "Toby Smithe" claims here that "umount" should be avoided in favor of "eject", also with no citation to supporting evidence: http://ubuntuforums.org/showpost.php?p=1737197&postcount=5 However, virtually all the hits claim (also without any support) that umount *does* flush buffers. There are some intriguing references to a BLKFLSBUF ioctl that's supposed to be related to this (perhaps is flushes dirty buffers for a block device?), but no documentation on what it actually does that I can find. The .h says "flush buffer cache", but there are a couple different things I could imagine that meaning. I also found a comment on an Ubuntu bug by "John Dong" that appears to say that sync(1) should be run after unmounting. However, I didn't read all the context -- it looks to me like perhaps he's talking about a situation where the GUI does an asynchronous umount in the background, so users don't know whether the umount is actually complete. https://bugs.launchpad.net/ubuntu/+source/kdebase/+bug/61946/comments/27 I also found a Sun manpage saying that Solaris needs you to run eject(1) on USB devices. Obviously this doesn't apply here until Debian/Solaris gets off the ground. Since it doesn't exist right now according to http://www.debian.org/ports, that will take a while. :-) So, the situation as I see it is that there's no authoritative information one way or another -- everyone's just repeating hearsay of one sort or another, as often happens on the Internet. Christian Neumair might have actual information, but he didn't say where he got his information from, so I don't know whether to believe him or not. I've also skimmed over the source for eject(1), the program being touted as somehow safer than umount() -- I haven't had time to read it thoroughly, nor do I intend to do so (although it's fairly short at 1449 lines of code), but I did do some quick searches in the source code. It makes no references to "flush" or "sync" anywhere in comments or source code: "grep -ir flush" and "grep -ir sync" come up empty. If it is doing something magical, though, I'd expect it to be via an ioctl. The only ioctls I see in there are: (1) a bunch of ioctls that are obviously CD-ROM manipulation (e.g., CDROMEJECT) and don't apply here. (2) some ioctls for sg devices (SG_*). These could be relevant. It appears that eject(1) sends the commands ALLOW_MEDIUM_REMOVAL, START_STOP, and START_STOP again. I don't know SCSI well, and there's no documentation for this, so it could be doing just about anything, and the first one sounds like it *could* have to do with flushing buffers. (3) The BLKRRPART ioctl, with a comment saying that it forces the kernel to re-read the partition table when the next disc is inserted. (4) The FDEJECT ioctl, which appears to eject a floppy disk. (5) An ioctl to eject a tape from a tape drive. So, I can't rule out eject(1) doing something clever for sg devices (and IIRC usb devices go through the sg layer). There's also this thread, where a user claims that eject's function is to turn power to the USB device off. Google hits I got for "ALLOW_MEDIUM_REMOVAL usb" suggest that this is indeed the function of ALLOW_MEDIUM_REMOVAL when sent to a usb-storage device. The user also states that he has no idea whether this is actually safer or not; I wonder (and this is pure idle speculation) if maybe the issue is that some USB devices don't handle being hot-unplugged well. http://www.free-bees.co.uk/articles/pclinuxos93a/comments/6/ This would also explain how umount() can work and still leave the stick broken: if somehow being unplugged without being turned off confuses the electronics on the stick itself and causes it to corrupt its data, then flushing block caches won't help at all (compare this to shutting down a hard disk that does write-buffering in volatile memory before its buffers have fully flushed). NOTE however that I have absolutely zero evidence that this actually happens; I'm just generating a hypothesis to explain why it might actually matter. On the other hand, apparently another side effect of ALLOW_MEDIUM_REMOVAL is to make some USB devices stop saying "do not disconnect". This seems like the sort of thing that would cause users to believe that their data isn't safe even though it is. With all that said: I would expect to see some loud arguments about umount not flushing write caches if it didn't do that, and I would expect to see warnings in the umount(8) man page; if you have to run eject in order to avoid actual data corruption, I would similarly expect to see loud warnings in visible places about this. "umount to remove" is the behavior I've always expected as a user of Linux going back to when I used floppies, and I believe that other Linux users expect the same thing. Moving to personal anecdotes, I've never experienced corruption as a result of yanking a USB stick as soon as I umounted it, and I am quite certain that some sort of flushing takes place, because of two facts that you can verify for yourself: (1) umount(8) takes longer the more data you've written to the device. (2) if you run sync(1) before umount(8), the device is unmounted immediately, with no blocking. My theory about flushing even makes a testable prediction: if you run umount(8) first and have no other removable media, then sync(1) should complete almost immediately. I don't think I've ever done this, unlike the two points above, but you're welcome to try it yourself and let us know if it works. :-) So I'll probably continue using umount(8) to unmount my USB sticks; I have no evidence that it isn't enough, and plenty of support for the notion that it really is enough (as noted above, I haven't lost data doing this). But if you're nervous, you certainly can run eject(1) instead, or run sync(1) before yanking them. It won't hurt anything at all to do so and I can't prove that it's not required. :-) > 2) I measured the time, and it seems to me that the `sync' command takes a > little longer than `umount' to stop. I don't know for sure, but this could be because "sync" flushes the write buffers on all devices, not just the one you're pulling out. Daniel -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
