Re: FEATURE Req: integrate badblocks check into fsck.reiser*
On Thu, 07 Sep 2006 12:35:06 -0500, David Masover wrote: all snip. All I was getting at originally was to make fsck.reiser4 behave similarly to other fsck/mkfs programs so that it _could_, when requested, check a disk/partition for bad blocks. Despite the improved capabilities of modern drives, sh*t happens. And I, for one, would like the option of knowing before formatting a new partition if it's OK to do so. Sure, I can run badblocks myself, but this is counter to what I do with other fsck programs. A wrapper script would work fine, and I suppose, some sort of pipe output. However, I'm just looking at this from a userland perspective, not that of a sysadmin. Obviously, a sysadmin would not put any disk into production that has not been carefully tested. If this is superfluous, that's OK. I can easily work around it, but it should be noted somewhere in a README that reiserfs/4 does not do bad block checking during format and only a quick format is done. -- Peter + Do not reply to this email, it is a spam trap and not monitored. I can be reached via this list, or via jabber: pete4abw at jabber.org ICQ: 73676357
Re: FEATURE Req: integrate badblocks check into fsck.reiser*
David Masover wrote: Hans Reiser wrote: Ric Wheeler wrote: Having mkfs ignore bad writes would seem to encourage users to create a new file system on a disk that is known to be bad most likely not going to function well. If a user ever has a golden opportunity to toss a drive in the trash, it is when they notice mkfs fails ;-) This option to mkfs sounds like an invitation to disaster. Yes, you are right, the option should be to run badblocks and then fail if it finds any. Unless it creates significantly more work for us, there should be an option to run badblocks, and if it finds any, it should prompt the user (with BIG FAT CAPSLOCK WARNINGS) whether they want to format anyway. Formatting anyway should work, and we should be able to have blocks marked bad. I think that you are missing the way modern drives behave. To give a typical example, on a 300 GB drive, we typically have 2000 or more extra sectors that are used for automatic remapping. Theses sectors are consumed only when the drive retries a failed write multiple times. If you fail a write, that means (barring even worse failure modes like a whole head going south) that all of these sectors have been consumed. If they have not been consumed, the user will never see the remapping (it happens as part of a normal write, just takes longer than usual). We really, really do not need a list of bad blocks to avoid during writing a new file system image. I think that the more interesting case is handling bad blocks during recovery. It is not clear to me that fsck needs a list, but we have worked with Hans and Vladamir to get support for doing a reverse mapping (given a list of bad blocks, show the user what files, etc got hit). It would also be nice to be able to change this later -- to pass in a list of badblocks to, say, fsck (which I think is the original request). This is especially nice for recovery, if you don't have the luxury of copying a whole disk image to another drive before running fsck. That's not to say that we should automatically detect and relocate bad blocks during normal operation (while the FS is mounted), but deliberately removing functionality to protect you from yourself isn't the Linux Way. Linux has a long history of kernel config options that say things like YOU WILL LOSE DATA. You have been warned. The linux way is not to review ideas and see if they merit us coding them up. LKML is nothing if not a long list of good/bad/weird ideas that get proposed, reviewed and often as not dumped in the dust bin of history ;-) Ideas are good, discussion is great, but we should not invest in features that are known to produce a definite failure. I spend a lot of time monitoring and helping debug file system/disk failures on a huge installed base of reiser3 file systems (running on sata and pata drives). As part of this, I spend a lot of time talking to disk vendors about how and when to pull the plug on bad drives. ric
Re: FEATURE Req: integrate badblocks check into fsck.reiser*
Ric Wheeler wrote: David Masover wrote: Hans Reiser wrote: Ric Wheeler wrote: Having mkfs ignore bad writes would seem to encourage users to create a new file system on a disk that is known to be bad most likely not going to function well. If a user ever has a golden opportunity to toss a drive in the trash, it is when they notice mkfs fails ;-) This option to mkfs sounds like an invitation to disaster. Yes, you are right, the option should be to run badblocks and then fail if it finds any. Unless it creates significantly more work for us, there should be an option to run badblocks, and if it finds any, it should prompt the user (with BIG FAT CAPSLOCK WARNINGS) whether they want to format anyway. Formatting anyway should work, and we should be able to have blocks marked bad. I think that you are missing the way modern drives behave. To give a typical example, on a 300 GB drive, we typically have 2000 or more extra sectors that are used for automatic remapping. Theses sectors are consumed only when the drive retries a failed write multiple times. Oh, I'm not disputing that mkfs should discourage users from using broken drives. Presumably, smart admins wouldn't see this often, because they'd be monitoring SMART. We really, really do not need a list of bad blocks to avoid during writing a new file system image. Why do you presume to make this decision for users? I don't think we need CONFIG_LEGACY_PTYS -- they're insecure, and almost never needed. But we should still leave them in. The burden is on us to show that it's taking real work to implement and maintain. I think that the more interesting case is handling bad blocks during recovery. It is not clear to me that fsck needs a list, but we have worked with Hans and Vladamir to get support for doing a reverse mapping (given a list of bad blocks, show the user what files, etc got hit). Yes, it seems like fsck would be much better off that way. In this case, of course, I'd prefer to avoid hitting that problem -- use RAID, make regular backups, toss out the disk and restore. Being able to repair bad blocks would tend to encourage a user to keep using a bad disk, but I don't want to force my opinion on everyone when there's a reasonable way for all of us to be happy.
Re: FEATURE Req: integrate badblocks check into fsck.reiser*
David Masover wrote: Ric Wheeler wrote: I think that you are missing the way modern drives behave. To give a typical example, on a 300 GB drive, we typically have 2000 or more extra sectors that are used for automatic remapping. Theses sectors are consumed only when the drive retries a failed write multiple times. Oh, I'm not disputing that mkfs should discourage users from using broken drives. Presumably, smart admins wouldn't see this often, because they'd be monitoring SMART. We really, really do not need a list of bad blocks to avoid during writing a new file system image. Why do you presume to make this decision for users? It's not a decision that I want to make for users, it is a decision that Hans and his team need to make about how best to spend their limited resources. Allowing users to put down reiser3/4 file systems on crap drives takes effort on their part and will result in an increased work load. It will also give more users a bad experience with the file system, since users rarely have the in depth knowledge required to make this kind of choice. I don't think we need CONFIG_LEGACY_PTYS -- they're insecure, and almost never needed. But we should still leave them in. The burden is on us to show that it's taking real work to implement and maintain. This is a request for a new feature to allow users to do something, by design, that is extremely likely to lose all of their data. Not to extend support for an existing (braindead) legacy. I think that the more interesting case is handling bad blocks during recovery. It is not clear to me that fsck needs a list, but we have worked with Hans and Vladamir to get support for doing a reverse mapping (given a list of bad blocks, show the user what files, etc got hit). Yes, it seems like fsck would be much better off that way. In this case, of course, I'd prefer to avoid hitting that problem -- use RAID, make regular backups, toss out the disk and restore. Being able to repair bad blocks would tend to encourage a user to keep using a bad disk, but I don't want to force my opinion on everyone when there's a reasonable way for all of us to be happy. Here we mostly agree. The need for enhanced tools is not to encourage people to keep using bad drives, rather to allow them to fsck remount a drive for data recovery. If you cannot mount fsck fails to repair enough to give you at least a readable file system, then you are in real trouble ;-) Also, unlike failing writes, disk read errors are quite often ephemeral and will be self correcting on the next write (you might get an error from dust, etc that gets swept clean on the next write).
Re: FEATURE Req: integrate badblocks check into fsck.reiser*
Ric Wheeler wrote: David Masover wrote: Why do you presume to make this decision for users? It's not a decision that I want to make for users, it is a decision that Hans and his team need to make about how best to spend their limited resources. Agreed. It's not important if it takes more than, say, an hour. It will also give more users a bad experience with the file system, since users rarely have the in depth knowledge required to make this kind of choice. While it's true that most users just click through dialog boxes, I imagine this would be sufficient: ===WARNING===WARNING===WARNING=== - THIS DISK IS BAD! If you continue with the format, we will not help you when you lose data. When, not if. You are strongly encouraged to THROW THIS DISK OUT! - ARE YOU ABSOLUTELY SURE YOU WANT TO CONTINUE? (yes/no): And require an actual yes or no answer. No y/n. Now, compare that to a filesystem which doesn't allow badblocks in mkfs at all. While it's rare, I suspect that would be a worse experience if you actually had a real need for it. If you've got a huge 300 gig drive with some bad blocks, you can always throw some stuff on it anyway, for backup, or stuff you don't care about, even knowing it'll fail soon. Again, probably not a high priority item at all, but it certainly won't make the user experience worse. Any user who says yes to the above warning does not get to complain about their experience. Here we mostly agree. The need for enhanced tools is not to encourage people to keep using bad drives, rather to allow them to fsck remount a drive for data recovery. If you cannot mount fsck fails to repair enough to give you at least a readable file system, then you are in real trouble ;-) Also, unlike failing writes, disk read errors are quite often ephemeral and will be self correcting on the next write (you might get an error from dust, etc that gets swept clean on the next write). Either one, I would personally feel quite a lot safer grabbing a disk image and doing the fsck once the image was on known good media. One thing that would be even better here, though, if you don't want to spend the time for a huge backup: A way to tell badblocks to only scan space which is actually being used, and then enough free space to make sure relocations work. If you're mounting readonly, you shouldn't care about marking every single bad sector in free space. I guess this would require a lot more intelligence from fsck, though -- it would have to be able to constantly check for bad blocks, as opposed to just running badblocks once and grabbing a list to avoid.
Re: FEATURE Req: integrate badblocks check into fsck.reiser*
Hans Reiser wrote: Ric Wheeler wrote: Having mkfs ignore bad writes would seem to encourage users to create a new file system on a disk that is known to be bad most likely not going to function well. If a user ever has a golden opportunity to toss a drive in the trash, it is when they notice mkfs fails ;-) This option to mkfs sounds like an invitation to disaster. Yes, you are right, the option should be to run badblocks and then fail if it finds any. Unless it creates significantly more work for us, there should be an option to run badblocks, and if it finds any, it should prompt the user (with BIG FAT CAPSLOCK WARNINGS) whether they want to format anyway. Formatting anyway should work, and we should be able to have blocks marked bad. It would also be nice to be able to change this later -- to pass in a list of badblocks to, say, fsck (which I think is the original request). This is especially nice for recovery, if you don't have the luxury of copying a whole disk image to another drive before running fsck. That's not to say that we should automatically detect and relocate bad blocks during normal operation (while the FS is mounted), but deliberately removing functionality to protect you from yourself isn't the Linux Way. Linux has a long history of kernel config options that say things like YOU WILL LOSE DATA. You have been warned.
Re: FEATURE Req: integrate badblocks check into fsck.reiser*
Ric Wheeler wrote: David Masover wrote: Peter wrote: On Fri, 01 Sep 2006 17:27:20 -0500, David Masover wrote: snip... both mkfs.reiserfs and fsck.reiserfs have -B option to accept list of bad blocks. We thought that should be enough. It really should. Why bother with a patch? Just write a wrapper script that runs badblocks and passes in the list to mkfs. It was just a thought from userland. My perspective was that a user, not a hard-boiled geek, might get lulled into a false sense of security but may not have the wherewithal to write a wrapper. If nothing else, when the final doc is written (did I say final?:)), it should include a notice about not running badblocks. Well, let's see... Most hard drives come more thoroughly tested at the factory than anything badblocks would do. Also, it seems redundant to have every single mkfs have to implement a badblocks flag.. I'd suggest a universal wrapper, then, or a modification to the mkfs frontend, so that this works the same way across all filesystems. Something like mkfs -B -t reiser4 I don't think that modern drives that fail writes are worth using for a brand new file system. While failing reads is quite common and can be caused by temporal issues (dirt on the platter, a bad write, etc), failed writes are almost always a sign that you have a serious issue. Almost all modern drives remap each failed write to a bad sector automatically. This action only fails if you have exhausted this remapping area (or have some really nasty issue like a bad cable, bad write head, etc). Having mkfs ignore bad writes would seem to encourage users to create a new file system on a disk that is known to be bad most likely not going to function well. If a user ever has a golden opportunity to toss a drive in the trash, it is when they notice mkfs fails ;-) This option to mkfs sounds like an invitation to disaster. Yes, you are right, the option should be to run badblocks and then fail if it finds any. The other tools (debugreiserfs, reiserfsck, etc) do need to be able to handle bad blocks as well as possible since they are often needed during a salvage operation. in order to recover data (which might need to be migrated to a new disk). It is not clear to me that passing a list of bad blocks helps them as much as a robust general purpose error recovery support.
Re: FEATURE Req: integrate badblocks check into fsck.reiser*
David Masover wrote: Peter wrote: On Fri, 01 Sep 2006 17:27:20 -0500, David Masover wrote: snip... both mkfs.reiserfs and fsck.reiserfs have -B option to accept list of bad blocks. We thought that should be enough. It really should. Why bother with a patch? Just write a wrapper script that runs badblocks and passes in the list to mkfs. It was just a thought from userland. My perspective was that a user, not a hard-boiled geek, might get lulled into a false sense of security but may not have the wherewithal to write a wrapper. If nothing else, when the final doc is written (did I say final?:)), it should include a notice about not running badblocks. Well, let's see... Most hard drives come more thoroughly tested at the factory than anything badblocks would do. Also, it seems redundant to have every single mkfs have to implement a badblocks flag.. I'd suggest a universal wrapper, then, or a modification to the mkfs frontend, so that this works the same way across all filesystems. Something like mkfs -B -t reiser4 I don't think that modern drives that fail writes are worth using for a brand new file system. While failing reads is quite common and can be caused by temporal issues (dirt on the platter, a bad write, etc), failed writes are almost always a sign that you have a serious issue. Almost all modern drives remap each failed write to a bad sector automatically. This action only fails if you have exhausted this remapping area (or have some really nasty issue like a bad cable, bad write head, etc). Having mkfs ignore bad writes would seem to encourage users to create a new file system on a disk that is known to be bad most likely not going to function well. If a user ever has a golden opportunity to toss a drive in the trash, it is when they notice mkfs fails ;-) This option to mkfs sounds like an invitation to disaster. The other tools (debugreiserfs, reiserfsck, etc) do need to be able to handle bad blocks as well as possible since they are often needed during a salvage operation. in order to recover data (which might need to be migrated to a new disk). It is not clear to me that passing a list of bad blocks helps them as much as a robust general purpose error recovery support.
Re: FEATURE Req: integrate badblocks check into fsck.reiser*
Hello On Friday 01 September 2006 22:23, Peter wrote: Perhaps this has been mentioned before. If so, sorry. IMHO, it would be useful to integrate a call to badblocks in the fsck/mkfs.reiser* programs so that more thorough disk checking can be done at format time. Sort of like the option e2fsck -c. If this is added, the output could be fed immediately to the reiser format program and badblocks spared prior to filesystem use. JM$0.02 both mkfs.reiserfs and fsck.reiserfs have -B option to accept list of bad blocks. We thought that should be enough.
Re: FEATURE Req: integrate badblocks check into fsck.reiser*
Vladimir V. Saveliev wrote: Hello On Friday 01 September 2006 22:23, Peter wrote: Perhaps this has been mentioned before. If so, sorry. IMHO, it would be useful to integrate a call to badblocks in the fsck/mkfs.reiser* programs so that more thorough disk checking can be done at format time. Sort of like the option e2fsck -c. If this is added, the output could be fed immediately to the reiser format program and badblocks spared prior to filesystem use. JM$0.02 both mkfs.reiserfs and fsck.reiserfs have -B option to accept list of bad blocks. We thought that should be enough. I'll take a patch;-)
Re: FEATURE Req: integrate badblocks check into fsck.reiser*
Vladimir V. Saveliev wrote: Hello On Friday 01 September 2006 22:23, Peter wrote: Perhaps this has been mentioned before. If so, sorry. IMHO, it would be useful to integrate a call to badblocks in the fsck/mkfs.reiser* programs so that more thorough disk checking can be done at format time. Sort of like the option e2fsck -c. If this is added, the output could be fed immediately to the reiser format program and badblocks spared prior to filesystem use. JM$0.02 both mkfs.reiserfs and fsck.reiserfs have -B option to accept list of bad blocks. We thought that should be enough. It really should. Why bother with a patch? Just write a wrapper script that runs badblocks and passes in the list to mkfs.
Re: FEATURE Req: integrate badblocks check into fsck.reiser*
David Masover wrote: Vladimir V. Saveliev wrote: Hello On Friday 01 September 2006 22:23, Peter wrote: Perhaps this has been mentioned before. If so, sorry. IMHO, it would be useful to integrate a call to badblocks in the fsck/mkfs.reiser* programs so that more thorough disk checking can be done at format time. Sort of like the option e2fsck -c. If this is added, the output could be fed immediately to the reiser format program and badblocks spared prior to filesystem use. JM$0.02 both mkfs.reiserfs and fsck.reiserfs have -B option to accept list of bad blocks. We thought that should be enough. It really should. Why bother with a patch? Just write a wrapper script that runs badblocks and passes in the list to mkfs. For average sysadmins, it would be nice to just add a flag and it happens. I will take a patch, but I won't write it.;-)
Re: FEATURE Req: integrate badblocks check into fsck.reiser*
On Fri, 01 Sep 2006 17:27:20 -0500, David Masover wrote: snip... both mkfs.reiserfs and fsck.reiserfs have -B option to accept list of bad blocks. We thought that should be enough. It really should. Why bother with a patch? Just write a wrapper script that runs badblocks and passes in the list to mkfs. It was just a thought from userland. My perspective was that a user, not a hard-boiled geek, might get lulled into a false sense of security but may not have the wherewithal to write a wrapper. If nothing else, when the final doc is written (did I say final?:)), it should include a notice about not running badblocks. -- Peter + Do not reply to this email, it is a spam trap and not monitored. I can be reached via this list, or via jabber: pete4abw at jabber.org ICQ: 73676357
Re: FEATURE Req: integrate badblocks check into fsck.reiser*
Peter wrote: On Fri, 01 Sep 2006 17:27:20 -0500, David Masover wrote: snip... both mkfs.reiserfs and fsck.reiserfs have -B option to accept list of bad blocks. We thought that should be enough. It really should. Why bother with a patch? Just write a wrapper script that runs badblocks and passes in the list to mkfs. It was just a thought from userland. My perspective was that a user, not a hard-boiled geek, might get lulled into a false sense of security but may not have the wherewithal to write a wrapper. If nothing else, when the final doc is written (did I say final?:)), it should include a notice about not running badblocks. Well, let's see... Most hard drives come more thoroughly tested at the factory than anything badblocks would do. Also, it seems redundant to have every single mkfs have to implement a badblocks flag.. I'd suggest a universal wrapper, then, or a modification to the mkfs frontend, so that this works the same way across all filesystems. Something like mkfs -B -t reiser4
Re: FEATURE Req: integrate badblocks check into fsck.reiser*
David Masover wrote: Peter wrote: On Fri, 01 Sep 2006 17:27:20 -0500, David Masover wrote: snip... both mkfs.reiserfs and fsck.reiserfs have -B option to accept list of bad blocks. We thought that should be enough. It really should. Why bother with a patch? Just write a wrapper script that runs badblocks and passes in the list to mkfs. It was just a thought from userland. My perspective was that a user, not a hard-boiled geek, might get lulled into a false sense of security but may not have the wherewithal to write a wrapper. If nothing else, when the final doc is written (did I say final?:)), it should include a notice about not running badblocks. Well, let's see... Most hard drives come more thoroughly tested at the factory than anything badblocks would do. They leave more tested, they arrive ;-) and you assume it is a new drive. Also, it seems redundant to have every single mkfs have to implement a badblocks flag.. I'd suggest a universal wrapper, then, or a modification to the mkfs frontend, so that this works the same way across all filesystems. Something like mkfs -B -t reiser4
