Re: online fsck
Hello On Wed, 2005-05-25 at 03:26, marco wrote: > E.Gryaznova wrote: > > > Hello. > > reiserfsck can check reiserfs filesystem mounted read-only. > > Running 'fsck.reiserfs --check -y ' will do the trick. The > equivalent for ext3 and other filesystems is > 'fsck -n ', the ext3 version detects that it is mounted rw, > and does a readonly check. > > What I wonder is if we can get fsck.reiser[fs,4] to honour option > processing in a manner more consistant with > the other fscks, so that adding reiserfs to a list of user choosable > filesystems is not going to cause a nightmare > of an overhead of filesystem detection, command option re-mapping and > other related headaches. > The main problem why reiserfsck does not process similar to other fscks is that reiserfs (as well as reiser4) moves data on filesystem modifications so that even files which were not modified recently change their locations. Online reiserXck should be probably implemented as kernel thread. Or be able to block filesystem updates, or I do not know what. So, in short, reiserXck for r/w mounted filesystem can not be implemened as plain user land tool,imho. Also imho, currently the problem is that it is not very clear how to do it. We would be happy to hear any thoughts on it. > > > > Thanks, > > Lena > > > > btinsley wrote: > > > >> Is there or was there a plan to support running reiserfsck on a > >> mounted v3 filesystem (just a check, not a fix or rebuild)? I seem to > >> remember this being mentioned here at some point in time, but I was > >> unable to find it in the mailing list archive. > >> > >> Thanks! > >> > >> > >> > >> > > > > > > > > > >
Re: online fsck
> I'd rather donate for a reiser4 online repacker. By the time > something's fsck'd, so to speak, I'd rather take it offline and possibly > pull in backups. But a repacker (even an offine one) and a resizer > (even an offline one) are two things that we even have in the Linux > ntfs-tools, and it's also something that people would have an immediate > use for. Pulling in backups is not always practical though. It's no fun restoring a 900GB filesystem from tape, especially when the data has to be transferred over the network :-) I would like to see the repacker though. I have certain filesystems that could really use it. Would it be practical to have an option to enable the repacker to include some level of consistency checking? It's been my experience that corruptions requiring a rebuild-tree are pretty obvious to reiserfsck ;-) > > Another killer feature would be to bring back metadata, or at least a > way to create special (plugin) files, and the ability to write certain > kinds of userland plugins, thus giving us FS-level support for things > like zipfiles. Yet another killer feature would be stable crypto and > compression, and the ability to enable it only for certain directories. I would go for this one as well. Our particular application stores data in an industry-standard format. Being able to index/search/crypt/etc. this at the FS level would let me get rid of the cost and overhead of Oracle :-)
Re: online fsck
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Yiannis Mavroukakis wrote: > Philippe Gramoullé wrote: > >> Hello, >> >> On Tue, 24 May 2005 22:02:20 -0500 >> btinsley <[EMAIL PROTECTED]> wrote: >> >> | What i'm looking for is a check on a reiserfs filesystem that is >> | mounted read-write. Many modern filesystems, especially those on NAS >> | devices, can run periodic background consistency checks on filesystems >> | with almost zero impact on performance. Some devices are reportedly >> | running these checks constantly, possibly correcting errors without >> | user intervention... even a notification to a sysadmin would be a good >> | feature. >> >> Last time i asked Hans about the possibility to have a --rebuild-tree >> (for reiserfs) while the fs is online and mounted rw, >> he told me to send 30.000 US Dollars in, and that it could eventually >> be done ;) >> >> Sure lots of people would find this a killer feature, but someone will >> definitely have to pay get such a feature implemented. >> >> Thanks, >> >> Philippe >> >> > Then Hans should setup a paypal account for donations. I'd gladly give > what I can spare, and I think so would a lot of people here. I'd rather donate for a reiser4 online repacker. By the time something's fsck'd, so to speak, I'd rather take it offline and possibly pull in backups. But a repacker (even an offine one) and a resizer (even an offline one) are two things that we even have in the Linux ntfs-tools, and it's also something that people would have an immediate use for. Another killer feature would be to bring back metadata, or at least a way to create special (plugin) files, and the ability to write certain kinds of userland plugins, thus giving us FS-level support for things like zipfiles. Yet another killer feature would be stable crypto and compression, and the ability to enable it only for certain directories. And finally, there's the Windows port. A full, stable Windows port, using the ReactOS libs, under a licence which requires blatant attribution, so that the user knows that they are using a third-party filesystem. This would certainly ease the migration to Linux, because Reiser4 beats NTFS, and Linux doesn't really have reliable NTFS support (captive is slow and crash-prone). Think of the first feature and the last one together. You shrink the NTFS partition, create a new Reiser4 partition, copy your Windows files over, nuke the NTFS partition, resize the r4 partition (backwards), and your Windows is on Reiser4. Next, you install Linux to a directory in that r4 partition. Now all your Windows files are accessible from Linux and vice versa, until you stop dual-booting, in which case you don't have to repartition or reinstall. These are all good features, and they are all something I would rather donate money towards than an online fsck. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQpekPHgHNmZLgCUhAQK5pxAAkmVS/NzA2xDMuiNH2LCBSaUpGl/nbMzP gV5XZgyBJOhmEEJR1EoOe+cftygooqwY+FuQZJ2HVqI12mm2BTpG9zxTwUmcqGw9 3wNBqHII1/VWthxTchChwvY+NppH9IKyvWOXSHvTOlce97WDsMu0wBBIxA+VbcfP cNop/MiDFs37fjWNqXBVpl8WO6vNg+RPHjG4j0n8ljziPDJUY/9A3vBfLgXYzUQN dXcw+xkPI4vN5W4HoNrnfJhwzHTyEd34sWpC6MdQO2uzxAmAlP8paruMpslozFP/ mqr24zMcOgWlisglyn5Zoc43098QqhvX0mJSoj4Z6C++bJWUn+FdCKxbUDfe3cCK jtRhDJUPRA2kutT5VrcOQbuYuDrOtCbfeXHPnnfxI2DNjbj+wz5pEyMflJvskoyn Arfb6z/zMDeblQFE2xYCE9GjbdaVzh0237G1RlukurvyDlitrejDSGEFCsoMv3Fn 9ljYTm5DXEkoBNS76SXmHR6/ZXSxor+Hyy6lhz0V762MWdZntmW9/7f8n4s60rcR Kq0Q9ujJdllNBYTBwyIDriRJbXdL9FP6qhdPf9fD7shFGKbAk2uHoO6yoJ6bLjfm jTUKWPeEgzRbZg8GSPnltJQnp5BZrPMNdpPcxNo/o68VprXppnKtZlTqXwbqREYr BJa9XhP0Tdk= =//7g -END PGP SIGNATURE-
Re: online fsck
On Fri, May 27, 2005 at 10:32:14AM +0100, Yiannis Mavroukakis wrote: >Then Hans should setup a paypal account for donations. I'd gladly give >what I can spare, and I think so would a lot of people here. http://namesys.com/support.html I've donated that way and it's worked every time. Maybe some feature-specific pools with stats on the web would be nice, and iirc no one really shot the idea down. After all, I first mentioned it probably a year ago or so :) -- mjt signature.asc Description: Digital signature
Re: online fsck
Philippe Gramoullé wrote: Hello, On Tue, 24 May 2005 22:02:20 -0500 btinsley <[EMAIL PROTECTED]> wrote: | What i'm looking for is a check on a reiserfs filesystem that is | mounted read-write. Many modern filesystems, especially those on NAS | devices, can run periodic background consistency checks on filesystems | with almost zero impact on performance. Some devices are reportedly | running these checks constantly, possibly correcting errors without | user intervention... even a notification to a sysadmin would be a good | feature. Last time i asked Hans about the possibility to have a --rebuild-tree (for reiserfs) while the fs is online and mounted rw, he told me to send 30.000 US Dollars in, and that it could eventually be done ;) Sure lots of people would find this a killer feature, but someone will definitely have to pay get such a feature implemented. Thanks, Philippe Then Hans should setup a paypal account for donations. I'd gladly give what I can spare, and I think so would a lot of people here. __ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email __
Re: online fsck
Hello, On Tue, 24 May 2005 22:02:20 -0500 btinsley <[EMAIL PROTECTED]> wrote: | What i'm looking for is a check on a reiserfs filesystem that is | mounted read-write. Many modern filesystems, especially those on NAS | devices, can run periodic background consistency checks on filesystems | with almost zero impact on performance. Some devices are reportedly | running these checks constantly, possibly correcting errors without | user intervention... even a notification to a sysadmin would be a good | feature. Last time i asked Hans about the possibility to have a --rebuild-tree (for reiserfs) while the fs is online and mounted rw, he told me to send 30.000 US Dollars in, and that it could eventually be done ;) Sure lots of people would find this a killer feature, but someone will definitely have to pay get such a feature implemented. Thanks, Philippe
Re: online fsck
btinsley wrote: What i'm looking for is a check on a reiserfs filesystem that is mounted read-write. Many modern filesystems, especially those on NAS devices, can run periodic background consistency checks on filesystems with almost zero impact on performance. Some devices are reportedly running these checks constantly, possibly correcting errors without user intervention... even a notification to a sysadmin would be a good feature. I, too, am looking for a solution to be able to do some sort of check _regardless_ of whether and how the filesystem is mounted. On 5/24/05, marco <[EMAIL PROTECTED]> wrote: Running 'fsck.reiserfs --check -y ' will do the trick. The equivalent for ext3 and other filesystems is 'fsck -n ', the ext3 version detects that it is mounted rw, and does a readonly check. What I wonder is if we can get fsck.reiser[fs,4] to honour option processing in a manner more consistant with the other fscks, so that adding reiserfs to a list of user choosable filesystems is not going to cause a nightmare of an overhead of filesystem detection, command option re-mapping and other related headaches.
Re: online fsck
What i'm looking for is a check on a reiserfs filesystem that is mounted read-write. Many modern filesystems, especially those on NAS devices, can run periodic background consistency checks on filesystems with almost zero impact on performance. Some devices are reportedly running these checks constantly, possibly correcting errors without user intervention... even a notification to a sysadmin would be a good feature. On 5/24/05, marco <[EMAIL PROTECTED]> wrote: > > > E.Gryaznova wrote: > > > Hello. > > reiserfsck can check reiserfs filesystem mounted read-only. > > Running 'fsck.reiserfs --check -y ' will do the trick. The > equivalent for ext3 and other filesystems is > 'fsck -n ', the ext3 version detects that it is mounted rw, > and does a readonly check. > > What I wonder is if we can get fsck.reiser[fs,4] to honour option > processing in a manner more consistant with > the other fscks, so that adding reiserfs to a list of user choosable > filesystems is not going to cause a nightmare > of an overhead of filesystem detection, command option re-mapping and > other related headaches. > > > > > Thanks, > > Lena > > > > btinsley wrote: > > > >> Is there or was there a plan to support running reiserfsck on a > >> mounted v3 filesystem (just a check, not a fix or rebuild)? I seem to > >> remember this being mentioned here at some point in time, but I was > >> unable to find it in the mailing list archive. > >> > >> Thanks! > >> > >> > >> > >> > > > > > > > > > > > > -- > Marco Grigull www.accessplus.com.au AccessPlus Internet Solutions > Ph +61 7 5451 1146 Fax +61 7 5451 1945 Email [EMAIL PROTECTED] > >
Re: online fsck
E.Gryaznova wrote: Hello. reiserfsck can check reiserfs filesystem mounted read-only. Running 'fsck.reiserfs --check -y ' will do the trick. The equivalent for ext3 and other filesystems is 'fsck -n ', the ext3 version detects that it is mounted rw, and does a readonly check. What I wonder is if we can get fsck.reiser[fs,4] to honour option processing in a manner more consistant with the other fscks, so that adding reiserfs to a list of user choosable filesystems is not going to cause a nightmare of an overhead of filesystem detection, command option re-mapping and other related headaches. Thanks, Lena btinsley wrote: Is there or was there a plan to support running reiserfsck on a mounted v3 filesystem (just a check, not a fix or rebuild)? I seem to remember this being mentioned here at some point in time, but I was unable to find it in the mailing list archive. Thanks! -- Marco Grigull www.accessplus.com.au AccessPlus Internet Solutions Ph +61 7 5451 1146 Fax +61 7 5451 1945 Email [EMAIL PROTECTED]
Re: online fsck
Hello. reiserfsck can check reiserfs filesystem mounted read-only. Thanks, Lena btinsley wrote: Is there or was there a plan to support running reiserfsck on a mounted v3 filesystem (just a check, not a fix or rebuild)? I seem to remember this being mentioned here at some point in time, but I was unable to find it in the mailing list archive. Thanks!
online fsck
Is there or was there a plan to support running reiserfsck on a mounted v3 filesystem (just a check, not a fix or rebuild)? I seem to remember this being mentioned here at some point in time, but I was unable to find it in the mailing list archive. Thanks!
Re: online fsck
On Thu, 22 Apr 2004, Jure Pe??ar wrote: > Is it theoretically posible? It is actually implemented in the BSD kernels, for UFS. Look for "softdep" or "softupdates". As for other file systems, when crashing while the write cache is enabled (unfortunately, it is in FreeBSD for instance), it can royally screw up your file system. Beyond repair, to a "use your backup" state. -- Matthias Andree Encrypt your mail: my GnuPG key ID is 0x052E7D95
Re: online fsck
On Thu, Apr 22, 2004 at 10:51:12AM -0700, Hans Reiser wrote: > Chris Mason wrote: > > > >Online check is easy, just use lvm or evms to make a snapshot and then > >check the snapshot. > > > Requires that users use lvm before discovering the need for fsck, but, > yes. What would be ideal would be some support for finding the > inconsistency on the snapshot, and then fixing it on the real fs using > the information learned from the snapshot fsck. Taking a snapshot of the entire FS to fix metadata seems somewhat expensive. Any thoughts on intercepting FS activity to locate metadata that needs to be checked now rather than during the next moment the fs or disk is idle? Snapshotting 40G has a pretty small additional cost. Snapshotting 1TB is somewhat more expensive. > > >Online rebuild tree would be impossible by > >definition (since it throws out everything above the leaf level and > >rebuilds). Build a new tree and keep the old tree until the new tree is finished? -- Chris Dukes Been there, done that, got the slightly-charred t-shirt. -- Crowder
Re: online fsck
Chris Mason wrote: On Thu, 2004-04-22 at 13:51, Hans Reiser wrote: Chris Mason wrote: On Thu, 2004-04-22 at 09:00, Jure Pečar wrote: Hi all, Is it theoretically posible? Like, does it need a drastic redesing of reiserfs or just sufficient $$ directed to the team to be implemented? Because i think that reiserfsck --check in 12h + --rebuild-tree in 18h is still waaay too much downtime for a 500gb mail server... Online check is easy, just use lvm or evms to make a snapshot and then check the snapshot. Requires that users use lvm before discovering the need for fsck, but, yes. What would be ideal would be some support for finding the inconsistency on the snapshot, and then fixing it on the real fs using the information learned from the snapshot fsck. Which should be possible, espeically for corruptions at the leaf levels. Things like incorrect i_size, pointers to files that don't exist, etc. Corruptions that require a full blow rebuild-tree will be much harder. I was wrong to say that an online rebuild-tree would be impossible, but it certainly does seem tricky. Basically you could freeze the old tree, using it for readonly lookups. Rebuild to new tree in the background, and verify things you find in the old tree in the new tree (to catch a file that has been deleted while the FS was mounted but is present in the old tree). -chris rebuild-tree would likely be a major engineering effort which would require funding from someone who cared enough to pay for it., finding inconsistencies on a snapshot and applying them to a live system would be less difficult but still substantive work.
Re: online fsck
On Thu, 2004-04-22 at 13:51, Hans Reiser wrote: > Chris Mason wrote: > > >On Thu, 2004-04-22 at 09:00, Jure Pečar wrote: > > > > > >>Hi all, > >> > >>Is it theoretically posible? > >> > >>Like, does it need a drastic redesing of reiserfs or just sufficient $$ > >>directed to the team to be implemented? > >> > >> > >>Because i think that reiserfsck --check in 12h + --rebuild-tree in 18h is > >>still waaay too much downtime for a 500gb mail server... > >> > >> > > > >Online check is easy, just use lvm or evms to make a snapshot and then > >check the snapshot. > > > Requires that users use lvm before discovering the need for fsck, but, > yes. What would be ideal would be some support for finding the > inconsistency on the snapshot, and then fixing it on the real fs using > the information learned from the snapshot fsck. Which should be possible, espeically for corruptions at the leaf levels. Things like incorrect i_size, pointers to files that don't exist, etc. Corruptions that require a full blow rebuild-tree will be much harder. I was wrong to say that an online rebuild-tree would be impossible, but it certainly does seem tricky. Basically you could freeze the old tree, using it for readonly lookups. Rebuild to new tree in the background, and verify things you find in the old tree in the new tree (to catch a file that has been deleted while the FS was mounted but is present in the old tree). -chris
Re: online fsck
Chris Mason wrote: On Thu, 2004-04-22 at 09:00, Jure Pečar wrote: Hi all, Is it theoretically posible? Like, does it need a drastic redesing of reiserfs or just sufficient $$ directed to the team to be implemented? Because i think that reiserfsck --check in 12h + --rebuild-tree in 18h is still waaay too much downtime for a 500gb mail server... Online check is easy, just use lvm or evms to make a snapshot and then check the snapshot. Requires that users use lvm before discovering the need for fsck, but, yes. What would be ideal would be some support for finding the inconsistency on the snapshot, and then fixing it on the real fs using the information learned from the snapshot fsck. Online rebuild tree would be impossible by definition (since it throws out everything above the leaf level and rebuilds). -chris
Re: online fsck
Jure PeÄar wrote: On Thu, 22 Apr 2004 16:24:09 +0100 Chris Dukes <[EMAIL PROTECTED]> wrote: It is worth mentioning that FreeBSD supposedly has an online in the background fsck for UFS2. Yes ... such online fsck is the only major feature i miss on Linux. (my boss is pushing Veritas on me because it has such feature ...) If you are a large site, for the cost of Veritas we can possibly write one for you.
Re: online fsck
On Thu, Apr 22, 2004 at 07:45:08PM +0400, Nikita Danilov wrote: > Chris Dukes writes: > > > > It is worth mentioning that FreeBSD supposedly has an online in the > > background fsck for UFS2. > > Wait a second. Assuming that kernel code has no bugs, the only > corruption that may happen when soft-updates are used is leaked disk > space. As I understand it, FreeBSD's background fsck fixes this problem > and only it. Reading through http://www.shiningsilence.com/dbsdlog/archives/000212.html That would appear to be the case. And current experience indicates that UFS2 does not recover gracefully when used on a disk that has been in service beyond its life expectancy. > > But, under the same assumption, journalled file system needs _no_ fsck > at all. In this age of hot plugging, rough environments, drives that have write caching turned on no matter how hard folks try to turn it off, petabytes of spinning storage, and the continuing misunderstanding of MTBF, that is becoming an increasingly poor assumption. Measures to verify that the metadata are sane without making all of the data unavailable are needed. Oh well back to fighting X for a project where things don't die when some important metadata is corrupted, they just cough and sputter a bit before continuing. -- Chris Dukes Been there, done that, got the slightly-charred t-shirt. -- Crowder
Re: online fsck
On Thursday 22 April 2004 12:45, Nikita Danilov wrote: > Chris Dukes writes: > > On Thu, Apr 22, 2004 at 04:24:02PM +0200, Jure Pe??ar wrote: > > > On Thu, 22 Apr 2004 09:34:14 -0400 > > > > > > Chris Mason <[EMAIL PROTECTED]> wrote: > > > > Online check is easy, just use lvm or evms to make a snapshot and > > > > then check the snapshot. Online rebuild tree would be impossible by > > > > definition (since it throws out everything above the leaf level and > > > > rebuilds). > > > > > > Well yes, one can do --check on ro mounted fs. > > > > > > I was thinking about fixing things up on an active rw filesystem ... > > > where if something is found that does not look right, it is reported > > > and fixed on the fly. > > > > It is worth mentioning that FreeBSD supposedly has an online in the > > background fsck for UFS2. > > Wait a second. Assuming that kernel code has no bugs, the only > corruption that may happen when soft-updates are used is leaked disk > space. As I understand it, FreeBSD's background fsck fixes this problem > and only it. > > But, under the same assumption, journalled file system needs _no_ fsck > at all. > > > I don't know the implementation details as I am busy fighting XF86 4.4 > > on FreeBSD. > > > > > So is it theoretically posible to fix stuff like these on the fly? > > > > -- > > Chris Dukes > > Been there, done that, got the slightly-charred t-shirt. -- Crowder > > Nikita. That's exactly my thought. Wasn't write cache enabled on those disks ? That certainly would defeat any journaling filesystem. Marcelo Pacheco
