[Jfs-discussion] Re: Directory no longer accessible after jfs_fsck recovery
On Thu, 2005-03-03 at 18:09 -0800, Junfeng Yang wrote:
> Hi,
>
> We are from the Stanford Checker team and now are working on a
> file system checker called FiSC. We run FiSC on Linux 2.6.8.1
> with the latest jfs file system utilities and it complains that
> after recovery of a crashed disk image, a directory is no longer
> accessible.
>
> The OS we are running is
> Linux 2.6.8.1 #13 Thu Feb 24 20:41:52 PST 2005 i686 GNU/Linux
This is a pretty old kernel. I'm not sure, but it's possible that this
bug was fixed by this changeset:
http://linux.bkbits.net:8080/linux-2.5/[EMAIL PROTECTED]
which was first in 2.6.11-rc1
> The fsck we use is
> jfs_fsck version 1.1.7, 22-Jul-2004
>
> Disk image can be obtained at
> http://fisc.stanford.edu/bug1/crash.img.bz2
>
> crash.img is obtained by running a bunch of file system
> operations. The last operation is symlink("/007", "/0005/0009").
> Right after this operation, we "crash" and run jfs_fsck to recover
> the crashed disk image. After the first run of jfs_fsck, we'll
> get "Permission denied" error when we try to access directory
> 0005. Second run of jfs_fsck fixes the problem.
>
> As usual, confirmations/clarifications are appreciated.
Can you try to recreate the problem on a 2.6.11 kernel?
>
> Thanks,
> -Junfeng
Thanks,
Shaggy
--
David Kleikamp
IBM Linux Technology Center
___
Jfs-discussion mailing list
Jfs-discussion@www-124.ibm.com
http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
[Jfs-discussion] JFS mailing lists have been migrated to sourceforge.net
JFS is being migrated from the IBM developerworks site to sourceforge.net. The mailing lists are migrated, and the migration of the rest of the site should be complete by Friday. After that, the old site should redirect you to the new one. As this is the first post to the new groups, it is also a test posting, but I'm assuming it will be successful. I'm spamming both the old and new lists with this post, so I apologize if you receive this multiple times. If anyone wants to respond, please only cc [EMAIL PROTECTED] Thanks, Shaggy -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
[Jfs-discussion] Re: me too: JFS on alpha (!= 4k pagesize)
On Sat, 2005-02-12 at 23:50 -0500, John S. Bucy wrote: > I saw the bug tracker entry about JFS only works for 4k pages; we'd > like to use JFS on alpha so this breaks for us too. In the meantime, > it should probably refuse to mount rather than oops! I've meant to drop such a patch, but never did. I need to do this in the 2.4 kernel, but I'm close to a real fix on 2.6. I can give you a patch to test. I currently have one against 2.6.10 and 2.6.11-rc2-mm1. Let me know what 2.6 kernel you want a patch against and I'll send it to you. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] FW: problem with jfs corrputed inode after system crash
On Mon, 2005-02-07 at 09:41 -0500, Guerra, Jim wrote: > Shaggy - > >That patch you sent worked nicely. Thank you > >jfs_fsck ran for 4+ hours but in the end it worked out. Wow, 4+ hours! > is there any > other information that you want/need from the system >to help with a diagnosis ? I'm not too confident I'll be able to learn much, but I'd like to see what the contents of that duplicate allocation was. Can you run jfs_debugfs, and send me the output of "d 3876468"? Thanks, Shaggy -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
[Jfs-discussion] Re: problem with jfs corrputed inode after system crash
On Thu, 2005-02-03 at 11:37 -0500, Guerra, Jim wrote:
> Shaggy -
>
>we just had a system crash with the following kernel messages
>
>
> Feb 2 15:40:26 va008 kernel: c016337b
> Feb 2 15:40:26 va008 kernel: PREEMPT SMP
> Feb 2 15:40:26 va008 kernel: Modules linked in: dm_mod
> Feb 2 15:40:26 va008 kernel: CPU:3
> Feb 2 15:40:26 va008 kernel: EIP:0060:[bd_forget+43/96]Not tainted
> VLI
> Feb 2 15:40:26 va008 kernel: EIP:0060:[]Not tainted VLI
> Feb 2 15:40:26 va008 kernel: EFLAGS: 00010206 (2.6.10)
> Feb 2 15:40:26 va008 kernel: EIP is at bd_forget+0x2b/0x60
I haven't seen a crash like this before. I'm guessing that maybe
inode->i_devices might have gotten corrupted, but I don't know for sure.
bd_forget doesn't do too much.
> after we rebooted fsck.jfs came back with
>
>
> fsck.jfs version 1.1.7, 22-Jul-2004
> processing started: 2/3/2005 11.20.38
> The current device is: /dev/vg00/lv00
> Open(...READ/WRITE EXCLUSIVE...) returned rc = 0
> Primary superblock is valid.
> The type of file system for the device is JFS.
> Block size in bytes: 4096
> Filesystem size in blocks: 731791360
> **Phase 0 - Replay Journal Log
> LOGREDO: Log already redone!
> logredo returned rc = 0
> **Phase 1 - Check Blocks, Files/Directories, and Directory Entries
> Primary metadata inode A16 is corrupt.
> Errors detected in the Primary File/Directory Allocation Table.
> Duplicate reference to 4 block(s) beginning at offset 3876468 found in file
> system object IA16.
> Inode A16 has references to cross linked blocks.
> Multiple metadata references to 4 blocks beginning at offset 3876468 have
> been detected.
> Duplicate block references have been detected in Metadata. CANNOT CONTINUE.
> Filesystem is dirty.
> processing terminated: 2/3/2005 11:24:13 with return code: 0 exit code:
> 4.
>
>
> how can I fix the problem ?
This one's pretty ugly. I'm not sure how this could have happened, but
this little patch might be able to salvage the partition.
This is a special case to fix this specific partition. After running
the patched version of jfs_fsck, throw it away. It will always
invalidate an inode extent beginning at block 3876468.
I once sent a similar patch to someone, but I never heard back whether
or not it worked. :^(
If it helps, maybe I can do something to make fsck do this kind of thing
automatically.
Good Luck,
Shaggy
--
David Kleikamp
IBM Linux Technology Center
diff -urp jfsutils-1.1.7/fsck/fsckwsp.c jfsutils-jguerra/fsck/fsckwsp.c
--- jfsutils-1.1.7/fsck/fsckwsp.c 2004-04-28 13:58:42.0 -0500
+++ jfsutils-jguerra/fsck/fsckwsp.c 2005-02-03 12:57:15.0 -0600
@@ -3157,6 +3157,17 @@ int process_extent(struct fsck_inode_rec
? agg_recptr->highest_valid_fset_datablk : extent_addr +
extent_length - 1;
+ /*
+ * Special case to fix jguerra's file system
+ */
+ if (extent_addr == 3876468) {
+ *adjusted_length = 0;
+ *extent_is_valid = 0;
+ agg_recptr->corrections_needed = 1;
+
+ return FSCK_OK;
+ }
+
if (((first_valid > agg_recptr->highest_valid_fset_datablk)
&& (inorecptr->inode_type != metadata_inode)) ||
/*
Re: [Jfs-discussion] Weird problem with a file
On Tue, 2005-02-01 at 20:15 -0500, Chris Shafer wrote: > Hello, > > One of my users uploaded a file to JFS partition that appears it might contain > characters from a different character set. The file shows up in the tab > completion list for bash, and in the listing of the directory inode in > jfs_debugfs but when I try to rename it, it can not stat the file. Same when I > try to excute any command on it for that matter. I was curious if there was a > way to rename the file through jfs_debugfs. You can, but it might be easier to mount the partition with -o iocharset=utf8, and you should be able to rename the file to an ascii name, then remount it normally. To do it through jfs_debugfs, you should find the inode number of the containing directory, ls -i, and use the "dtree" subcommand in jfs_debugfs to navigate the directory tree. If you need to do this, it shouldn't be too hard to figure out from there. > Thank you > Chris -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Corrupted 4.4TB JFS FS
It looks pretty messed up. With some work we could probably get a little further, but I don't think it's worth it. It looks like there is a lot that is messed up. The scary part is that we really don't know how it happened, so we don't know if it will happen again. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Corrupted 4.4TB JFS FS
This first superblock is good, but it's unhappy with another data structure, so fsck is wanting to compare an address in the primary superblock with that of the secondary (which I didn't have you copy). I'm not sure if this is going to get you much further, but we can try. If you still have the sparse file that you ran mkfs against, run this to copy the secondary superblock: dd if=/somefile of=/dev/VolGroup00/lvol0 bs=4096 skip=15 seek=15 count=1 On Thu, 2005-01-27 at 15:12 -0600, Sean Murphy wrote: > [EMAIL PROTECTED] ~]# fsck.jfs -v /dev/VolGroup00/lvol0 > fsck.jfs version 1.1.7, 22-Jul-2004 > processing started: 1/27/2005 15.11.50 > Using default parameter: -p > The current device is: /dev/VolGroup00/lvol0 > Open(...READ/WRITE EXCLUSIVE...) returned rc = 0 > Invalid data (13) detected in the superblock (P). > Invalid magic number in the superblock (S). > Superblock is corrupt and cannot be repaired > since both primary and secondary copies are corrupt. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Corrupted 4.4TB JFS FS
I expected that to at least get by the superblock check. Can you run fsck.jfs again with the -v flag? You can also use jfs_debug to see if the superblock looks somewhat sane. On Thu, 2005-01-27 at 13:51 -0600, Sean Murphy wrote: > I tried copying in the fresh superblock and ended up with this. > Anything else? > It about 10 days worth of computing on there, so I'm only spending > another few hours > on it before we just regenerate it all. > > [EMAIL PROTECTED] ~]# fsck.jfs -f /dev/VolGroup00/lvol0 fsck.jfs version > 1.1.7, 22-Jul-2004 > processing started: 1/27/2005 13.49.9 > The current device is: /dev/VolGroup00/lvol0 > Superblock is corrupt and cannot be repaired > since both primary and secondary copies are corrupt. > > CANNOT CONTINUE. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Corrupted 4.4TB JFS FS
If you want to try to fix the superblock, you can create a sparse file that is the same size as the volume: dd if=/dev/zero of=/somefile bs=4096 seek=1172111359 count=1 Then run "mkfs -tjfs /somefile" Then copy the superblock: dd if=/somefile of=/dev/VolGroup00/lvol0 bs=4096 skip=8 seek=8 count=1 I can't guarantee how much further fsck will get, but it may be worth a try. On Wed, 2005-01-26 at 18:37 -0600, Sean Murphy wrote: > Ok, I put the output of sup s below. It does look pretty random. But > if > I do jfs_debug a d 8 I can recognize directory names in the blocks. So > the data isn't completely trashed. d 8 should show you the superblock, so directory names here would be a problem. But then again, jfs_debug bases this on the block size s_bsize, so who knows where it's reading from. > I'd be happy to give you a login on this machine if you want to try to > debug it. Let me know if fixing the superblock makes any progress. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Corrupted 4.4TB JFS FS
On Wed, 2005-01-26 at 14:28 -0600, Sean Murphy wrote: > I've got a Dual Xeon machine running Fedora Core 3 attached to an Apple > Xserve RAID with 2 x 2.2TB devices. > Then I use LVM to put them together into a 4.4TB device. Then there > once was a JFS filesystem on this 4.4TB device. > Yesterday the filesystem had gone read-only spitting out lots of these > Jan 25 03:24:28 higgs08 kernel: ERROR: (device dm-0): DT_GETPAGE: dtree > page corrupt > I unmounted the filesystem and run fsck and it fixed a bunch of stuff > (sorry didn't capture that). > I remounted it and all was happy until this morning I started getting > these > Jan 25 22:15:58 higgs08 kernel: ERROR: (device dm-0): dbAllocNext: > Corrupt dmap page > and I unmounted the filesystem and now it won't remount. jfs_fsck says > that both superblocks are corrupt. > I used jfs_debug to look at the two superblocks and they both contain > the same gibberish. > > I read one place that possible copying a superblock from a fresh > mkfs.jfs of an identical disk could fix this. This may get you a little further, but it looks as though more than just the superblocks are corrupted. The primary superblock rarely gets touched during normal operations (and only at mount & unmount time) and the secondary superblock isn't touched at all. I don't know what could cause random data to be written over both superblocks, but my guess is that the damage is more widespread. Were there any syslog errors that appear to come from the lower-level drivers (dm, scsi)? > Any other idea for resurrecting this data? > > > fsck output > [EMAIL PROTECTED] log]# jfs_fsck /dev/VolGroup00/lvol0 > jfs_fsck version 1.1.7, 22-Jul-2004 > processing started: 1/26/2005 14.24.55 > Using default parameter: -p > The current device is: /dev/VolGroup00/lvol0 > > The superblock does not describe a correct jfs file system. > > If device /dev/VolGroup00/lvol0 is valid and contains a jfs file system, > then both the primary and secondary superblocks are corrupt > and cannot be repaired, and fsck cannot continue. > > Otherwise, make sure the entered device /dev/VolGroup00/lvol0 is > correct. > > jfs_debug output > > su > [1] s_magic:'Pm??' [15] s_ait2.addr1: 0x47 > [2] s_version: 2042101673 [16] s_ait2.addr2: > 0xdad0bf23 > > s2p This doesn't display the secondary superblock. It is a slightly different format as the 'sup' command. "sup s" will show you the secondary superblock. > [1] s_magic:'Pm??' [16] s_aim2.len:15262231 > [2] s_version: 2042101673 [17] s_aim2.addr1: > 0xfb > > > > > LMV info I don't know enough about LVM to know it anything here is unusual. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Dense files
On Tue, 2005-01-25 at 01:26 +0200, Hirsch, Michael wrote: > Hi, > > I'm probably asking something so obvious ... but I admit that I'm > somewhat of a beginner. Actually, the answer isn't so obvious. > I saw dense files mentioned in some of the publications on the open > source JFS, but have not found a way of doing this from within the > docs, the code, the mkfs or the tune utility. I've tried to trace the > *SPARSE flags in the fs/jfs and jfsutils. I've tried searching the > jfs-*.mbox files on the official jfs website for "sparse" and "dense" > and "contiguous" but nothing turned up. Looked through the file > layout paper which has a great example of a sparse file, but I'm > looking for a dense example. jfs on Linux does not support the dense file option. The docs mention dense files because the code was originally ported from OS/2, where dense-file creation was the default behavior. At the time of the port, it seemed a reasonable thing to eventually support both dense and sparse options in Linux, as this would be more compatible with OS/2. As time went on, the dense file support was never much of a priority, and it probably won't be. Applications that would benefit from dense files can write a file sequentially when creating it, and get the same behavior. > I'm running RH ES3 AS update 2, kernel 2.4.21-20.Elsmp on an x86_64 > platform if any of this is relevant. > > Pointers would be appreciated. > > Thanks in advance, > Michael. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] mount -o remount,resize bug report
On Fri, 2005-01-21 at 17:50 -0600, Jon Nelson wrote: > On Fri, 21 Jan 2005, Dave Kleikamp wrote: > > If you have CONFIG_MAGIC_SYSRQ defined, you could 'echo t > > > /proc/sysrq-trigger' and send me the dmesg output. (Please don't cc > > the mailing list) > > I don't know what could be private in these files, but if there is > just let me know. Nothing private. I wasn't sure how big the files would be and didn't think it was necessary to spam everyone on the list with large attachments. > Included are two files: x, and y. > > x is taken while in runlevel 5, right after the hang. > y is taken while in runlevel 1, also after the hang. > > I thought that perhaps y might help narrow things down for you. It looks like the same hang as before. I actually was optimistic that changing the low-water mark would fix the problem. That would have changed the numbers back to about what they have previously been before an earlier change, and I didn't recall having seen this hang before. Upon further reflection, though, our performance team had some problems before that required them to increase the number of tlocks, so I will have to come up with a more complete solution. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] mount -o remount,resize bug report
On Thu, 2005-01-20 at 19:38 -0600, Jon Nelson wrote: > On Thu, 20 Jan 2005, Jon Nelson wrote: > > > NOTE: I'll be trying the latter (002013) patch tonight. > > I ran the exact same test as before with the new module. > On the third execution of find (other terminal still executing the first > run of fstest), it (the filesystem) locked. :-( > > I'll leave it locked for a while (at least a day) if there are any > specific diagnostics you'd like me to perform. If you have CONFIG_MAGIC_SYSRQ defined, you could 'echo t > /proc/sysrq-trigger' and send me the dmesg output. (Please don't cc the mailing list) -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
[Jfs-discussion] Re: [PATCH 6/40] fs/jfs_logmgr: replace schedule_timeout() with msleep()
On Thu, 2005-01-20 at 11:02 -0800, Nishanth Aravamudan wrote:
> Hi,
>
> Please consider applying.
I will. It looks good.
> Description: Use msleep() instead of schedule_timeout() to guarantee the task
> delays as expected. The current code uses TASK_INTERRUPTIBLE; however, it does
> not check for signals, so I do not think the change to msleep() is necessarily
> bad.
Right, TASK_INTERRUPTIBLE doesn't make sense here, so msleep is a better
fit.
Thanks,
Shaggy
> Signed-off-by: Nishanth Aravamudan <[EMAIL PROTECTED]>
>
> --- 2.6.11-rc1-kj-v/fs/jfs/jfs_logmgr.c 2005-01-15 16:55:41.0
> -0800
> +++ 2.6.11-rc1-kj/fs/jfs/jfs_logmgr.c 2005-01-18 10:53:46.0 -0800
> @@ -67,6 +67,7 @@
> #include/* for sync_blockdev() */
> #include
> #include
> +#include
> #include "jfs_incore.h"
> #include "jfs_filsys.h"
> #include "jfs_metapage.h"
> @@ -1612,8 +1613,7 @@ void jfs_flush_journal(struct jfs_log *l
>*/
> if ((!list_empty(&log->cqueue)) || !list_empty(&log->synclist)) {
> for (i = 0; i < 800; i++) { /* Too much? */
> - current->state = TASK_INTERRUPTIBLE;
> - schedule_timeout(HZ / 4);
> + msleep(250);
> if (list_empty(&log->cqueue) &&
> list_empty(&log->synclist))
> break;
--
David Kleikamp
IBM Linux Technology Center
___
Jfs-discussion mailing list
Jfs-discussion@www-124.ibm.com
http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] backporting jfs patches to kernel-2.4.27 and kernel-2.4.28
On Wed, 2005-01-19 at 08:49 -0800, usr local wrote: > The patch mailing list shows a number of patches post > jfs 1.1.7 that seem important. > > Are there any 'must-have' jfs patches that should be > backported for systems using kernel-2.4.27 or > kernel-2.4.28? The most important are in 2.4.29-rc1: o JFS: Fix extent overflow bugs o JFS: avoid assert in lbmfree o JFS: Fix endian errors o JFS: fix race in jfs_commit_inode I'm not sure if any of these are must-have, but they shouldn't cause any problems either. > Debian is about to release Debian 3.1 with > kernel-2.4.27 which includes jfs patches included in > kernel-2.4.28. It would be nice if jfs shipped with > Debian 3.1 is as stable as possible (I've been > recommending it to many). > > And are there plans to release jfsutils 1.1.8 in the > near future? Yes, I've been meaning to fix one more thing and release 1.1.8. Is there a specific deadline that I should make? Thanks, Shaggy -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] mount -o remount,resize bug report
On Wed, 2005-01-19 at 08:34 -0600, Jon Nelson wrote: > On Wed, 19 Jan 2005, Dave Kleikamp wrote: > > > Jon, > > I'm sorry for not getting back to you. I've been meaning to try to > > experiment with resizing to see if I can reproduce this, but I haven't > > gotten around to it yet. I have to admit, I haven't played with the > > resize function in a while. > > > > I haven't forgotten this one. > > Good to know. How goes things with the other bug I reported (lock FS + > cause mild corruption)? I believe this patch fixes the problem: http://oss.software.ibm.com/pipermail/jfs-discussion/2005-January/002006.html and this one is probably even better: http://oss.software.ibm.com/pipermail/jfs-discussion/2005-January/002013.html I failed to cc you when I sent these, but they did go to the mailing list. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] mount -o remount,resize bug report
Jon, I'm sorry for not getting back to you. I've been meaning to try to experiment with resizing to see if I can reproduce this, but I haven't gotten around to it yet. I have to admit, I haven't played with the resize function in a while. I haven't forgotten this one. Thanks, Shaggy On Wed, 2005-01-12 at 12:16 -0600, Jon Nelson wrote: > [ This is a repost from an earlier message so that it can be it's own > thread. ] > > I needed to enlarge one of my jfs filesystems that had run out of space. > > I ran: > lvresize --size=+1G /dev/ > Then I ran: > mount -o remount,resize / > > Everything went smoothly. df showed the size increase. > Shortly thereafter, I got *ONE* message from a userland process about it > being out of space (err. ENOSPC) and then 3 or 4 "read-only > file system" (EROFS). /var/log/messages had but one message for each of > the "read-only file system" messages: > > ERROR: (device dm-3): diAllocIno: nfreeinos = 0, but iag on freelist > > I didn't see a formal bug reporting mechanism so I'm trying this. > > -- > Jon Nelson <[EMAIL PROTECTED]> > ___ > Jfs-discussion mailing list > Jfs-discussion@www-124.ibm.com > http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion > > > > -- > Jon Nelson <[EMAIL PROTECTED]> > > ___ > Jfs-discussion mailing list > Jfs-discussion@www-124.ibm.com > http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] jfs kernel oops and fsck.jfs segfault.
On Wed, 2005-01-19 at 12:13 +0100, Marcel Lanz wrote: > Hi > > I have some troubles with a jfs partition on an external USB drive > (maxtor onetouch 200GB) > > I had a jfs related kernel oops (see files below), then rebootet and did > a fsck which segfaulted: > http://hacklabs.ch/lanz/jfs_error/jfs_error_fsckh The segfault is fixed in jfsutils-1.1.7 (you are running 1.1.6). There was a bad format string, which tried to print an integer as a string. http://www10.software.ibm.com/developer/opensource/jfs/project/pub/jfsutils-1.1.7.tar.gz > logs are here: > http://hacklabs.ch/lanz/jfs_error/ > > Actually I don't know what I should do now, or not do, because it seems > something went wrong and I can't access my partition anymore. I didn't > try to mount or did anyting after the fsck segfault. I'm not sure what to make of all of the scsi messages: Current sda: sense key No Sense I'm not sure if they are any indication of I/O errors or not. You may want to try fixing the partition with the 1.1.7 version of jfs_fsck, and see whether or not the problems go away. > I have a 2.6.10 vanialla kernel (.config on request) > > regards > marcel -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] [PATCH (again)] Strange problem with JFS
On Wed, 2005-01-12 at 12:00 -0600, Steven Pratt wrote: > Sonny Rao wrote: > >So, to summarize this logic, if you haven't set anything and you have > >more than 1GB of ram total, the number of TxLocks is maxed out. If > >you haven't set anything and you have 1GB of RAM or less, then you > >get something like a quarter of the number of kilobytes of ram worth > >of txlocks? If you have exactly 1GB of RAM, then won't that number > >be too big? ( 1048576 / 4 == 262144 ) > > > No, because the math is on pages not KB. So 262144 (pages) / 4 = 65536 > locks, or max. Right. I'm not sure if these are ideal numbers, but they seem reasonable. It comes out to 1 tlock for every 4 pages of physical memory. > > > >Both the number of txlocks and txblocks are limited to a 16bit number > >correct? > > > > > Yes, I think. Yes. They are represented by a u16 field. I once tried to widen this to 32 bits, but I messed up both structures because they overlay a lot of stuff, so I changed them back. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] [PATCH (again)] Strange problem with JFS
On Tue, 2005-01-11 at 12:30 -0600, Steven Pratt wrote:
> Also, is it maybe time to increase the default number of locks? We had
> similar problems on large systems in SLES9 testing and we still have to
> run SPECSFS with TxLocks set to 65k.
How's this look?
= fs/jfs/jfs_txnmgr.c 1.64 vs edited =
--- 1.64/fs/jfs/jfs_txnmgr.c2004-11-01 07:59:47 -06:00
+++ edited/fs/jfs/jfs_txnmgr.c 2005-01-11 13:44:12 -06:00
@@ -93,15 +93,15 @@
} TxStat;
#endif
-static int nTxBlock = 512; /* number of transaction blocks */
+static int nTxBlock = -1; /* number of transaction blocks */
module_param(nTxBlock, int, 0);
MODULE_PARM_DESC(nTxBlock,
-"Number of transaction blocks (default:512, max:65536)");
+"Number of transaction blocks (max:65536)");
-static int nTxLock = 4096; /* number of transaction locks */
+static int nTxLock = -1; /* number of transaction locks */
module_param(nTxLock, int, 0);
MODULE_PARM_DESC(nTxLock,
-"Number of transaction locks (default:4096, max:65536)");
+"Number of transaction locks (max:65536)");
struct tblock *TxBlock;/* transaction block table */
static int TxLockLWM; /* Low water mark for number of txLocks used */
@@ -249,6 +249,25 @@
int txInit(void)
{
int k, size;
+ struct sysinfo si;
+
+ /* Set defaults for nTxLock and nTxBlock if unset */
+
+ if (nTxLock == -1) {
+ if (nTxBlock == -1) {
+ /* Base default on memory size */
+ si_meminfo(&si);
+ if (si.totalram > (256 * 1024)) /* 1 GB */
+ nTxLock = 64 * 1024;
+ else
+ nTxLock = si.totalram >> 2;
+ } else if (nTxBlock > (8 * 1024))
+ nTxLock = 64 * 1024;
+ else
+ nTxLock = nTxBlock << 3;
+ }
+ if (nTxBlock == -1)
+ nTxBlock = nTxLock >> 3;
/* Verify tunable parameters */
if (nTxBlock < 16)
@@ -259,6 +278,9 @@
nTxLock = 256; /* No one should set it this low */
if (nTxLock > 65536)
nTxLock = 65536;
+
+ printk(KERN_INFO "JFS: nTxBlock = %d, nTxLock = %d\n",
+ nTxBlock, nTxLock);
/*
* initialize transaction block (tblock) table
*
@@ -266,8 +288,8 @@
* tid = 0 is reserved.
*/
TxLockLWM = (nTxLock * 4) / 10;
- TxLockHWM = (nTxLock * 8) / 10;
- TxLockVHWM = (nTxLock * 9) / 10;
+ TxLockHWM = (nTxLock * 7) / 10;
+ TxLockVHWM = (nTxLock * 8) / 10;
size = sizeof(struct tblock) * nTxBlock;
TxBlock = (struct tblock *) vmalloc(size);
___
Jfs-discussion mailing list
Jfs-discussion@www-124.ibm.com
http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] [PATCH (again)] Strange problem with JFS
On Tue, 2005-01-11 at 12:30 -0600, Steven Pratt wrote: > Also, is it maybe time to increase the default number of locks? We had > similar problems on large systems in SLES9 testing and we still have to > run SPECSFS with TxLocks set to 65k. Yeah, I've thought of this before and haven't done it. I think I'll base it on the amount of physical memory, so we don't introduce any new problems for small-memory machines. > Steve -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] [PATCH (again)] Strange problem with JFS
On Tue, 2005-01-11 at 11:38 -0500, Sonny Rao wrote: > Yeah, that seems to fix it for this particular case. I wonder if > we're just delaying the inevitable though? Would it make sense to > pre-allocate tlocks somehow before holding important semaphores? > Admittedly, my understanding of the txnmgr is a bit limited, so this > may be total nonsense. That sounds like a good idea. I'm not sure how easy that will be to implement, but I'll take a look at it. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
[Jfs-discussion] Re: [PATCH (again)] Strange problem with JFS
I think you can throw away the first patch. I no longer believe that the code can deadlock due to the commit_sem. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
[Jfs-discussion] [PATCH (again)] Strange problem with JFS
On Tue, 2005-01-11 at 08:20 -0600, Dave Kleikamp wrote: > I also noticed that several of the static functions called by diAlloc do > show up in this latest stack trace, so I believe I was mistaken about > the cause of the earlier deadlock. I now think that the thread in > diAlloc was trying to grab the AG_LOCK, and never made it down into > diNewIAG. I'm afraid there may still be two different problems that > still need to be figured out. No, I think it's the same problem. I only saw a subset of the blocked threads in the earlier case, and I believe there probably was a thread blocked in diNewIAG. This patch simply blocks new transactions earlier when the tlocks are starting to get scarce. When I implemented the multiple commit threads, I split TxLockVHWM out of TxLockHWM, which was 80% of nTxLock. So this patch puts TxLockVHWM back to the old value of TxLockHWM. Maybe I was a little too bold waiting until 90% of the tlocks were in use before blocking new transactions. (jfsSync thread is awakened when TxLockHWM is hit, and transactions are actually blocked when TxLockVHWM is hit.) = fs/jfs/jfs_txnmgr.c 1.64 vs edited = --- 1.64/fs/jfs/jfs_txnmgr.c2004-11-01 07:59:47 -06:00 +++ edited/fs/jfs/jfs_txnmgr.c 2005-01-11 09:42:49 -06:00 @@ -266,8 +266,8 @@ * tid = 0 is reserved. */ TxLockLWM = (nTxLock * 4) / 10; - TxLockHWM = (nTxLock * 8) / 10; - TxLockVHWM = (nTxLock * 9) / 10; + TxLockHWM = (nTxLock * 7) / 10; + TxLockVHWM = (nTxLock * 8) / 10; size = sizeof(struct tblock) * nTxBlock; TxBlock = (struct tblock *) vmalloc(size); ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Re: [PATCH] Strange problem with JFS
On Mon, 2005-01-10 at 19:03 -0500, Sonny Rao wrote: > On Mon, Jan 10, 2005 at 01:39:34PM -0600, Dave Kleikamp wrote: > > Does this patch help? > > > > It doesn't appear to help. It looks like pdflush is trying to > writeback inodes and is blocking waiting for a tlock in txLockAlloc > again, maybe this is holding everyone else from completing their > transactions? Also, you can clearly see one of the fstest threads in > the diNewIAG path waiting on a tlock. Yeah, that seems to be the trouble. We're not doing a good enough job of preventing tlock starvation. I also noticed that several of the static functions called by diAlloc do show up in this latest stack trace, so I believe I was mistaken about the cause of the earlier deadlock. I now think that the thread in diAlloc was trying to grab the AG_LOCK, and never made it down into diNewIAG. I'm afraid there may still be two different problems that still need to be figured out. > Also, I'm doing this on a 2.6.8.1 kernel, would that make any difference? I don't think so. > I've attached the full backtrace output and txstats output. > > Sonny -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
[Jfs-discussion] [PATCH] Strange problem with JFS
Does this patch help?
= fs/jfs/jfs_imap.c 1.37 vs edited =
--- 1.37/fs/jfs/jfs_imap.c 2004-12-07 15:38:08 -06:00
+++ edited/fs/jfs/jfs_imap.c2005-01-10 13:37:12 -06:00
@@ -2532,15 +2532,17 @@
* to include a new iag.
*/
+ /* Must grab commit_sem before getting write lock on ipimap */
+ down(&JFS_IP(ipimap)->commit_sem);
+
/* acquire inode map lock */
IWRITE_LOCK(ipimap);
if (ipimap->i_size >> L2PSIZE != imap->im_nextiag + 1) {
- IWRITE_UNLOCK(ipimap);
- IAGFREE_UNLOCK(imap);
jfs_error(imap->im_ipimap->i_sb,
"diNewIAG: ipimap->i_size is wrong");
- return -EIO;
+ rc = -EIO;
+ goto unlock;
}
@@ -2551,11 +2553,8 @@
* number limit.
*/
if (iagno > (MAXIAGS - 1)) {
- /* release the inode map lock */
- IWRITE_UNLOCK(ipimap);
-
rc = -ENOSPC;
- goto out;
+ goto unlock;
}
/*
@@ -2566,12 +2565,8 @@
/* Allocate extent for new iag page */
xlen = sbi->nbperpage;
- if ((rc = dbAlloc(ipimap, 0, (s64) xlen, &xaddr))) {
- /* release the inode map lock */
- IWRITE_UNLOCK(ipimap);
-
- goto out;
- }
+ if ((rc = dbAlloc(ipimap, 0, (s64) xlen, &xaddr)))
+ goto unlock;
/* assign a buffer for the page */
mp = get_metapage(ipimap, xaddr, PSIZE, 1);
@@ -2581,11 +2576,8 @@
*/
dbFree(ipimap, xaddr, (s64) xlen);
- /* release the inode map lock */
- IWRITE_UNLOCK(ipimap);
-
rc = -EIO;
- goto out;
+ goto unlock;
}
iagp = (struct iag *) mp->data;
@@ -2617,22 +2609,17 @@
* addressing structure pointing to the new iag page;
*/
tid = txBegin(sb, COMMIT_FORCE);
- down(&JFS_IP(ipimap)->commit_sem);
/* update the inode map addressing structure to point to it */
if ((rc =
xtInsert(tid, ipimap, 0, blkno, xlen, &xaddr, 0))) {
txEnd(tid);
- up(&JFS_IP(ipimap)->commit_sem);
/* Free the blocks allocated for the iag since it was
* not successfully added to the inode map
*/
dbFree(ipimap, xaddr, (s64) xlen);
- /* release the inode map lock */
- IWRITE_UNLOCK(ipimap);
-
- goto out;
+ goto unlock;
}
/* update the inode map's inode to reflect the extension */
@@ -2660,11 +2647,6 @@
*/
imap->im_freeiag = iagno;
- /* Until we have logredo working, we want the imap inode &
-* control page to be up to date.
-*/
- diSync(ipimap);
-
/* release the inode map lock */
IWRITE_UNLOCK(ipimap);
}
@@ -2688,11 +2670,17 @@
*iagnop = iagno;
*mpp = mp;
+ goto out;
+
+ unlock:
+ IWRITE_UNLOCK(ipimap);
+ up(&JFS_IP(ipimap)->commit_sem);
+
out:
/* release the iag free lock */
IAGFREE_UNLOCK(imap);
- return (rc);
+ return rc;
}
/*
___
Jfs-discussion mailing list
Jfs-discussion@www-124.ibm.com
http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Strange problem with JFS
On Mon, 2005-01-10 at 10:59 -0600, Dave Kleikamp wrote:
> This thread must be in diUpdatePMap, trying to acquire ipimap's
> rdwrlock: IREAD_LOCK(). The funny thing is that the only place a write
> lock is taken on this inode is in diNewIAG, which is only called under
> diAlloc, which I don't see in any stacks.
>
> I can't find any error path that would leave the lock taken, so I can't
> account for why this thread would be blocked. I'm sure you would have
> noticed if a thread oopsed in this path, leaving the lock locked.
I found the deadlock. This thread is the missing link:
fstestD C0348640 0 4598 4561 4599 4597 (NOTLB)
dbc11d88 0086 dbd57770 c0348640
df1ed83c d9912d50 d8dc7700 000f424a dbd57918
df7dd244
0286 dbc1 dbd57770 c026d7d7 df7dd24c 0001 dbd57770
c0118714
Call Trace:
[] __down+0x8b/0xfd
[] default_wake_function+0x0/0x12
[] __down_failed+0x8/0xc
[] .text.lock.jfs_imap+0x205/0x422 [jfs]
[] alloc_inode+0xf2/0x19b
[] ialloc+0x5b/0x1b0 [jfs]
[] jfs_mkdir+0x5d/0x2f0 [jfs]
[] recalc_task_prio+0x93/0x188
[] permission+0x67/0x79
[] link_path_walk+0xccf/0xdb2
[] cached_lookup+0x23/0x85
[] permission+0x67/0x79
[] vfs_mkdir+0xad/0x137
[] sys_mkdir+0xb7/0xf6
[] schedule_tail+0x41/0x4d
[] sysenter_past_esp+0x52/0x71
The compiler does a lot of inlining, but I do believe this is in
diNewIAG. The thread is doing a down(&JFS_IP(ipimap)->commit_sem) while
holding the rdwr_lock (which is a recent change). I will have to see
how hard this is to fix.
--
David Kleikamp
IBM Linux Technology Center
___
Jfs-discussion mailing list
Jfs-discussion@www-124.ibm.com
http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Strange problem with JFS
Sorry it's taken me so long to look into this, and thanks for all your help with this bug. On Sun, 2004-12-19 at 14:10 -0500, Sonny Rao wrote: > Hmm, okay I had thought it might be caused by a lack of available > transaction blocks or locks, but that doesn't seem to be the case > based on the data you provided. Actually, we do appear to be out of transaction locks (tlocks). jfsSync D C0348640 0 1803 1 2174 1802 (L-TLB) de94bdd8 0046 dee198b0 c0348640 def1d72c 0008 c01f1c3a def1d72c d808f600 000f424b dbe060f0 12707480 000f424b dee19a58 de94a000 de94bde8 c0118714 de94be08 e097fe87 dee198b0 Call Trace: [] generic_make_request+0x15e/0x1de [] default_wake_function+0x0/0x12 [] txLockAlloc+0xa7/0x160 [jfs] [] default_wake_function+0x0/0x12 [] default_wake_function+0x0/0x12 [] txLock+0x275/0x470 [jfs] [] lbmStartIO+0xb1/0xe0 [jfs] [] lbmWrite+0x10f/0x150 [jfs] [] __get_metapage+0x5e/0x3d0 [jfs] [] lmGCwrite+0xdd/0xf0 [jfs] [] diWrite+0x190/0x5f0 [jfs] [] txCommit+0x1b3/0x320 [jfs] [] txEnd+0x3b/0x140 [jfs] [] jfs_sync+0x21f/0x2c0 [jfs] [] default_wake_function+0x0/0x12 [] ret_from_fork+0x6/0x14 [] default_wake_function+0x0/0x12 [] jfs_sync+0x0/0x2c0 [jfs] [] kernel_thread_helper+0x5/0xb This is the thread that is trying to free some tlocks when they start getting low. I think generic_make_request is just stack noise. We really don't want txLockAlloc to block here. We try really hard not to exhaust all of the tlocks so that this thread can make some progress. I think the real problem though is the jfsCommit thread. If it were not blocked, I suspect that there would be a lot of tlocks freed. jfsCommit D C0348640 0 1802 1 1803 1801 (L-TLB) de429ee0 0046 dee18800 c0348640 df7dc000 dec76234 4000 e09357e4 de428000 00270284 dbe060f0 f6e6e500 000f424a dee189a8 dec7607c de429ef4 dee18800 dec76234 c026e3cd deeaa000 1bcc dec76080 dec76080 Call Trace: [] rwsem_down_read_failed+0x8f/0x17c [] .text.lock.jfs_imap+0x3ff/0x422 [jfs] [] txUpdateMap+0xae/0x250 [jfs] [] txEnd+0xd7/0x140 [jfs] [] txLazyCommit+0x20/0xe0 [jfs] [] jfs_lazycommit+0x1b8/0x1e0 [jfs] [] default_wake_function+0x0/0x12 [] ret_from_fork+0x6/0x14 [] default_wake_function+0x0/0x12 [] jfs_lazycommit+0x0/0x1e0 [jfs] [] kernel_thread_helper+0x5/0xb This thread must be in diUpdatePMap, trying to acquire ipimap's rdwrlock: IREAD_LOCK(). The funny thing is that the only place a write lock is taken on this inode is in diNewIAG, which is only called under diAlloc, which I don't see in any stacks. I can't find any error path that would leave the lock taken, so I can't account for why this thread would be blocked. I'm sure you would have noticed if a thread oopsed in this path, leaving the lock locked. > I am able to reproduce the problem on a laptop I have with me as well, > I'm running a 2.6.8.1 kernel. > > It looks like all of the processes are stuck waiting on a semaphore > somewhere in namei (.text.lock.namei) and upon reboot i have a bit of > filesystem corruption. I'd be interested in any info from jfs_fsck as to the nature of the filesystem corruption. > Shaggy will have to look into this some more, thanks for the report. Thanks for all your help. Shaggy -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Strange behavior after a crash
On Fri, 2005-01-07 at 17:03 -0600, John Goerzen wrote: > > Actually, jfs does the equivalent of data=writeback. In the 2.4 kernel, > > it does the equivalent of data=ordered, but I changed the behavior do > > avoid a deadlock in 2.6 which I never found the correct fix for. > > You might consider this one vote for the data=ordered behavior. This is > probably why I haven't noticed this problem from ext3 filesystems. Noted. For what it's worth, I haven't personally noticed this kind of corruption, and haven't had reports of it, so I have assumed that in practice it is a kind of rare problem. Maybe you crashed at an unlucky point, or it may be more common than I'm aware of. > > The data=ordered behavior would prevent random data from showing up in a > > file, but it would not prevent the blocks of NULLs that you reported (if > > these are really file holes as I suspect). These would be caused when a > > file written non-sequentially was committed before it was completely > > written to. > > I believe -- but can't prove at the moment -- that these files would > have been written to sequentially. That may be true, and the nulls you saw in the files could have just been the same kind of random data, that happened to be null, due to some past use of the disk blocks. > In any case, if the file was written to non-sequentially, couldn't you > resort to the old data in the file instead of the NULLs? > > And if you mean a sparse file -- where the program seeks past the end, > then writes -- the NULLs do seem to be the proper behavior there. Yes, I did mean sparse files. > > I'll have to look again at implementing the data=ordered behavior. > > That would be great, thanks! > > I wonder -- does anyone know what the other journaling filesystems (XFS, > Reiser3/4, etc.) are doing? > > FWIW, the original reason I switched to JFS was because Reiser was doing > this same thing, just much worse :-) > > -- John -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Strange behavior after a crash
On Fri, 2005-01-07 at 13:49 -0600, John Goerzen wrote: > On Tue, Dec 14, 2004 at 10:39:25AM -0600, Dave Kleikamp wrote: > > On Tue, 2004-12-14 at 09:32 -0600, John Goerzen wrote: > > > Is there any practical way I could try to address this? I would rather > > > have the files truncated, or even re-linked to /lost+found or something, > > > than have them contain bad data. I also never seemed to encounter this > > > behavior with either ext2 or ext3. Was I just lucky, or is there > > > something fundamentally different about JFS? > > > > I'm not sure how I would address this in jfs. I don't know about ext3, > > Well, in ext3 one has some options (mount(8) quoted below). When I > mount it data=ordered, things seem fine. General wisdom holds that this > is the preferred way to go. It sounds like JFS is approximating > data-journal. Actually, jfs does the equivalent of data=writeback. In the 2.4 kernel, it does the equivalent of data=ordered, but I changed the behavior do avoid a deadlock in 2.6 which I never found the correct fix for. The data=ordered behavior would prevent random data from showing up in a file, but it would not prevent the blocks of NULLs that you reported (if these are really file holes as I suspect). These would be caused when a file written non-sequentially was committed before it was completely written to. I'll have to look again at implementing the data=ordered behavior. > >data=journal / data=ordered / data=writeback > Specifies the journalling mode for file data. Metadata is > always journaled. To use modes other than ordered on the root > file system, pass the mode to the kernel as boot parameter, e.g. > rootflags=data=journal. > > journal > All data is committed into the journal prior to being > written into the main file system. > > ordered > This is the default mode. All data is forced directly > out to the main file system prior to its metadata being > committed to the journal. > > writeback > Data ordering is not preserved - data may be written into > the main file system after its metadata has been commit- > ted to the journal. This is rumoured to be the highest- > throughput option. It guarantees internal file system > integrity, however it can allow old data to appear in > files after a crash and journal recovery. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
[Jfs-discussion] Re: [patch 1/1] delete unused file
Accepted. Thank you. On Sun, 2004-12-26 at 16:17 +0100, [EMAIL PROTECTED] wrote: > Remove nowhere referenced file. (egrep "filename\." didn't find anything) > > Signed-off-by: Domen Puncer <[EMAIL PROTECTED]> > --- -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] how to get used/unused space from commandline
On Fri, 2004-12-17 at 22:40 +0100, B.Hakvoort wrote: > Hi ppl, > > I've included support(create,grow,copy) for JFS in gparted > (http://gparted.sourceforge.net) using the CLI tools. > > The only thing i couldn't get done is to read the amount of used/unused > space on a filesystem. I guess this can be done using jfs_debugfs, but i > can't figure out how. You can get it from the "dmap" command in jfs_debugfs. "echo dm | jfs_debugfs /dev/whatever". dn_free is the number of free blocks, and you can get the used blocks by subtracting dn_free from dn_mapsize. Right now, and maybe forever, the block size is always 4K, but it's possible that someday other block sizes will be supported. jfs_debugfs always prints this as "Aggregate Block Size" if you want to play it safe and parse it. > Any insights/hints are appreciated :) > > Bart > > PS. please CC me, since i'm not on this list -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list Jfs-discussion@www-124.ibm.com http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Strange behavior after a crash
On Tue, 2004-12-14 at 09:32 -0600, John Goerzen wrote: > Is there any practical way I could try to address this? I would rather > have the files truncated, or even re-linked to /lost+found or something, > than have them contain bad data. I also never seemed to encounter this > behavior with either ext2 or ext3. Was I just lucky, or is there > something fundamentally different about JFS? I'm not sure how I would address this in jfs. I don't know about ext3, but jfs will create and commit a transaction to create a file as soon as it is opened. What you want is that no transaction gets committed until the file is closed. That would be nearly impossible in jfs since the transaction creating the file modifies the containing directory, and we can't hold up pending changes to the directory if other files are being created before the file is closed. Mounting with -osync would definitely help, but that would really hurt performance. (jfs doesn't fully support the -osync flag, but it will help in the normal write path.) Similarly, tuning /proc/sys/vm/dirty_writeback_centisecs and /proc/sys/vm/dirty_expire_centisecs should reduce the number of files affected, but may adversely affect performance as well. > As for the .so files, I was running apt-get dist-upgrade at the time, so > they were being created/modified at the time of crash; it was just the > magnitude of the problem that was startling, especially given that dpkg > first unpacks things with a temporary filename, then renames them to > their permanent name to try to avoid any corruption in cases like this. > > I don't know if it does a fsync(), though. Actually, I think dpkg does call fsync. I'm not too familiar with debian, but I wonder whether dpkg unpacks the .so files, or if they get dynamically created by ld. I wouldn't expect the files to contain holes if they were simply unpacked and renamed. Are the .so files the only ones with the problem? Shaggy -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Strange behavior after a crash
On Tue, 2004-12-14 at 08:07 -0600, John Goerzen wrote: > Hello, > > I recently had a system crash, and I have experienced some behavior from > JFS on restart that reminds me of why I switched from reiserfs to JFS. > That's not a Good Thing :-) > > 1. Some files contained data that no process could possibly have been > writing to them. It appeared to be blocks of NULLs in some cases. Some > files were .so files, and I lack the expertise to know specifically > which chunks were bad, but I did know that they were corrupt (ldconfig > told me). Some of the files may have contained blocks from other files > also (but I'm not certain of this either). This can normally happen for files that are created shortly before the crash. Blocks of nulls may be file holes. I don't think .so files are created sequentially, so it may be possible that there are portions of the file that had not yet been committed to disk. It's possible that newly-created files may contain stale data, but I don't think this happens often in practice (but I'm not sure). > 2. Some files were truncated. This is not unexpected in a crash > situation, but there were many more files like this than I would have > expected. This would be normal if the files were newly-created. The transaction that creates the file will be committed to disk earlier than the transaction(s) which extend the file when data is written. > 3. The total number of files touched by #1 and #2 far exceeds the number > of files open for writing at the instant the system went down. Files aren't committed when they are closed. pdflush usually makes sure dirty data is committed to disk within 30 seconds. I believe the typical laptop mode configuration changes the pdflush interval to 10 minutes. > fsck didn't seem to really help. Just to make sure there isn't anything abnormal going on, did fsck run through all of it's phases, or just phase 0? Use the -f or -n flags to force it to check everything. > Is this behavior expected from JFS? Is there anything I can do to help > out next time? As long as the affected files were all created or extended within /proc/sys/vm/dirty_writeback_centisecs, this is expected behavior. > I'm running a stock 2.6.6 kernel on this machine. > > -- John -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Any way to undelete files from jfs partition?
On Sat, 2004-12-11 at 21:59 -0500, Matthew Cline wrote: > Hello, > > Basically, I deleted a bunch of files from a JFS partition accidentally, > and I was wondering if there was a way to undelete the data? I did > have backups of the most critical data, but I would like to recover > the other information. > > Once I realized my mistake, I immediately unmounted the JFS partition > in hope that something could be done. I don't have a very detailed > understanding of how the JFS filesystem operates, but I know that some > filesystems have an undelete mechanism. I had some ideas, but I don't > know if any of these are feasible: > > 1) use the JFS journal to roll back all transactions past a certain time > index? It would be possible to do this in many cases, since a lot of the information you need can be reconstructed from the journal. It wouldn't be reliable in all cases, in that you would be able to reconstruct what disk blocks were allocated to a file, but I'm not how easily you could determine the logical file offset of these blocks. That information may still be available in the inode, but I'm not positive. I don't think there's any way to recover the size of the file (other than an estimate, based on where the last extent is). I think the directory entries may be recoverable in most cases, as long as more files weren't added to the directories after the deletes were done. > 2) alter the inodes of deleted files from "deleted" to "not-deleted"? > (I think this is how undelete utilities for other filesystems (such as > ext2) work.) The files are truncated as they are deleted, so only doing this would give you empty files (with no path to them). There might still be enough information in the inodes to reconstruct the xtree, but you'd have to determine the number of extents to try to recover, and wouldn't really have an accurate file size. > Any suggestions would be greatly appreciated. Thank you in advance. This is a year old, so I don't know if there has been any progress on this, but you may want to contact Knut and ask about the status of jfs support in sleuthkit: http://oss.software.ibm.com/pipermail/jfs-discussion/2003-December/001551.html -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Any way to undelete files from jfs partition?
On Tue, 2004-12-14 at 00:21 +0100, Knut Eckstein wrote: > Hello all, > > >> 1) use the JFS journal to roll back all transactions past a certain time > >> index? > > > > It would be possible to do this in many cases, since a lot of the > > information you need can be reconstructed from the journal. It wouldn't be > > reliable in all cases, in that you would be able to reconstruct what disk > > blocks were allocated to a file, but I'm not how easily you could determine > > the logical file offset of these blocks. That information may still be > > available in the inode, but I'm not positive. I don't think there's any > > way to recover the size of the file (other than an estimate, based on where > > the last extent is). > > IMHO, the logical offset of a disk block should be visible, since it is > part of struct xad_t and as such it should be possible to find it > in a journal entry that contains extent allocation changes in an inode. > That same entry should also hold the filesize, since it is a member > of struct jfs_inode. The journal will only contain the parts of the inode that have been modified, so it probably won't contain all of the xad entries, but as long as the inode was not re-used, this metadata is still intact in the inode. I was completely wrong when I stated below that the inode is truncated when it is deleted. > I'm saying 'should' because I have not examined those types of journal > entries yet. I've been focusing on directory entry journal items, see > below. > >> 2) alter the inodes of deleted files from "deleted" to "not-deleted"? (I > >> think this is how undelete utilities for other filesystems (such as ext2) > >> work.) > > > > The files are truncated as they are deleted, so only doing this would give > > you empty files (with no path to them). There might still be enough > > information in the inodes to reconstruct the xtree, but you'd have to > > determine the number of extents to try to recover, and wouldn't really have > > an accurate file size. > > >From what I've seen with my JFS module for sleuthkit, the files are not > >really > truncated, but just marked as deleted, i.e. the file size and the full xtree > (including xtree parts not stored inside the inode) can be seen with the istat > tool. That means that once you know the inode of the deleted file you are > looking for (either by guessing or by looking at journal entries), you can use > icat to undelete that file, if the following three conditions are met: > > 1. The inode you are looking for has not yet been reused > 2. "External" xtree blocks (in case they were required to >describe the exact allocation of that file on disk) have >not yet been overwritten > 3. The disk blocks containing the actual file have > > not yet been >reused for another file or metadata. Yeah, thanks for correcting me. The "truncation" that is done at deletion time only marks the disk blocks as available. It doesn't alter the inode (other than di_nlink) or xtree structure. Thank you, Knut. I haven't read your paper yet, but I will. Shaggy -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Partition corruption, help!!
On Wed, 2004-12-08 at 08:20 -0600, Dave Kleikamp wrote:
> Here is a patch that should get you by this specific error,
I bet you'd rather have a patch that compiles. Apply this one instead.
--
David Kleikamp
IBM Linux Technology Center
Index: jfsutils/fsck/fsckmeta.c
===
RCS file: /usr/cvs/jfs/jfsutils/fsck/fsckmeta.c,v
retrieving revision 1.19
diff -u -p -r1.19 fsckmeta.c
--- jfsutils/fsck/fsckmeta.c 24 Sep 2004 14:43:53 - 1.19
+++ jfsutils/fsck/fsckmeta.c 8 Dec 2004 14:27:45 -
@@ -1370,8 +1370,9 @@ int validate_fs_metadata()
intermed_rc = inode_get(aggregate_inode, which_fsit, ino_idx, &ino_ptr);
if (intermed_rc != FSCK_OK) {
/* can't get the inode */
- vfm_rc = FSCK_CANTREADFSEXT;
- goto vfm_exit;
+ //vfm_rc = FSCK_CANTREADFSEXT;
+ //goto vfm_exit;
+ goto read_root; /* Who really cares? */
}
/* got superinode extension inode */
@@ -1379,14 +1380,16 @@ int validate_fs_metadata()
intermed_rc = verify_fs_super_ext(ino_ptr, msg_info_ptr,&inode_updated);
if (intermed_rc < 0) {
/* something really really bad happened */
- vfm_rc = intermed_rc;
- goto vfm_exit;
+ //vfm_rc = intermed_rc;
+ //goto vfm_exit;
+ goto read_root;
}
if (intermed_rc != FSCK_OK) {
/* inode is bad */
- vfm_rc = FSCK_FSETEXTBAD;
- goto vfm_exit;
+ //vfm_rc = FSCK_FSETEXTBAD;
+ //goto vfm_exit;
+ goto read_root;
}
/* superinode extension inode is ok */
@@ -1394,9 +1397,9 @@ int validate_fs_metadata()
/* need to write the superinode extension */
vfm_rc = inode_put(ino_ptr);
}
- if (vfm_rc != FSCK_OK)
- goto vfm_exit;
-
+ //if (vfm_rc != FSCK_OK)
+ // goto vfm_exit;
+read_root:
/* read the root directory inode */
ino_idx = ROOT_I;
intermed_rc = inode_get(aggregate_inode, which_fsit, ino_idx,
Re: [Jfs-discussion] Partition corruption, help!!
On Tue, 2004-12-07 at 23:59 -0800, Sang Nguyen Van wrote:
> processing terminated: 12/8/2004 8:49:47 with return
> code: 10052 exit code: 8. [xchkdsk.c:472]
Here is a patch that should get you by this specific error, but I'm not
sure how much further you'll get. This is a failure to read an inode
that isn't used. fsck shouldn't give up here. But with both the
journal superblock corrupt, and fsck unable to read this inode, I have a
suspicion that fsck won't get a whole lot farther.
If you can't get fsck to finish, you may try to mount the partition
read-only (mount -oro). If that succeeds, you may be able to recover
some data.
> About the syslog, before we installed the new SuSe
> 9.2, my administrator checked the system and said
> there was an error message like the system is trying
> to write to the area out of the hardisk space (?!).
> I have a question: the problem comes from my old SuSe
> 7.3 (old kernel), and so the jfs handler or something
> else? Eventhough there may not be much hope for
> recovering the partition, I just don't want it to
> occur again!
I'm not sure what happened, but jfs is much more stable in SuSe 9.2 than
in 7.3.
> Thanks,
> Sang Nguyen.
Shaggy
--
David Kleikamp
IBM Linux Technology Center
Index: jfsutils/fsck/fsckmeta.c
===
RCS file: /usr/cvs/jfs/jfsutils/fsck/fsckmeta.c,v
retrieving revision 1.19
diff -u -p -r1.19 fsckmeta.c
--- jfsutils/fsck/fsckmeta.c 24 Sep 2004 14:43:53 - 1.19
+++ jfsutils/fsck/fsckmeta.c 8 Dec 2004 14:12:25 -
@@ -1370,8 +1370,9 @@ int validate_fs_metadata()
intermed_rc = inode_get(aggregate_inode, which_fsit, ino_idx, &ino_ptr);
if (intermed_rc != FSCK_OK) {
/* can't get the inode */
- vfm_rc = FSCK_CANTREADFSEXT;
- goto vfm_exit;
+ //vfm_rc = FSCK_CANTREADFSEXT;
+ //goto vfm_exit;
+ goto read_root; /* Who really cares? */
}
/* got superinode extension inode */
@@ -1379,14 +1380,16 @@ int validate_fs_metadata()
intermed_rc = verify_fs_super_ext(ino_ptr, msg_info_ptr,&inode_updated);
if (intermed_rc < 0) {
/* something really really bad happened */
- vfm_rc = intermed_rc;
- goto vfm_exit;
+ //vfm_rc = intermed_rc;
+ //goto vfm_exit;
+ goto read_root;
}
if (intermed_rc != FSCK_OK) {
/* inode is bad */
- vfm_rc = FSCK_FSETEXTBAD;
- goto vfm_exit;
+ //vfm_rc = FSCK_FSETEXTBAD;
+ //goto vfm_exit;
+ goto read_root;
}
/* superinode extension inode is ok */
@@ -1394,8 +1397,8 @@ int validate_fs_metadata()
/* need to write the superinode extension */
vfm_rc = inode_put(ino_ptr);
}
- if (vfm_rc != FSCK_OK)
- goto vfm_exit;
+ //if (vfm_rc != FSCK_OK)
+ // goto vfm_exit;
/* read the root directory inode */
ino_idx = ROOT_I;
Re: [Jfs-discussion] Partition corruption, help!!
On Mon, 2004-12-06 at 23:56 -0800, Sang Nguyen Van wrote: > --- Dave Kleikamp <[EMAIL PROTECTED]> wrote: > > Okay, I've seen this before and this patch to > > jfsutils should get fsck > > past phase 1. Could you please give it a try? > > > I've just built and installed the new jfsutils with > the patches in those 3 files which you pointed out. > The results of jfs_fsck seems to puzzle me, with > jfs_fsck /dev/hdb1 and jfs_fsck -f /dev/hdb1 I have > the same message: > The current device is: /dev/hdb1 > Block size in bytes: 4096 > Filesystem size in blocks: 40019915 > **Phase 0 - Replay Journal Log > logredo failed (rc=-268). fsck continuing. This is not good. -268 indicates that fsck did not recognize the journal's superblock. > **Phase 1 - Check Blocks, Files/Directories, and > Directory Entries > > It stopped here, without any message telling "the > system is clean" as before. Any suggestion? Hmm, this is what the patch was supposed to fix. Could you try running fsck with the -d flag (debug)? > Thanks, > Sang Nguyen Thanks, Shaggy -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] performance probs - 2.4.28, jsf117, raid5
On Mon, 2004-12-06 at 19:01 +0100, Per Jessen wrote: > OK, I can tell inode_cache is using up a lot here. Apart from using > a multi-level subdir structure for my 500.000 files, is there anything > else I can tweak to assist the process? find is going to bring all of the inodes into memory regardless of whether or not they are split between multiple directories or not. Splitting them up will only help if you only run find against a subset of the subdirs. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Partition corruption, help!!
On Mon, 2004-12-06 at 07:26 -0800, Sang Nguyen Van wrote:
> ... # jfs_fsck /dev/hdb1
> jfs_fsck version 1.1.7, 22-Jul-2004
> processing started: 12/6/2004 16.17.41
> Using default parameter: -p
> The current device is: /dev/hdb1
> Block size in bytes: 4096
> Filesystem size in blocks: 40019915
> **Phase 0 - Replay Journal Log
> logredo failed (rc=-268). fsck continuing.
> **Phase 1 - Check Blocks, Files/Directories, and
> Directory Entries
> Filesystem is clean.
> I could not get to phase 2,... of jfs_fsck after many
> tries, and could not mount the 2nd harddisk even after
> trying various option of mount command: -o
> errors=continue ..., still the message:
> mount: wrong fs type, bad option, bad superblock on
> /dev/hdb1,
>or too many mounted file systems
Okay, I've seen this before and this patch to jfsutils should get fsck
past phase 1. Could you please give it a try?
Thanks,
Shaggy
--
David Kleikamp
IBM Linux Technology Center
Index: jfsutils/fsck/fsckmeta.c
===
RCS file: /usr/cvs/jfs/jfsutils/fsck/fsckmeta.c,v
retrieving revision 1.18
retrieving revision 1.19
diff -u -p -r1.18 -r1.19
--- jfsutils/fsck/fsckmeta.c 17 Dec 2003 20:28:47 - 1.18
+++ jfsutils/fsck/fsckmeta.c 24 Sep 2004 14:43:53 - 1.19
@@ -3322,6 +3322,7 @@ int verify_fs_super_ext(struct dinode *i
if (inorecptr->ignore_alloc_blks
|| (vfse_rc != FSCK_OK)) {
inode_invalid = -1;
+vfse_rc = FSCK_OK;
}
}
}
Index: jfsutils/fsck/xchkdsk.c
===
RCS file: /usr/cvs/jfs/jfsutils/fsck/xchkdsk.c,v
retrieving revision 1.48
retrieving revision 1.49
diff -u -p -r1.48 -r1.49
--- jfsutils/fsck/xchkdsk.c 22 Jul 2004 14:29:37 - 1.48
+++ jfsutils/fsck/xchkdsk.c 24 Sep 2004 14:43:53 - 1.49
@@ -2103,10 +2103,6 @@ int phase1_processing()
if (p1_rc != FSCK_OK) {
agg_recptr->fsck_is_done = 1;
exit_value = FSCK_OP_ERROR;
- if (p1_rc > 0) {
- /* this isn't a fsck failure */
- p1_rc = 0;
- }
}
return (p1_rc);
}
Index: jfsutils/libfs/logredo.c
===
RCS file: /usr/cvs/jfs/jfsutils/libfs/logredo.c,v
retrieving revision 1.26
retrieving revision 1.27
diff -u -p -r1.26 -r1.27
--- jfsutils/libfs/logredo.c 29 Jun 2004 19:36:53 - 1.26
+++ jfsutils/libfs/logredo.c 24 Sep 2004 13:33:58 - 1.27
@@ -791,14 +791,9 @@ int jfs_logredo(caddr_t pathname, int32_
if (vopen[k].state != VOPEN_OPEN)
continue;
- /* don't update the maps if the aggregate/lv is
- * FM_DIRTY since fsck will rebuild maps anyway
- */
- if (!vopen[k].is_fsdirty) {
- if ((rc = updateMaps(k)) != 0) {
-fsck_send_msg(lrdo_ERRORCANTUPDMAPS);
-goto error_out;
- }
+ if ((rc = updateMaps(k)) != 0) {
+ fsck_send_msg(lrdo_ERRORCANTUPDMAPS);
+ goto error_out;
}
/* Make sure all changes are committed to disk before we
Re: [Jfs-discussion] FS benchmarks with recent 2.6 kernel
jfs does have some performance issues on disks that don't do write-caching. dbench really seems to illustrate this problem. I'm working with our performance team to try to come up with some improvements. We're hoping to have something ready before the holidays. Thanks, Shaggy On Fri, 2004-12-03 at 10:14 +0100, Bjoern JACKE wrote: > Hi Shaggy, > > Tridge did some EA filesystem benchmarks to find out about good > filesystems for Samba4. You can find the results on > http://samba.org/~tridge/xattr_results/. JFS performs very poor here > with 2.6.10rc kernel; I can remember ACL fs benchmarks from Andreas > Grünbacher, which showed quite good performance for JFS with 2.4.x > kernel. Maybe you want to have a look at it and see why JFS is so poor > here. > > Bjoern -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Re: Lost Partition -> Recovery problem !
On Sun, 2004-11-21 at 22:57 +0100, Gilles Meier wrote: > I always have a problem... > > If I recuper the partition with parted, the partition go from > 0.000 to 238433.211, which gives me the sane superblock. > When I create a new partition, taking the whole disk, it go from > 0.031 to 238472.688 and have corrupted superblocks :'( > > I think I'll format my disk again tomorrow... Will fdisk let you create a partition beginning at 0.000? What did you use originally to create the partition? Just to exhaust every possibility, have you tried to mount /dev/hdd? -- Dave Kleikamp IBM Linux Technology Center [EMAIL PROTECTED] ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Re: Lost Partition -> Recovery problem !
On Sun, 2004-11-21 at 11:57 +0100, Gilles Meier wrote: > Tkanks both of you... > > With using the first proposal from parted, both superblocks seem to be > sane (identical to the first superblock from the other propasal)... > But there's always this FS size problem. > > Parted refuse to modify the size of a JFS filesystem :( > > May jfs_fsck say that superblocks are corrupted "only" if the size of > the FS don't match with the size of partition ? Yes, it would do this. Running with the -v flag (verbose) should give you a better error message. It should probably do this without the -v flag. I'll add that to my to-do list. > Or is there another problem ? I hope not. > If I've fine understood, shrinking a JFS filesystem size is impossible, > so the only way is to grow the partition size from 238433.21Mb to > 238472.688 Mb. Parted says me that opening a JFS partition isn't yet > implemented, so does someone know how to perform this operation ? Raw > writing in the table ? You need to delete the the partition, then create a new partition filling the whole disk (rebooting afterward), and you should be okay. -- Dave Kleikamp IBM Linux Technology Center [EMAIL PROTECTED] ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Re: Lost Partition -> Recovery problem !
On Sat, 2004-11-20 at 15:32 -0600, Dave Kleikamp wrote: > On Sat, 2004-11-20 at 22:02 +0100, Gilles Meier wrote: > > Nothing to do, always the same error :( > > > > Isn't there some method to "repair" the superblocks with jfs_debug or > > some other utility ? > > I would guess that if the partition table could be fixed, there > shouldn't be a need to repair the superblocks. I'm not sure that they > are being read from the right location. I would hope that nothing > overwrote both superblocks. If something had, the file system may be > beyond recovery. > > > > If I've all understood, the only thing who was messed up is the > > partition table, so datas are always on the hdd... I don't have any idea > > on how information are organised in partition table, but can't I > > discover data location with the journal or something else ? > > > (parted) rescue > > Start? 0.000 > > End? 238475.179 > > Information: A jfs primary partition was found at 0.000Mb -> > > 238433.211Mb. Do you want to add it to the partition table? > > Yes/No/Cancel? N > > My guess is that this is a remnant of running mkfs.jfs against /dev/hdd. > I would ingore this one. After some more thought, and with the results of jfs_debugfs (and Michaels comments), I'm thinking that this is the right partition to use. Some calculations confirm that the secondary superblock is stored .028 Mb from the primary, so the second partition reported by parted is based on interpreting the secondary superblock as the primary. I was assuming that the partition table of a partitioned disk is stored at 0.000Mb, but to be honest, I really don't know where the partition table is stored. Again, based on what I saw in the superblock, the partition needs to be a bit larger than parted is reporting for the superblock be valid. > > Information: A jfs primary partition was found at 0.028Mb -> > > 238433.238Mb. Do you want to add it to the partition table? > > Yes/No/Cancel? N > > This is probably the correct location. Did you try to rescue the disk > using this partition, and then reboot? If you did, and still get the > error from fsck, run jfs_debugfs /dev/hdd1, then the subcommands "sup" > and "sup s". -- Dave Kleikamp IBM Linux Technology Center [EMAIL PROTECTED] ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Re: Lost Partition -> Recovery problem !
On Sat, 2004-11-20 at 22:47 +0100, Gilles Meier wrote: > I still got the same errors, here are the output from jfs_debug : > > sup : > > > sup > > sup > [1] s_magic:'JFS1' [15] s_ait2.addr1: 0x00 > [2] s_version: 1[16] s_ait2.addr2: 0x1d3a > [3] s_size: 0x1d1b09b0s_ait2.address:7482 > [4] s_bsize:4096 [17] s_logdev: 0x > [5] s_l2bsize: 12 [18] s_logserial: 0x > [6] s_l2bfactor:3[19] s_logpxd.len: 8192 > [7] s_pbsize: 512 [20] s_logpxd.addr1:0x00 > [8] s_l2pbsize: 9[21] s_logpxd.addr2:0x03a368b0 > [9] pad:Not Displayed s_logpxd.address: 61040816 > [10] s_agsize: 0x0008 [22] s_fsckpxd.len: 1914 > [11] s_flag:0x10200900 [23] s_fsckpxd.addr1: 0x00 > JFS_LINUX[24] s_fsckpxd.addr2: 0x03a36136 > JFS_COMMITJFS_GROUPCOMMIT s_fsckpxd.address: 61038902 > JFS_INLINELOG [25] s_time.tv_sec: 0x419cc59c > [26] s_time.tv_nsec:0x > [27] s_fpack: '' > [12] s_state: 0x > FM_CLEAN > [13] s_compress:0 > [14] s_ait2.len:4 This looks sane, but the size of the partition needed would be 238472.6875 MB. ((Address of logpxd * length of logpxd) * 4K) The output from parted indicates that the size of the partition would be 238433.21 MB. There isn't an easy way to shrink the file system by 40 MB, but you may be able to extend the partition using parted. > > sup s > [1] s_magic:'' [15] s_ait2.addr1:0xff > [2] s_version: 0[16] s_ait2.addr2:0x > [3] s_size: 0x4000s_ait2.address: 1099511627775 > [4] s_bsize:256 [17] s_logdev:0x > [5] s_l2bsize: 8[18] s_logserial: 0x > [6] s_l2bfactor:0[19] s_logpxd.len: 16777215 > [7] s_pbsize: 85 [20] s_logpxd.addr1:0xff > [8] s_l2pbsize: 4[21] s_logpxd.addr2:0x > [9] pad:Not Displayed s_logpxd.address:1099511627775 > [10] s_agsize: 0xff05 [22] s_fsckpxd.len: 16777215 > [11] s_flag:0x [23] s_fsckpxd.addr1: 0xff >JFS_OS2 JFS_LINUX [24] s_fsckpxd.addr2: 0x >JFS_COMMIT JFS_GROUPCOMMIT s_fsckpxd.address:1099511627775 > JFS_LAZYCOMMIT JFS_INLINELOG[25] s_time.tv_sec: 0x > JFS_BAD_SAITJFS_SPARSE [26] s_time.tv_nsec:0x > DASD_ENABLEDDASD_PRIME [27] s_fpack: > 'ïïï' > [12] s_state: 0x > Unknown State > [13] s_compress: -1 > [14] s_ait2.len:16777215 This does look bad, which can mean that the file system is further damaged. But if the primary superblock is good, the secondary isn't looked at. -- Dave Kleikamp IBM Linux Technology Center [EMAIL PROTECTED] ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Re: Lost Partition -> Recovery problem !
On Sat, 2004-11-20 at 22:02 +0100, Gilles Meier wrote: > Nothing to do, always the same error :( > > Isn't there some method to "repair" the superblocks with jfs_debug or > some other utility ? I would guess that if the partition table could be fixed, there shouldn't be a need to repair the superblocks. I'm not sure that they are being read from the right location. I would hope that nothing overwrote both superblocks. If something had, the file system may be beyond recovery. > > If I've all understood, the only thing who was messed up is the > partition table, so datas are always on the hdd... I don't have any idea > on how information are organised in partition table, but can't I > discover data location with the journal or something else ? > (parted) rescue > Start? 0.000 > End? 238475.179 > Information: A jfs primary partition was found at 0.000Mb -> > 238433.211Mb. Do you want to add it to the partition table? > Yes/No/Cancel? N My guess is that this is a remnant of running mkfs.jfs against /dev/hdd. I would ingore this one. > Information: A jfs primary partition was found at 0.028Mb -> > 238433.238Mb. Do you want to add it to the partition table? > Yes/No/Cancel? N This is probably the correct location. Did you try to rescue the disk using this partition, and then reboot? If you did, and still get the error from fsck, run jfs_debugfs /dev/hdd1, then the subcommands "sup" and "sup s". > Tanks a lot for your help, it's good to talk with kind people :) I try to help out when I can. I'm not sure if my help is going to do you any good though, but we can try. -- Dave Kleikamp IBM Linux Technology Center [EMAIL PROTECTED] ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Re: Lost Partition -> Recovery problem !
On Sat, 2004-11-20 at 21:24 +0100, Gilles Meier wrote: > Sadly, I've putted all my datas on this new hdd just after formatting it > :( So, it's very important for me to rescue them... > > Next time I'll always reboot and test the new partition before putting > someting on it ;) If there is still anything to recover, you should reboot between recovering the partitions with fdisk/parted and trying to use the partitions. It sounds like it might be too late so save anything, though. -- Dave Kleikamp IBM Linux Technology Center [EMAIL PROTECTED] ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Lost Partition -> Recovery problem !
On Sat, 2004-11-20 at 20:40 +0100, Gilles Meier wrote: > Hi everybody, > > I've some problems with a JFS Partition. > Yesterday I have bought a new HDD (250 Go). > After the installation, I've created one partition, and then I have > formatted it with JFS. > Sadly, I've made a mistake : I've typed `mkfs.jfs /dev/hdd`. > Then I launch fdisk again, the partition were still here, so I think > there was no problem and formatted my hdd with the right commande > (`mkfs.jfs /dev/hdd1`). Everything goes fine, I moved my data on this > new drive. > But after the restart, the partition has disapeared. You can mess up the partition table on disk, and the kernel won't be aware of the changes until the system is rebooted. > I've tried rescue mode with fdisk, but nothing. > Then, I try parted, and he says me : > > (parted) print > Disk geometry for /dev/hdd: 0.000-238475.179 megabytes > Disk label type: msdos > MinorStart End Type Filesystem Flags > (parted) rescue > Start? 0.000 > End? 238475.179 > Information: A jfs primary partition was found at 0.000Mb -> > 238433.211Mb. Do you want to add it to the partition table? > Yes/No/Cancel? N > Information: A jfs primary partition was found at 0.028Mb -> > 238433.238Mb. Do you want to add it to the partition table? > Yes/No/Cancel? N > (parted) > > But nor the first, nor the seconde gave results... > When I try to create them, and then run fsck.jfs, he complains : > > $ fsck.jfs /dev/hdd1 > fsck.jfs version 1.1.6, 28-Apr-2004 > processing started: 11/20/2004 14.47.52 > Using default parameter: -p > The current device is: /dev/hdd1 > Superblock is corrupt and cannot be repaired > since both primary and secondary copies are corrupt. > > CANNOT CONTINUE. > > I'm new to JFS and don't know what to try anymore, so if someone can > help me, many thanks :) As this is a new disk, it sounds like you don't have any data that needs to be preserved on this disk. I think you would do best by using fdisk or parted to completely remove any existing partitions and start over, rather than trying to preserve what is there. Reboot after creating any new partitions before trying to use them. > PS : sorry for my poor english... I understood you perfectly. :^) Good luck, Shaggy -- Dave Kleikamp IBM Linux Technology Center [EMAIL PROTECTED] ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] fsck taking very very long time
On Sat, 2004-11-20 at 10:17 -0500, Mike Kincer wrote: > I have a 4x36 GB disk SCSI RAID5 (AMI Megaraid) drive under Fedora Core 2. > One drive crashed. Replaced the drive. Low level megaraid BIOS indicated > replication was good. > Booted Linux, and indicated partition was corrupt. > It did mount, and looks like it sent 75% of the files to lost+found > > Running jfs_fsck -d -v /dev/sda1 > I say running since it is still running after 7 days! > It is "working" and not stuck in a loop as the messages (many many > messages) are still updating. > Wondering why it is taking so long... ideas? If your jfsutils is older than version 1.1.5, there is code that does duplicate block checking that runs very, very slowly. This code isn't run often, but when it is, it's bad. This was fixed in version 1.1.5, so I would suggest getting a newer version. > Can I eventually retrieve files from lost+found? Yes, but it might not be easy. If you are lucky, you'll find subdirectories in lost+found that will contain their contents intact. Individual files may be harder to identify, as they no longer have their pathnames associated with them. -- Dave Kleikamp IBM Linux Technology Center [EMAIL PROTECTED] ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] JFS mkfs problem
On Mon, 2004-11-01 at 15:30, Richard Allen wrote:
> On Mon, Nov 01, 2004 at 09:01:11PM +, Richard Allen wrote:
> > On Mon, Nov 01, 2004 at 08:39:58PM +, Richard Allen wrote:
> >
> > > Continue? (Y/N) Y
> > > The specified disk did not finish formatting.
> > > Internal error3: Format failed rc=
> >
> > Running mkfs.jfs in strace gives:
> >
> > write(3, "!Ce\207\1\0\0\0\0\0\0\0\0 \0\0\0\20\0\0\f\0\0\0\0\t \20"..., 4096) = -1
> > EFAULT (Bad address)
>
> I'm sorry if I'm mailbombing you guys.But I think I found the problem.
> I traced the problem to ujfs_rw_diskblocks() in libfs/devices.c
> EFAULT in write() means the buffer to write is outside my accessible address
> space.
>
> The function jfs_logform() has a structure defined called log_sup of type
> logsuper and it's defined as static. It's then used as argument 4 to the
> function ujfs_rw_diskblocks() which is located in another file.
The problem is that struct logsuper is smaller than 4K, and we are
writing 4K from that structure. I'm not sure why you're the first to
report the problem. Maybe it's a newer compiler or linker.
> rc = ujfs_rw_diskblocks(fd, (log_begin + LOGPSIZE),
> (unsigned) LOGPSIZE, (char *) &log_sup, PUT);
>
> in ujfs_rw_diskblocks argument 4 becomes void *data_buffer which is used
> in read and write functions.
>
> case PUT:
> Bytes_Transferred = write(dev_ptr, data_buffer, disk_count);
> break;
>
> After making the structure non static, it formats perfectly:
There's still a problem with this. For an external journal, we are
going to read 4K into logsup to verify that the uuid matches, and this
will cause other data on the stack to be overwritten. With the static
declaration, somehow we've been lucky and not run into a problem before
(at least one that has been reported). Even if we're not reading, we
will still be writing some random stack contents to the disk after the
superblock, which is not so good.
The right fix would be to either pad (and zero) struct jfslog to be 4K
bytes long, or to change jfslog to a pointer and malloc a 4K buffer.
This patch does a calloc instead of declaring the structure statically.
Does it fix your problem?
? jfsutils/mkfs/.initmap.c.swp
Index: jfsutils/libfs/logform.c
===
RCS file: /usr/cvs/jfs/jfsutils/libfs/logform.c,v
retrieving revision 1.15
diff -u -p -u -r1.15 logform.c
--- jfsutils/libfs/logform.c23 Sep 2002 21:07:43 - 1.15
+++ jfsutils/libfs/logform.c1 Nov 2004 22:24:44 -
@@ -63,7 +63,7 @@ int jfs_logform(int fd, /* this is a fi
int npages, rc, k;
char logpages[4 * LOGPSIZE];
struct logpage *logp; /* array of 4 log pages */
- static struct logsuper log_sup;
+ static struct logsuper *log_sup;
struct lrd *lrd_ptr;
int64_t log_begin; /* the location of the beginning of the log
* inside of the file system. ( in bytes )
@@ -73,6 +73,12 @@ int jfs_logform(int fd, /* this is a fi
int Working_counter;
char *Working[5];
+ log_sup = calloc(1, LOGPSIZE);
+ if (log_sup == NULL) {
+ printf("logform: calloc failed!\n");
+ return -1;
+ }
+
/* find the log superblock location
*/
log_begin = log_start << s_l2bsize;
@@ -106,12 +112,12 @@ int jfs_logform(int fd, /* this is a fi
else {
/* Verify existing uuid matches uuid passed in */
rc = ujfs_rw_diskblocks(fd, (log_begin + LOGPSIZE),
- (unsigned) LOGPSIZE, (char *)
&log_sup, GET);
+ (unsigned) LOGPSIZE, (char *) log_sup,
GET);
if (rc)
return -1;
- ujfs_swap_logsuper(&log_sup);
+ ujfs_swap_logsuper(log_sup);
- if (!uuid_compare(log_sup.uuid, uuid)) {
+ if (!uuid_compare(log_sup->uuid, uuid)) {
printf("Invalid log device\n");
return -1;
}
@@ -125,36 +131,36 @@ int jfs_logform(int fd, /* this is a fi
/*
* init log superblock: log page 1
*/
- log_sup.magic = LOGMAGIC;
- log_sup.version = LOGVERSION;
- log_sup.state = LOGREDONE;
+ log_sup->magic = LOGMAGIC;
+ log_sup->version = LOGVERSION;
+ log_sup->state = LOGREDONE;
/* Assign fs s_flag to log superblock.
* Currently s_flag carries the inlinelog info and commit option
* ( i.e. group commit or lazy commit, etc.. )
*/
- log_sup.flag = s_flag;
- log_sup.size = npages;
- log_sup.bsize = ag
[Jfs-discussion] Re: [2.6 patch] JFS: make some symbols static
On Sat, 2004-10-30 at 02:23, Adrian Bunk wrote: > The patch below makes some JFS symbols that are only used inside the > file they are defined in static. > > ... Looks sane. I'll apply this and push it to Andrew. Thanks, Shaggy -- Dave Kleikamp IBM Linux Technology Center [EMAIL PROTECTED] ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
[Jfs-discussion] Re: [2.6 patch] jfs_metapage.c: remove an unused function
On Thu, 2004-10-28 at 19:18, Adrian Bunk wrote:
> The patch below removes an unused function from fs/jfs/jfs_metapage.c
>
> --- patch removed ---
I guess I had intended to call alloc_metapage when I wrote that code,
but I never did. This must have been my intent:
diff -urp linux.orig/fs/jfs/jfs_metapage.c linux/fs/jfs/jfs_metapage.c
--- linux.orig/fs/jfs/jfs_metapage.c2004-10-29 08:55:16.670499440 -0500
+++ linux/fs/jfs/jfs_metapage.c 2004-10-29 08:56:35.603499800 -0500
@@ -289,7 +289,7 @@ again:
*/
mp = NULL;
if (JFS_IP(inode)->fileset == AGGREGATE_I) {
- mp = mempool_alloc(metapage_mempool, GFP_ATOMIC);
+ mp = alloc_metapage(1);
if (!mp) {
/*
* mempool is supposed to protect us from
@@ -306,7 +306,7 @@ again:
struct metapage *mp2;
spin_unlock(&meta_lock);
- mp = mempool_alloc(metapage_mempool, GFP_NOFS);
+ mp = alloc_metapage(0);
spin_lock(&meta_lock);
/* we dropped the meta_lock, we need to search the
It seems cleaner to call alloc_metapage, since we have a corresponding
free_metapage, but this version is less cryptic:
diff -urp linux.orig/fs/jfs/jfs_metapage.c linux/fs/jfs/jfs_metapage.c
--- linux.orig/fs/jfs/jfs_metapage.c2004-10-29 08:55:16.670499440 -0500
+++ linux/fs/jfs/jfs_metapage.c 2004-10-29 09:01:24.560571648 -0500
@@ -108,9 +108,9 @@ static void init_once(void *foo, kmem_ca
}
}
-static inline struct metapage *alloc_metapage(int no_wait)
+static inline struct metapage *alloc_metapage(int gfp_mask)
{
- return mempool_alloc(metapage_mempool, no_wait ? GFP_ATOMIC : GFP_NOFS);
+ return mempool_alloc(metapage_mempool, gfp_mask);
}
static inline void free_metapage(struct metapage *mp)
@@ -289,7 +289,7 @@ again:
*/
mp = NULL;
if (JFS_IP(inode)->fileset == AGGREGATE_I) {
- mp = mempool_alloc(metapage_mempool, GFP_ATOMIC);
+ mp = alloc_metapage(GFP_ATOMIC);
if (!mp) {
/*
* mempool is supposed to protect us from
@@ -306,7 +306,7 @@ again:
struct metapage *mp2;
spin_unlock(&meta_lock);
- mp = mempool_alloc(metapage_mempool, GFP_NOFS);
+ mp = alloc_metapage(GFP_NOFS);
spin_lock(&meta_lock);
/* we dropped the meta_lock, we need to search the
--
Dave Kleikamp
IBM Linux Technology Center
[EMAIL PROTECTED]
___
Jfs-discussion mailing list
[EMAIL PROTECTED]
http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] fsck.jfs segfaults in phase 5
On Tue, 2004-10-26 at 13:07, Kurt Albershardt wrote:
> Fedora Core 1
> 2.4.22-1.2199.nptl
> jfs 1.1.3
>
> System hangs during startup. Minor glitch in root FS was easily repaired with
> fsck.jfs but I get a segfault when checking /dev/hda2:
>
> Phase 4 completes after a bunch of "cannot repair the data format error in this
> file, will release"
Hmm. This shouldn't happen, but Redhat has never been very good about
keeping up with the latest jfs kernel code. jfs is much more stable now
than in the 2.4.22 kernel.
> Phase 5 first event says: filesystem object df299783 is linked as {snip long path}
> segmentation fault
> #
This sounds like a bug that was fixed in jfsutils-1.1.5. I would
recommend upgrading to jfsutils-1.1.7.
--
Dave Kleikamp
IBM Linux Technology Center
[EMAIL PROTECTED]
___
Jfs-discussion mailing list
[EMAIL PROTECTED]
http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Scary syslog messages from JFS
On Fri, 2004-10-22 at 16:39, Raymond Prach wrote: > Are these messages simply the result of the filesystem being full? Yeah, that's all it is. These warnings have been removed from the 2.6 kernel, but they are still in 2.4. I'll get rid of them. > Wouldn't one expect to see a more traditional "no space left on device" > sort of message instead? You should see this from user-space. There really is no need to write anything to the syslog. > Or does this indicate something more dire? I ran > jfs_fsck -f on the affected filesystem and it found no errors. > > Any information you could provide would be greatly appreciated. > > Thanks! Thanks, Shaggy -- Dave Kleikamp IBM Linux Technology Center [EMAIL PROTECTED] ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Corrupted HD partition
Quick answer (I'm about to go get lunch) is to try to mount the drive read-only and copy off the data. I'll look at the fsck code when I get back from lunch to see if there is something we can work around to make fsck fix it right. On Tue, 2004-10-19 at 08:49, Matthew Cline wrote: > After a power failure at home, I have a hard drive partition > (/dev/hda4) that will not mount. I get the following errors from fsck: > > # jfs_fsck -v -f /dev/hda4 > jfs_fsck version 1.1.4, 30-Oct-2003 > processing started: 10/19/2004 9.12.23 > The current device is: /dev/hda4 > Open(...READ/WRITE EXCLUSIVE...) returned rc = 0 > Primary superblock is valid. > The type of file system for the device is JFS. > Block size in bytes: 4096 > Filesystem size in blocks: 42198738 > **Phase 0 - Replay Journal Log > LOGREDO: Log already redone! > logredo returned rc = 0 > **Phase 1 - Check Blocks, Files/Directories, and Directory Entries > Unrecoverable error reading M from /dev/hda4. CANNOT CONTINUE. > Fatal error (-10015,30) accessing the filesystem (1,133353553920,16384,0). > processing terminated: 10/19/2004 9:12:39 with return code: -10015 > exit code: 8. > > Is there any hope of recovering the data from this partition? > > Thanks, > > Matt > ___ > Jfs-discussion mailing list > [EMAIL PROTECTED] > http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] stack overrun in dtSearch!
On Thu, 2004-09-30 at 02:08, Per Jessen wrote: > I have a 300Gb JFS drive used for backups only. A couple of days ago I noticed that > it had turned read-only, > and started researching it. I found the following in the log: > > Sep 24 03:00:59 jupiter kernel: ERROR: (device ide2(33,65)): stack overrun in > dtSearch! > Sep 24 03:00:59 jupiter last message repeated 696 times > Sep 25 03:00:56 jupiter kernel: ERROR: (device ide2(33,65)): stack overrun in > dtSearch! > Sep 25 03:01:06 jupiter last message repeated 2198 times > Sep 25 03:01:06 jupiter kernel: ERROR: (device ide2(33,65)): stack overrun in > dtSearch! > Sep 25 03:01:15 jupiter last message repeated 748 times > > The situation and the symptoms look very similar to what I reported here: > > http://www.mail-archive.com/[EMAIL PROTECTED]/msg01156.html > > This time it was kernel 2.4.26 and jfs-1.1.6. An fsck cleared the read-only > condition. 2.4.27 has some bug fixes that may fix this problem. If 2.4.27 doesn't fix it, and you see the problem again, the kernel will now print out more information to diagnose the nature of the dtree corruption. There was a specific bug where an out-of-space condition triggered this problem that is fixed in 2.4.27. Is there any chance this drive had been completely filled up? > regards, > Per Jessen -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Kernel error on IA64
On Tue, 2004-09-28 at 11:24, John Goerzen wrote: > Hmm. I've been considering using JFS on Alpha, which has had an 8K page > size by default for just about forever (since at least linux 2.2 IIRC, > maybe longer), and I believe that cannot be reduced in the kernel. Are > you saying it will not work? I'm afraid not. I'll try to put a little more priority into getting it to work with larger page sizes. Thanks for your input, Shaggy -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Kernel error on IA64
On Tue, 2004-09-28 at 11:06, Richard Allen wrote: > Is this a bug or something Im doing wrong ? It's a bug. jfs is broken if the page size in the kernel is not 4K bytes. The fix involves some design changes in the way jfs handles the metadata, so it's not going to be a quick fix. You'll either have to recompile the kernel with a 4K page size, or use another file system. Sorry! Shaggy -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] JFS filesystem crash after long time run.
Please don't post html to the mailing list. I had to manually approve
it.
On Thu, 2004-09-23 at 23:59, bzeng wrote:
> using dmesg command, i saw a lot error messages:
> ide0: unexpected interrupt, status=0xd0, count=218
> ide0: reset: success hda: recal_intr: status=0x51 { DriveReady
> SeekComplete Error }
> hda: recal_intr: error=0x04 { DriveStatusError }
> hda: write_intr error1: nr_sectors=1, stat=0x51
> hda: write_intr: status=0x51 { DriveReady SeekComplete Error }
> hda: write_intr: error=0x04 { DriveStatusError }
> hda: recal_intr: status=0x51 { DriveReady SeekComplete E
>
> But after reboot the system, all is ok. Howerer just after one day,
> this situation occured again.
>
> Who can't give me some tips?
It is either a failing disk, or a problem in the ide driver.
--
David Kleikamp
IBM Linux Technology Center
___
Jfs-discussion mailing list
[EMAIL PROTECTED]
http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] dbAllocNext: Corrupt dmap page
On Thu, 2004-09-23 at 13:52, Marcel Lanz wrote: > > fsck against it, and remounting it. Rebooting should also fix it, > > assuming a valid entry in /etc/fstab. > see below. > > > I'm not aware of any known bugs that would have caused the corruption. > > You haven't seen any i/o errors from the disk, have you? > no. i haven't. > > well after the fsck i have the following (trunc) in lost+found and the directory > tree is blown away: > > defiant:/mounts/wormhole# l lost+found/ | head -n 20 > total 36801744 > drwx-- 12 root root 4096 Sep 23 20:35 . > drwxr-xr-x 3 lanzm users 8 Sep 6 16:13 .. > drwxr-xr-x 2 lanzm lanzm 1 Jun 28 20:40 D004128.RCN > drwxr-xr-x 11 lanzm lanzm 80 Aug 7 14:24 D004160.RCN > drwxr-xr-x 3 lanzm lanzm 8 Jul 9 13:04 D020480.RCN > drwxr-xr-x 2 lanzm lanzm 1 Sep 17 11:18 D028090.RCN > drwxr-xr-x 2 lanzm lanzm 72 Jul 29 11:20 D098336.RCN > drwxr-xr-x 2 lanzm lanzm 64 Jul 12 06:24 D122880.RCN > drwxr-xr-x 2 lanzm lanzm 1 Sep 17 11:15 D135168.RCN > drwxr-xr-x 2 lanzm lanzm 72 Aug 23 07:55 D143392.RCN > drwxr-xr-x 2 root root 16 Sep 6 14:00 D147488.RCN > drwxr-xr-x 6 root root 56 Sep 23 11:34 D151584.RCN You may be able to determine from doing an ls in these directories, what they originally were and mv them back to the proper path. From the fsck output, it looks like the contents of lost+found came from the root of the file system, and one other directory. > -rw-r--r-- 1 lanzm lanzm 1 Jul 9 20:04 I004104.RCN > -rw-r--r-- 1 lanzm lanzm 507445248 Aug 7 14:35 I004105.RCN > -rw-r--r-- 1 root root 36748583424 Aug 4 11:38 I004106.RCN > -rw-r--r-- 1 lanzm lanzm 0 Jul 9 20:00 I049152.RCN > -rw-r--r-- 1 lanzm lanzm 137598377 Jul 9 20:01 I053248.RCN > -rw-r--r-- 1 lanzm lanzm 0 Jul 9 20:00 I057344.RCN > -rw-r--r-- 1 lanzm lanzm 0 Jul 9 20:00 I057345.RCN > > it seems the files are there, but well... It's tedious to go through 74 files to determine what they are, but if any of these files are vital, you can probably identify them and mv them back to the proper path after fixing the directories. Again they should all have come from the root and one other directory. The other directories should be intact under /lost+found/Dxx.RCN/ > I know that one can say that I should backup my stuff. > well this is my backup drive and I have those things redundant, but > hey. I used ext2 for years and never had something like this. (i > know I should say that) > > I'm interested if someone can tell me what this could be. its > enough if I hear something like: > "well something with you Y went wrongs while the fs commited X during > the Z transaction and sync up." I wish I knew what caused this, but I don't. :^( Can you send me the output of "jfs_fscklog -d -e /dev/sda1"? It probably won't be enough to figure out the cause of the problem, but it might help me to know where to look. Thanks, Shaggy -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] dbAllocNext: Corrupt dmap page
On Thu, 2004-09-23 at 09:32, Marcel Lanz wrote: > Today I had those messages in my syslog: > > kern.log:Sep 23 12:25:04 localhost kernel: ERROR: (device sda1): dbAllocNext: > Corrupt dmap page > kern.log:Sep 23 12:25:04 localhost kernel: ERROR: (device sda1): dbAllocNext: > Corrupt dmap page > > I found my filesystem remounted readonly. > > Kernel is a stock 2.6.8 > > Since I can't find anything on the web and archive about this error, I'd > report it here and I'd happy to hear someone who could tell me if I > should remount my fs ? You won't be able to remount it read-write until fsck is run against it. You're best off unmounting it completely (if possible), running fsck against it, and remounting it. Rebooting should also fix it, assuming a valid entry in /etc/fstab. I'm not aware of any known bugs that would have caused the corruption. You haven't seen any i/o errors from the disk, have you? -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] [Jfs-discussion] Re: unreadble JFS (input-output error with ls command)
On Mon, 2004-09-20 at 14:42, Petr Humlicek wrote: > 763 directories reconnected to /lost+found/. > 5844 files reconnected to /lost+found/. Ouch, those are a lot of files that you lost the path to. > > Ater this I can mount filesystem and I see data in lost+found. That really > wonderful, but i lost original directory tree. Maybe I want impossible, but can I > safe this data better? > But how I say, This result is really great and, I'm very grateful. If I can't get > better result, after all I'm glad for this. fsck could probably do a better job of recovering damaged directories, but it currently doesn't try to hard. Instead it throws out the directory, and all of the contents get put in lost+found. When a disk goes bad, no tool will ever be able to recover all of the lost data, so it's always recommended to back up any data that will need to recover. I'm glad I could help, and thanks for helping me find that bug in fsck. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] btstack overrun
This one's new to me. I'll look into it. Shaggy On Wed, 2004-09-15 at 15:21, Sonny Rao wrote: > I also got a kernel oops trying to umount the filesystem in question. > Here's the output from xmon: > > [c01e0a7f7220] c00d4ca4 .mpage_writepages+0x220/0xb04 (unreliable) > [c01e0a7f74b0] d03365dc .jfs_writepages+0x1c/0x34 [jfs] > [c01e0a7f7530] c007db2c .do_writepages+0x58/0x80 > [c01e0a7f75b0] c0077790 .__filemap_fdatawrite_range+0x9c/0xb8 > [c01e0a7f76b0] c00777f4 .filemap_fdatawrite+0x1c/0x2c > [c01e0a7f7730] d034cb78 .dbSync+0x510/0x59c [jfs] > [c01e0a7f7890] d034ccf8 .dbUnmount+0xf4/0xf8 [jfs] > [c01e0a7f7920] d033aa04 .jfs_umount+0xe8/0x1b0 [jfs] > [c01e0a7f79c0] d033511c .jfs_put_super+0x20/0x88 [jfs] > [c01e0a7f7a40] c00ae050 .generic_shutdown_super+0x2b0/0x30c > [c01e0a7f7ae0] c00ae0d0 .kill_block_super+0x24/0x58 > [c01e0a7f7b70] c00ae2c4 .deactivate_super+0xc0/0x120 > [c01e0a7f7c00] c00cd344 .__mntput+0x3c/0x58 > [c01e0a7f7c90] c00b828c .path_release_on_umount+0x74/0x8c > [c01e0a7f7d20] c00ce984 .sys_umount+0xd8/0x5b0 > [c01e0a7f7e30] c0010600 syscall_exit+0x0/0x18 > --- Exception: c01 (System Call) at 0ff7c3d0 > SP (e1b0) is in userspace > > > Sonny -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
[Jfs-discussion] Re: btstack overrun
On Wed, 2004-09-15 at 15:02, Santhosh Rao wrote: > Shaggy, have you ever seen an error like this? Not in a recent kernel. > Is it a problem with the device/low-level scsi stuff or with JFS (I'm > not sure I completely trust the device driver here) ? It's due to bad metadata in a directory, so the root cause may be corrupt data from the disk. If you suspect the device driver, it very well could be. > Kernel is 2.6.9-rc1 running on a 16-way p690 LPAR with 112 Raid-0 > arrays. > > > > > ERROR: (device sdbf): dtReadFirst: btstack overrun > btstack dump: > bn = 0, index = 0 > bn = 40001, index = 0 > bn = 0, index = 0 > bn = 40001, index = 0 > bn = 0, index = 0 > bn = 40001, index = 0 > bn = 0, index = 0 > bn = 1b5b30306d0ace90, index = 0 This one's really odd. 1b 5b 30 30 6d is '[00M', which I believe tells the tty to reset to default mode. Oh, never mind. The last entry on the stack is not yet initialized. Kind of interesting, though. > SCSI error : <11 0 0 6> return code = 0x802 > Current sdbf: sense = 70 4 > ASC=84 ASCQ= 0 > Raw sense data:0x70 0x00 0x04 0x00 0x00 0x00 0x00 0x98 0x00 0x00 0x00 > 0x00 0x84 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x01 0x00 0x00 0x00 0x00 > 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x88 0x0d 0x00 0x99 0x99 > 0x99 0x99 0x28 0x00 0x00 0x20 0x00 0x08 0x00 0x00 0x08 0x00 0x00 0x00 > 0x00 0x31 0x54 0x32 0x33 0x31 0x34 0x38 0x36 0x34 0x31 0x20 0x20 0x20 > 0x20 0x20 0x20 0x05 0x40 0x06 0x00 0x00 0x06 0x04 0x00 0x0a 0x00 0x00 > 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 > 0x00 > end_request: I/O error, dev sdbf, sector 2097160 > Buffer I/O error on device sdbf, logical block 262145 I'm no scsi expert, but I'm pretty sure this error is not caused by jfs. Any such errors that occurred before the btstack overrun could have caused the problem. > > > Sonny -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] fsck.jfs issues
On Mon, 2004-09-13 at 03:55, Ash wrote: > I'm using Kernel 2.6.7 on Redhat 8.0. > And yes, this does get reproduced. Here's what I have from a recent > crash test cycle. 2.6.7 is pretty recent, so it's probably not a problem that has been fixed yet. It would be helpful if you can recreate the problem and, after fsck -f fails the first time, send me the output of "jfs_fscklog -d -e /dev/cciss/c0d0p8" and "jfs_logdump -a /dev/cciss/c0d0p8". jfs_logdump will save the output as ./jfslog.dmp. Thanks, Shaggy -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Re: unreadble JFS (input-output error with ls command)
On Tue, 2004-09-14 at 05:30, Petr Humlicek wrote: > Yes same DT_GETPAGE: dtree page corrupt... What about "0 bytes in FIFO" and "timeout waiting for DMA"? On hdb1, did you run fsck with the -f flag, or did it simply run phase 0? > > Disable DMA doesn't help. I can't see any data. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] [CHECKER] jfs_fsck --replay_journal_only fails to repair partially written log (JFS2.4, Linux2.4.19, JFSUTILS1.1.5)
On Fri, 2004-09-10 at 11:52, Junfeng Yang wrote: > On Fri, 10 Sep 2004, Dave Kleikamp wrote: > > > I'm confused as to how only those sectors get written to disk, but not > > sectors 2 and 4. How are you monitoring this? > > Thanks for the reply. > > We have a tool that can permute all the sector level writes and check for > inconsistencies. Our assumption is that the disk sector may go bad and be > remapped automatically, so even if the sector numbers are sequential, the > sectors may not be adjacent on the disk. that implies sectors can be > written out of order. is our assumption correct? I guess that is possible, but I don't see any practical way of safeguarding against this. At least not without redefining the format of the log page to include a checksum or something. I would assume that the possibility of seeing this behavior in a real-world situation is very small. I would need to be convinced that this could be a common enough occurrence to warrant a fix. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] [CHECKER] jfs_fsck --replay_journal_only fails to repair partially written log (JFS2.4, Linux2.4.19, JFSUTILS1.1.5)
On Mon, 2004-09-06 at 22:56, Junfeng Yang wrote: > Hi, > > We had a mount failure when trying to mount an image that was created by > the following steps: > > 1. created a file under '/.' > > 2. created another file under '/.' > > 3. run kupdated to flush the log. crash after sectors 0, 1, 3, 5, 6, 7 of > the log block (block 3842 in our case) is written out. note that the > writes are log writes only. the rest of the disk is intact. so > presumably jfs_fsck --replay_journal_only should fix the problem. I'm confused as to how only those sectors get written to disk, but not sectors 2 and 4. How are you monitoring this? > 4. run jfs_fsck --replay_journal_only on the crashed disk image. it > complained > > sbin/jfs_fsck version 1.1.5, 04-Mar-2004 > processing started: 3/20/1933 18.52.48 > The current device is: /tmp/junfeng/cmc_blkdev_1_0_jfs_15788 > Block size in bytes: 4096 > Filesystem size in blocks: 4096 > **Phase 0 - Replay Journal Log > logredo failed (rc=-269). fsck continuing. > Filesystem is clean. It shouldn't be reporing that the filesystem is clean. > After step 4, the attempts to mount this image failed. jfs complained > that "Log is Dirty". This is the correct behavior. fsck without the --replay_journal_only should fix any problems. > For some early version of jfs, we got a kernel oops > at <2>BUG at jfs_dtree.c: assert((btstack)->top != > &((btstack)->stack[MAXTREEHEIGHT])) I think this is fixed in recent kernels. > Run jfs_fsck -p can fix the problem. > > The crashed image can be downloaded at > http://keeda.stanford.edu/~junfeng/crashed.img.bz2 > > As usual, any confirmation or clarification is appreciated. jfs assumes that each log page is written sequentially. sequence numbers at the beginning and end of each page must be correct. I can't imagine why sectors 0 & 7 can both be written, but not everything in between them. Did you have to do something unnatural to cause this to happen? > > -Junfeng Thanks, Shaggy -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] fsck.jfs issues
On Tue, 2004-09-07 at 05:57, szonyi calin wrote: > Hmm. I had the same problem here. > I have no idea why the filesystem is marked clean when it's > dirty. My idea is (i might be wrong) that after replaying the > journal fsck thinks that the filesystem is clean (after all > replaying the journal log means that the journal reflects the > state of the filesystem). Right. By design, fsck assumes that after replaying the journal, that the filesystem is sane. > This could lead in time to severe > filesystem corruption because the journal is not in sync > with the filesystem (i had this problem a cople of times in > the past). The journal should always be "in sync" with the file system. File system corruption may happen, due to hardware problems or software bugs, but the design goal is not to have fsck check everything during reboot. If corruption is detected during runtime, jfs marks the superblock as dirty, so that the next time fsck is run, it will check the entire file system. > My problem appeared after a kernel crash (bad hardware). That's > why i didn't report it because i thought to be caused by the > bad hardware (i.e. a poinetr gets corrupted and erroneous > data together with journal is written to the filesystem until > the crash) > > Bye > Calin Shaggy -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] fsck.jfs issues
On Tue, 2004-09-07 at 05:27, Ash wrote: > Hi > I was running a kind of crash test on JFS in which > I do a large number of filesystem operations with several threads and > crash the machine with a direct poweroff. > > On reboot, I ran fsck.jfs firstly with default options simply to replay the log. > This succeeded without reporting and problems and said that the > filesystem was marked clean. > > **Phase 0 - Replay Journal Log > Filesystem is clean. > > > Next, I ran fsck.jfs with "-f" option to verify if there are any > inconsistencies in the filesystem. > I got a few errors saying something like > > **Phase 4 - Report Problems > File system object FF106504 is linked as: /d15/run_23-1109 > cannot repair the data format error(s) in this file. > cannot repair FF106504. Will release. These errors should not be here. Replaying the journal is supposed to make sure that the metadata is consistent after a power-off. I can tell that you are using a recent version of jfsutils, but are you using a recent kernel? > > At completion though, it looked like fsck.jfs had found errors, > corrected them and > filesystem was marked clean > > Now, I again tried fsck.jfs with "-f" for a third time. I expected > that this time it would > not find any errors and report that the filesystem was clean. However, > it again reported > errors like > > **Phase 4 - Report Problems > File system object DF4109 is linked as: /d15 > Errors detected in Directory Index Table. Will Fix. > I'm aware of this problem. fsck didn't used to verify the contents of the directory index table. Now that it does, it points out another problem that fsck does not update this table when it removed a bad file from a directory. It's not a severe problem, but I do need to fix it. > Finally, at the 4th run of fsck, i noticed that it did not report any problems. > My question is, why did fsck report problems for the 2nd time also > even after it had > just done a full filesystem check and repair ? > I tried about 30 "crash" cycles and found that this problem got > reproduced 3 to 4 times. > Is there some other option that I need to specify to scan and fix all > problems ? Or am I missing > something here ? You're not missing anything. It's a bug. I'm more concerned about why there are errors in the first place. What kernel are you running? > > Thanks, > Ash Thanks, Shaggy -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Re: unreadble JFS (input-output error with ls command)
On Fri, 2004-09-10 at 01:59, Petr Humlicek wrote: > > Can you mount /dev/hdb1 successfully? > > No sucessfully, it's a same problem. fsck clean, I can mount it, but ls > doesn't work. So i do it just to be sure, if harddrive is OK. do you get the same errors from dmesg when mounting /dev/hdb1? Can you try disabling dma on the disk with hdparm -d0 to see if that helps? -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] fsck cannot continue
On Mon, 2004-09-06 at 13:30, Steve Gehlbach wrote:
> However, in the logs I see this:
>
> Sep 6 11:12:51 server70 kernel: hdc: dma_intr: status=0x51 { DriveReady
> SeekComplete Error }
> Sep 6 11:12:51 server70 kernel: hdc: dma_intr: error=0x40 {
> UncorrectableError }, LBAsect=46724440, sector=46724352
> Sep 6 11:12:51 server70 kernel: end_request: I/O error, dev 16:01 (hdc),
> sector 46724352
> Sep 6 11:12:53 server70 kernel: hdc: dma_intr: status=0x51 { DriveReady
> SeekComplete Error }
> Sep 6 11:12:53 server70 kernel: hdc: dma_intr: error=0x40 {
> UncorrectableError }, LBAsect=46724440, sector=46724360
> Sep 6 11:12:53 server70 kernel: end_request: I/O error, dev 16:01 (hdc),
> sector 46724360
> Sep 6 11:12:55 server70 kernel: hdc: dma_intr: status=0x51 { DriveReady
> SeekComplete Error }
> Sep 6 11:12:55 server70 kernel: hdc: dma_intr: error=0x40 {
> UncorrectableError }, LBAsect=46724440, sector=46724368
> Sep 6 11:12:55 server70 kernel: end_request: I/O error, dev 16:01 (hdc),
> sector 46724368
> Sep 6 11:12:56 server70 kernel: hdc: dma_intr: status=0x51 { DriveReady
> SeekComplete Error }
> Sep 6 11:12:56 server70 kernel: hdc: dma_intr: error=0x40 {
> UncorrectableError }, LBAsect=46724440, sector=46724376
> Sep 6 11:12:56 server70 kernel: end_request: I/O error, dev 16:01 (hdc),
> sector 46724376
>
> So maybe it is an HDD failure? The ide interface otherwise seems okay to
> the hda disk.
It definitely looks like a disk error.
> I can mount the drive RO and get data in many places; I have an image
> that is two months old so it is not so bad. But I was surprised to see
> fsck give up. After 25 years of Unix, Sun, and Linux, this the first
> time I have seen fsck give up.
There are some things that jfs's fsck doesn't fix that it should be able
to fix, but it isn't going to be able to recover when there are
excessive I/O errors from the disk.
> Usually it spews inodes but corrects
> them, which usually means loss of data but still the disk is then
> mountable r/w.
> >>I thought _journaling_ filesystems could roll back
> >>transactions and
> >>recover from structural problems.
The journaling doesn't exactly roll back transactions. It makes the
transaction atomic, such that either every part of the transaction gets
committed to disk, or none of it does. The journaling is meant to
prevent structural damage due to a system crash or power loss, not
really to recover if some kind of damage is introduced, either by
hardware failure or software bugs. That is fsck's job, but it's
independent of the journaling.
Shaggy
--
David Kleikamp
IBM Linux Technology Center
___
Jfs-discussion mailing list
[EMAIL PROTECTED]
http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] unreadble JFS (input-output error with ls command)
On Thu, 2004-09-02 at 17:09, Petr Humlicek wrote: > Good morning, > jfsutils-1.1.4-1 > I've problem with my hard drive with JFS filesystem. I using Fedora Core 2 with > default kernel 2.6.5-1.358 and jfsutils-1.1.4-1. > While my work I've got error message: > > Aug 29 18:41:26 vox kernel: ERROR: (device hdg1): diRead: i_ino != di_number > Aug 29 18:41:26 vox kernel: hdg: 0 bytes in FIFO > Aug 29 18:41:26 vox kernel: hdg: timeout waiting for DMA > Aug 29 18:41:26 vox kernel: ERROR: (device hdg1): diRead: i_ino != di_number > Aug 29 18:41:27 vox last message repeated 14 times > Aug 29 18:41:27 vox kernel: hdg: 0 bytes in FIFO > Aug 29 18:41:27 vox kernel: hdg: timeout waiting for DMA > Aug 29 18:41:27 vox kernel: hdg: 0 bytes in FIFO > Aug 29 18:41:27 vox kernel: hdg: timeout waiting for DMA I have to agree with Szonyi, that the problem is either the hardware or the disk device driver. > [EMAIL PROTECTED] root]# ls /mnt/120gb/ > ls: reading directory /mnt/120gb/: Input/output error > > So, I don't know what this mean. I thing that isn't hard drive > problem, because after that i do > [EMAIL PROTECTED] root]# dd if=/dev/hdg1 /dev/hdb1 > without any error, Can you mount /dev/hdb1 successfully? > and utils from manufacturer (SeaGate) says, there is no hardware > problem. And the problem is the same. I can mount partition after > fsck, but i can't list. I'm not sure why dd and fsck run okay, and you get error when the device is mounted. However, the errors are coming from a layer below the file system, so the problem has to be somewhere lower. > So I ask: I lost all of my date, without hope fo safe? > Realy there is no way how I get data back? Why happen this? Since dd from the partition works, you should be able to recover your data from a copy. Shaggy -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Problem after failover
On Wed, 2004-08-18 at 04:25, Dr.Peer-Joachim Koch wrote: > fsck.jfs version 1.0.24, 18-Oct-2002 > The current device is: /dev/emcpowera1 > Block size in bytes: 4096 > File system size in blocks: 65535151 > Phase 0 - Replay Journal Log > logredo failed (rc=-230). FSCK continuing. This is caused by a bug that was fixed in jfsutils-1.1.2. > Phase 1 - Check Blocks, Files/Directories, and Directory Entries. > The root directory has an invalid data format. Will correct. I'm not sure about this one. I don't know if it can be a side effect of the first bug, or not. There have been a number of other fixes since jfsutils-1.0.24 as well. > Phase 2 - Count Links. > Phase 3 - Rescan for Duplicate Blocks and Verify Directory Tree. > Phase 4 - Report Problems. > Phase 5 - Check Connectivity. > Phase 6 - Perform Approved Corrections. > 4 directories reconnected to /lost+found/. > 1 file reconnected to /lost+found/. > Phase 7 - Rebuild File/Directory Allocation Maps. > Phase 8 - Rebuild Disk Allocation Maps. > Phase 9 - Reformat File System Log. > File system is clean. > > > I would like to know what's going on here. How can I avoid this ? Please try jfsutils-1.1.7 from http://www10.software.ibm.com/developer/opensource/jfs/project/pub/jfsutils-1.1.7.tar.gz > Somehow it MUST be possible to switch of the machine and replay the > journal without "losing" the whole directory structure! Absolutely, that's the whole idea of having the journal. > Any idea ??? Please try jfsutils-1.1.7, and let me know if you still have problems. > > Software: > kernel 2.4.21.190-smp > jfsutils-1.0.24-1 > All offical patches from SuSE installed (not the newer kernel, becaue > the powerpath will not work with it ..) -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
[Jfs-discussion] [ANNOUNCE] jfsutils 1.1.7
Release 1.1.7 of jfsutils was made available today. This release include the following changes to the utilities: - --replay_journal_only should not clear FM_DIRTY - Ensure changes to disk occur in the proper order - Message corrections - Directory Index Table corrections for big-endian systems. The last of these is the most critical. Anyone using version 1.1.6 on a big-endian system (i.e. ppc64) should update to version 1.1.7. For more details about JFS, please see our website: http://oss.software.ibm.com/jfs -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Help: jfs_fsck Segfaults on failed drive
On Wed, 2004-07-21 at 21:39, Paul Lanier wrote: > I had both segfaults. The patch fixed the Phase 1 segfault but not the > Phase 0 problem. I've attached a gdb log with a stack backtrace. Let > me know if that's not what you wanted. I tried to compile jfsutils with > debug info but that stack trace still doesn't look very useful so maybe > I didn't set it up right. That's what I wanted, but it's not very helpful. I'm afraid I don't have a lot of time to try to get this debugged. I'm about to leave on a two-week vacation. > I was able to mount the drive in readonly mode (didn't think to try that > before). I think I'll just get what I need off and re-format it. Let > me know if you want another trace before I get rid of it. After recovering what you can, you can try to run "fsck.jfs --omit_journal_replay". This will try to fix everything up without replaying the journal. This may potentially cause a lot of files and directories to be tossed out, so it's a good idea to recover anything important first. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Quota support ?
On Wed, 2004-07-21 at 06:12, Ash wrote:
> Hi
>
> Are user/group quotas supported by JFS currently ? I
> am currently using JFS with the 2.6.7 kernel.
Not currently.
> If not, are there any plans to have support for quotas?
As a matter of fact, I've had a patch laying around for a while that
hasn't been tested too well or merged into the mainline. I updated it
to 2.6.8-rc2, and have run some preliminary tests, and it seems to
work. Care to try it? It also requires a patch to quota-tools:
http://sourceforge.net/projects/linuxquota/
Attached are the patches in case you or anybody else wants to experiment
with them.
Thanks,
Shaggy
--
David Kleikamp
IBM Linux Technology Center
diff -Nurp linux-2.6.8-rc2/fs/jfs/acl.c linux/fs/jfs/acl.c
--- linux-2.6.8-rc2/fs/jfs/acl.c 2004-07-21 10:04:09.0 -0500
+++ linux/fs/jfs/acl.c 2004-07-21 10:59:56.0 -0500
@@ -1,7 +1,7 @@
/*
- * Copyright (c) International Business Machines Corp., 2002
- * Copyright (c) Andreas Gruenbacher, 2001
- * Copyright (c) Linus Torvalds, 1991, 1992
+ * Copyright (C) International Business Machines Corp., 2004
+ * Copyright (C) Andreas Gruenbacher, 2001
+ * Copyright (C) Linus Torvalds, 1991, 1992
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -20,6 +20,7 @@
#include
#include
+#include
#include "jfs_incore.h"
#include "jfs_xattr.h"
#include "jfs_acl.h"
@@ -281,6 +282,12 @@ int jfs_setattr(struct dentry *dentry, s
if (rc)
return rc;
+ if ((iattr->ia_valid & ATTR_UID && iattr->ia_uid != inode->i_uid) ||
+ (iattr->ia_valid & ATTR_GID && iattr->ia_gid != inode->i_gid)) {
+ if (DQUOT_TRANSFER(inode, iattr))
+ return -EDQUOT;
+ }
+
rc = inode_setattr(inode, iattr);
if (!rc && (iattr->ia_valid & ATTR_MODE))
diff -Nurp linux-2.6.8-rc2/fs/jfs/inode.c linux/fs/jfs/inode.c
--- linux-2.6.8-rc2/fs/jfs/inode.c 2004-07-21 10:04:09.0 -0500
+++ linux/fs/jfs/inode.c 2004-07-21 10:36:37.0 -0500
@@ -1,6 +1,6 @@
/*
- * Copyright (c) International Business Machines Corp., 2000-2002
- * Portions Copyright (c) Christoph Hellwig, 2001-2002
+ * Copyright (C) International Business Machines Corp., 2000-2004
+ * Portions Copyright (C) Christoph Hellwig, 2001-2002
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -21,6 +21,7 @@
#include
#include
#include
+#include
#include "jfs_incore.h"
#include "jfs_filsys.h"
#include "jfs_imap.h"
@@ -134,6 +135,13 @@ void jfs_delete_inode(struct inode *inod
diFree(inode);
+ /*
+ * Free the inode from the quota allocation.
+ */
+ DQUOT_INIT(inode);
+ DQUOT_FREE_INODE(inode);
+ DQUOT_DROP(inode);
+
clear_inode(inode);
}
diff -Nurp linux-2.6.8-rc2/fs/jfs/jfs_dtree.c linux/fs/jfs/jfs_dtree.c
--- linux-2.6.8-rc2/fs/jfs/jfs_dtree.c 2004-07-21 10:04:09.0 -0500
+++ linux/fs/jfs/jfs_dtree.c 2004-07-21 12:51:04.504949360 -0500
@@ -101,6 +101,7 @@
*/
#include
+#include
#include "jfs_incore.h"
#include "jfs_superblock.h"
#include "jfs_filsys.h"
@@ -380,7 +381,8 @@ static u32 add_index(tid_t tid, struct i
* It's time to move the inline table to an external
* page and begin to build the xtree
*/
- if (dbAlloc(ip, 0, sbi->nbperpage, &xaddr))
+ if (DQUOT_ALLOC_BLOCK(ip, sbi->nbperpage) ||
+ dbAlloc(ip, 0, sbi->nbperpage, &xaddr))
goto clean_up; /* No space */
/*
@@ -405,7 +407,6 @@ static u32 add_index(tid_t tid, struct i
goto clean_up;
}
ip->i_size = PSIZE;
- ip->i_blocks += LBLK2PBLK(sb, sbi->nbperpage);
if ((mp = get_index_page(ip, 0)) == 0) {
jfs_err("add_index: get_metapage failed!");
@@ -447,7 +448,6 @@ static u32 add_index(tid_t tid, struct i
goto clean_up;
}
ip->i_size += PSIZE;
- ip->i_blocks += LBLK2PBLK(sb, sbi->nbperpage);
if ((mp = get_index_page(ip, blkno)))
memset(mp->data, 0, PSIZE); /* Just looks better */
@@ -946,6 +946,7 @@ static int dtSplitUp(tid_t tid,
struct dt_lock *dtlck;
struct tlock *tlck;
struct lv *lv;
+ int quota_allocation = 0;
/* get split page */
smp = split->mp;
@@ -992,7 +993,9 @@ static int dtSplitUp(tid_t tid,
split->pxdlist = &pxdlist;
rc = dtSplitRoot(tid, ip, split, &rmp);
- if (!rc)
+ if (rc)
+ dbFree(ip, xaddr, xlen);
+ else
DT_PUTPAGE(rmp);
DT_PUTPAGE(smp);
@@ -1017,6 +1020,14 @@ static int dtSplitUp(tid_t tid,
n = xlen + (xlen << 1);
else
n = xlen;
+
+ /* Allocate blocks to quota. */
+ if (DQUOT_ALLOC_BLOCK(ip, n)) {
+ rc = -EDQUOT;
+ goto extendOut;
+ }
+ quota_allocation += n;
+
if ((rc = dbReAlloc(sbi->ipbmap, xaddr, (s64) xlen,
(s64) n, &nxaddr)))
goto extendOut;
@@ -1285,6 +1296,10 @@ static int dtSplitUp(tid_t tid,
freeKeyName:
kfree(key.name);
+ /* Rollback quota allocation */
+ if (rc && quota_allocation)
+ DQU
Re: [Jfs-discussion] Help: jfs_fsck Segfaults on failed drive
On Tue, 2004-07-20 at 23:00, Paul Lanier wrote:
> I had a failure on my JFS drive that was preventing my Linux machine
> from booting. I booted up knoppix and ran fsck on it. It seemed to
> work and I was able to mount the drive under knoppix. However, on
> reboot, the drive still failed to load properly. So back into Knoppix
> and another try at fsck. This time I get a Segfault. I played a little
> with jfs_debugfs but I don't know how to do much other than look
> around. Below is the output of jfs_fsck. I also tried version 1.1.6
> and got the same output and Segfault.
Is this the 1.1.6 segfault in phase 0, or in phase 1? I have a patch
that may fix the segfault in phase 1. It should apply to jfsutils
1.1.6. The patch fixes a problem with a bad format specifier, and I
went ahead and fixed a lot of messages so that inode numbers don't show
up as negative. (They are unsigned.)
If the phase 0 segfault still occurs in 1.1.6, could you maybe get a
stack trace by running fsck.jfs from gdb?
> I also appended some outputs from
> jfs_debugfs. Anyone have an idea how to get this drive working again?
> At least enough to get some files off before I reformat it?
I'm not sure how badly the file system is damaged. Usually you can
mount read-only to recover data, but it sounds like you may have some
pretty serious problems if you can't boot.
--
David Kleikamp
IBM Linux Technology Center
Index: jfsutils/fsck/fsckconn.c
===
RCS file: /usr/cvs/jfs/jfsutils/fsck/fsckconn.c,v
retrieving revision 1.9
diff -u -p -r1.9 fsckconn.c
--- jfsutils/fsck/fsckconn.c 21 Jan 2004 19:13:59 - 1.9
+++ jfsutils/fsck/fsckconn.c 21 Jul 2004 19:33:15 -
@@ -432,7 +432,9 @@ int check_connectedness()
fsck_send_msg
(fsck_INCINOREF,
fsck_ref_msg(msg_info_ptr->msg_inopfx),
- msg_info_ptr->msg_inonum);
+ msg_info_ptr->msg_inonum,
+ fsck_ref_msg(msg_info_ptr->msg_inopfx),
+ this_inorec->parent_inonum);
}
}
}
Index: jfsutils/libfs/fsckmsgdef.c
===
RCS file: /usr/cvs/jfs/jfsutils/libfs/fsckmsgdef.c,v
retrieving revision 1.6
diff -u -p -r1.6 fsckmsgdef.c
--- jfsutils/libfs/fsckmsgdef.c 28 Apr 2004 18:58:43 - 1.6
+++ jfsutils/libfs/fsckmsgdef.c 21 Jul 2004 19:33:15 -
@@ -5,16 +5,16 @@ struct fsck_message msg_defs[fsck_highes
/* 1 */ { fsck_ALLFXD, "All observed inconsistencies have been repaired.", fsck_debug},
/* 2 */ { fsck_RIBADDATFMT, "Invalid data format detected in root directory.", fsck_quiet},
/* 3 */ { fsck_BADBLKCTTTL, "A combination of Minimum Free Blocks and Total Usable Blocks which is invalid for the filesystem size was detected in the superblock (%s).", fsck_debug},
- /* 4 */ { fsck_BADBLKNO, "Invalid block number(s) (%s) detected for file system object %s%s%d.", fsck_debug},
- /* 5 */ { fsck_BADBSBLCHN, "File system object %s%s%d has a corrupt backward sibling chain.", fsck_debug},
- /* 6 */ { fsck_BADFSBLCHN, "File system object %s%s%d has a corrupt forward sibling chain.", fsck_debug},
- /* 7 */ { fsck_BADINOTYP, "Inode %s%d has unrecognized type.", fsck_debug},
- /* 8 */ { fsck_BADINODXDFLDD, "File system object %s%s%d has invalid descriptor (%s).", fsck_debug},
- /* 9 */ { fsck_BADINOLKCT, "Inode %s%d has incorrect link count.", fsck_debug},
- /* 10 */ { fsck_BADINOREF, "Directory inode %s%d refers to a nonexistent inode %s%s (entry %u).", fsck_debug},
+ /* 4 */ { fsck_BADBLKNO, "Invalid block number(s) (%s) detected for file system object %s%s%u.", fsck_debug},
+ /* 5 */ { fsck_BADBSBLCHN, "File system object %s%s%u has a corrupt backward sibling chain.", fsck_debug},
+ /* 6 */ { fsck_BADFSBLCHN, "File system object %s%s%u has a corrupt forward sibling chain.", fsck_debug},
+ /* 7 */ { fsck_BADINOTYP, "Inode %s%u has unrecognized type.", fsck_debug},
+ /* 8 */ { fsck_BADINODXDFLDD, "File system object %s%s%u has invalid descriptor (%s).", fsck_debug},
+ /* 9 */ { fsck_BADINOLKCT, "Inode %s%u has incorrect link count.", fsck_debug},
+ /* 10 */ { fsck_BADINOREF, "Directory inode %s%u refers to a nonexistent inode %s%s (entry %u).", fsck_debug},
/* 11 */ { fsck_ERRONLOG, "Error (%d,%d) writing to the fsck service log (%d,%lld,%ld,%ld). Continuing.", fsck_debug},
/* 12 */ { fsck_BOOTSECFXD, "The boot sector has been refreshed.", fsck_debug},
- /* 13 */ { fsck_BADKEYS, "File system object %s%s%d has corrupt data (%d).", fsck_debug},
+ /* 13 */ { fsck_BADKEYS, "File system object %s%s%u has corrupt data (%d).", fsck_debug},
/* 14 */ { fsck_BADSBOTHR, "Invalid data (%s) detected in the superblock (%s).", fsck_debug},
/* 15 */ { fsck_BADSBAGSIZ, "Invalid allocation group size in the superblock (%s).", fsck_debug},
/* 16 */ { fsck_BADSBBLSIZ, "Invalid filesystem block size in the superblock (%s).",
Re: [Jfs-discussion] about dump and restore
On Tue, 2004-07-20 at 04:17, Avinesh Kumar wrote: > hi, > i want to know ... are there future plans > for supporting dump and restore on jfs ... > if yes then which version will probably have > these feature ? There are no plans for a dump and restore for jfs. The company line is to sell TSM as a backup solution. I personally just rely on tar. :^) Thanks, Shaggy -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Starvation with find?
On Tue, 2004-07-20 at 13:37, John Goerzen wrote: > Interestingly, after I upgraded my 2.6.x machine (the one experiencing > the find problem) to 2.6.7, my problem went away as well. No idea why, > but there it is. If anyone else is having the find problem, you may > consider upgrading to 2.6.7. I'm glad to hear this, since I was at a loss to explain why you were having problems. I trust you'll let me know if you have more problems with jfs. :^) > > -- John Shagy -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] JFS partitions, sizing and resizing.
On Mon, 2004-07-19 at 07:36, Ash wrote: > Hi > I recently started using JFS and had a few basic > questions to ask: > > 1. I could not see any "Size" option in mkfs.jfs. I > presume that means the FS I create will always fill up > the available partition size. Is there any way to > explicitly specify the size of the FS being created ? Not currently. I have in the past hacked up mkfs to do this for testing, but I didn't really anticipate a public need for this. It would be an easy feature to add. > 2. From the docs, I read that resize is possible with > "mount -o remount,resize /mount point". I tried > specifying resize= but it does not seem > to work and gives me "bad option" errors. That should work. should be the number of 4K blocks. If the new size is larger than the partition, remount fails with the "bad option" message. You may get a better message from dmesg. > Putting #1 > and #2 together, it looks like if I increase the > partition size of an existing JFS partition and then > do a "mount remount,resize", it will basically > increase the size of my FS to the new partition size. > Is this correct ? Yes > Is there any option in this case to > specify size explicitly ? resize= is supposed to work. > > 3. Is there any way to shrink the FS ? No. > > 4. What are the defragmentation issues? How do I find > out to what extent is my FS defragmented ? Is there > any tool to defrag a JFS partition. Not yet. One has been on my todo list, but hasn't received much attention. :^( > Any help/docs/pointers will be greatly appreciated. > > Thanks > Ashish Thanks, Shaggy -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
RE: [Jfs-discussion] Should I file a bug report for seg fault on 1.14?
On Fri, 2004-07-09 at 16:34, Edwards, Scott (MED, Kelly IT Resouces) wrote: > One strange thing, after about a dozen power cycles I have been seeing > the "read-only" problem. I'm testing on two drives at the same time and > it is not always on the same drive. Sometimes it fails on sda and the > next time it will be read-only on sdb. The last few power cycles both > drives have been fine. I will run a few more cycles before I return the > machine to active duty. The read-only problem should be accompanied by an error message to the syslog (dmesg or /var/log/messages). Can you find such a message? Also, once the volume goes read-only, can you unmount it and run "fsck -nv" against it? > > Thanks > -Scott Thanks, Shaggy -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] JFS, AIX, Linux, any news?
On Fri, 2004-07-09 at 06:20, Claudio Di Martino wrote: > Hi, > I'm trying to access a volume which contains an AIX JFS filesystem on > a Linux box. Sorry, there is no work in progress to make AIX's jfs accessible from linux. You will need AIX to do that. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Should I file a bug report for seg fault on 1.14?
On Tue, 2004-07-06 at 17:57, Edwards, Scott (MED, Kelly IT Resouces) wrote: > My question is should I file a bug report on one or both of these problems > or have they been fixed? Fedora Core 2 has the 1.14 utilities so I assume > it has the 1.14 version in the kernel (2.6.5)? I couldn't figure out how to > search the bug list and I didn't see anything like this in the open bugs > (and there were too many closed bugs for me to search manually). Updating to jfsutils-1.1.6 might solve either or both of your problems. Can you give it a try and let me know what the results are? I don't have rpm's built, you'll have to build from the source: http://www10.software.ibm.com/developer/opensource/jfs/project/pub/jfsutils-1.1.6.tar.gz -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
[Jfs-discussion] Re: Starvation with find?
On Tue, 2004-07-06 at 09:57, Andi Kleen wrote: > Perhaps one could add that? ext2/3 also only added it later with > a feature flag. Just needs some unused bits in the directory entry. I think it may be possible. I'll look into it. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
[Jfs-discussion] Re: Starvation with find?
On Tue, 2004-07-06 at 09:37, Andi Kleen wrote: > Dave Kleikamp <[EMAIL PROTECTED]> writes: > > > > find requires a lot of inodes to be read, since it needs to stat each > > file to see if it is a directory. Since the the VM isn't being asked to > > It shouldn't need to just for that. readdir returns this information in d_type. > >From a quick look at jfs_dtree it seems to be usually set by JFS > too, except for one case where it passes DT_UNKNOWN. Maybe that can > be fixed? My pretty old find seems to honour it. That one case is the common case. jfs passes back DT_DIR for . and .., but it's kind of obvious that those are directories anyway. jfs would have to read the inode to know whether or not the entry is a directory. It doesn't store that in the directory entry. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Starvation with find?
On Tue, 2004-07-06 at 08:36, John Goerzen wrote: > On this (2.4.26 machine with Amanda), it looks like this during a > backup: > > jfs_mp 432432 80991 : 252 126 > jfs_ip 64080 145859524 20837 208371 : 124 62 > > I don't know exactly what that means. > > Interestingly, on my 2.6.x machine that is having the starvation > problems with find, I observed this: > > # name : > tunables: slabdata > > jfs_mp41100 76 501 : tunables 120 600 : > slabdata 2 2 0 > jfs_ip163017 16301785292 : tunables 54 270 : > slabdata 18113 18113 0 > > In other works, the lightly-loaded workstation running find appears to > have more objects than the much more active server. That makes sense. find is only reading inodes and directories, so the system isn't needing much memory for anything else. backup is also reading all of the file data, so there is less memory available for caching inodes. > Also, those 18113 > numbers are many times larger than the closest second on the machine, > and the 163017 number is also the largest on the box. But I don't > know enough about the VM system to know what this all means. find requires a lot of inodes to be read, since it needs to stat each file to see if it is a directory. Since the the VM isn't being asked to do much else, it uses a lot of memory to cache the inodes. It is not necessarily behaving badly, but I think you can adjust swappiness to tune it to be more friendly to other apps. I don't claim to know much about VM tuning, though. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] trashed jfs system
On Mon, 2004-07-05 at 07:08, Henrik Hellerstedt wrote: > I have a "small" problem with my jfs partition. The box is hp nc-8000 > laptop with a plain filsystem layout, only /boot and /. > > It started with me running a find (which i doubt is the reason). > The box froze and some error messages appeared in dmesg. At the time > i didnt think the error was that big so i didnt save that dmesg :( > > Rebooted the box into singel user and tried to run jfs fsck on > the partition. It failed, complain it was missing libuuid and libacl, > i checked and they existed in /lib but i was not allowed to access > them. So i gave this up. Hmm. I don't know about any dependency on libacl. > I installed RIP into the swapspace (currently converted to fat) > and had great hope it would be able to fix my broken jfs partition, > but its fsck also fails. > > All info i could think of is avaliable at http://ecure.se/~henrik/jfs > If more info is needed i will gladly supply it, just tell me whats > needed. Ugh. There are some problems that fsck.jfs isn't able to deal with. One is "cross linked blocks" in the inode table. Ideally, it should recover as much of the inode table as possible, only losing those inodes that are unrecoverable. > > I also made an "long" S.M.A.R.T check to rule out any problems with > the hardware. smartctl -t long /dev/hda reports no errors at all. > > Possible reason to the problem? Will it happen again? I don't know what would have led to the problem, so I can't tell you if you're likely to see it again. > Can i solve it? Should i take this to some other list? This is the right list. If you have the time and patience, I may be able to fix fsck.jfs to recover the file system, although you will probably lose at least some of the files. If you need the laptop back right away, you may be able to recover what you may need by mounting the filesystem read-only (mount -oro), and then reinstalling. > Any help is welcome. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Starvation with find?
On Fri, 2004-07-02 at 13:42, John Goerzen wrote: > On Fri, Jul 02, 2004 at 01:16:51PM -0500, Dave Kleikamp wrote: > > > sendsize.20040628190003.debug:sendsize[21127]: time 3337.219: /bin/tar: > > > ./src/ke > > > rnel-source-2.6.6/arch/i386/boot/compressed/vmlinux.bin: Warning: Cannot > > > seek to > > > 0: Bad file descriptor > > > > > > Strange, eh? > > > > Yes, it is strange. Is that file otherwise accessible? > > Yes. Then I really don't understand it. > Vanilla 2.4.26 plus vserver patch. That's pretty current. :^) > Why is it that updatedb would cause things to swap out? It shouldn't be > using much RAM. Do we have a bug in the VM system where it > over-aggressively maximizes cache size? I've observed, that jfs's inode cache grows really big. Look at the jfs_ip entry in /proc/slabinfo. The behavior is tunable with /proc/sys/vm/swappiness. I'm not a vm expert, but I found this, which explains it better than I could: http://kerneltrap.org/node/view/3000 -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Starvation with find?
On Fri, 2004-07-02 at 09:04, John Goerzen wrote: > On Fri, Jul 02, 2004 at 08:53:04AM -0500, John Goerzen wrote: > > If I kill the find process, things immediately return to normal. > > > > CPU usage usually remains low but occasionally spends a couple of > > seconds at 100%. > > > > Any thoughts? > > Incidentally, I'm having similar problems with amanda, and with it, I'm > seeing errors like this: > > sendsize.20040628190003.debug:sendsize[21127]: time 3337.219: /bin/tar: > ./src/ke > rnel-source-2.6.6/arch/i386/boot/compressed/vmlinux.bin: Warning: Cannot > seek to > 0: Bad file descriptor > > Strange, eh? Yes, it is strange. Is that file otherwise accessible? I haven't responded to your earlier email about amanda. I've been trying to recall any similar problems I might have seen. What 2.4 kernel are you running? -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Starvation with find?
On Fri, 2004-07-02 at 09:58, John Goerzen wrote: > On Fri, Jul 02, 2004 at 03:54:57PM +0100, Antonio P. P. Almeida wrote: > > > CPU usage usually remains low but occasionally spends a couple of > > > seconds at 100%. > > > > > > Any thoughts? > > > > This is just to say that when I installed 2.6.4 in my laptop it > > presented similar symptoms. IIRC, there was *heavy*, really *heavy* > > I/O activity, ditto for the CPU -- basically the machine was less than > > Glad to know I am not alone :-) > > > sluggish when doing the daily updatedb. That's when I decided to stay > > with 2.4 until the 2.6 problems -- or what seems to be problems -- get > > sorted out. > > I should also add: I have not seen this problem in reiser, ext2, or > ext3. updatedb is notorious for causing everything else to swap out of memory, but I don't know why this would affect jfs more than any other file system. jfs does have a larger in-memory inode, but I would expect the result to be that fewer jfs inode would be cached. I would think that with updatedb running, you'd eventually push everything useful out of memory anyway. Anyway, I ran updatedb against my jfs volumes on 2.6.7 and 2.4.27-rc2, both with and without noatime, and neither brought the system anywhere near a crawl. I don't know of any changes in jfs between 2.6.4 & 2.6.7 that would have improved the situation. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Problems with charset.
On Mon, 2004-06-28 at 15:13, John Goerzen wrote: > > I wonder, though, if there is at least some way to always generate > valid, unique filenames, no matter what? If the same iocharset is consistently used, there shouldn't be a problem. Also, iocharset=utf8 should allow all files to be accessible, although jfs will not accept names that are not valid utf-8 strings. > I've experienced this before (in 2.6, prior to that patch, when I used > rsync to copy some files with Japanese names from my Zaurus). Files > created, but are undeletable 2 minutes later. There was a bad version of the patch in 2.6 for a little while. This was caused by an unintended sign extension. You shouldn't see that particular problem again (I hope). > Ahh. So on OS/2, JFS would translate a stream of bytes in charset A > (whever that may be) to the charset in use on the filesystem, and back? Yes, 16-bit unicode. (Is that UCS-16? The code in fs/nls/ just refers to it as Unicode.) > So on Linux, how does JFS know what charset it is coming *from*? (Say > you have iocharset=utf8, where does that conversion get used, and how > does it know what to convert to/from?) It uses the conversions in linux/fs/nls/nls_utf8.c (or whatever the iocharset is). On lookup, it will convert from utf8 to Unicode to search for the name. readdir will convert back from Unicode to utf-8. > > Don't get the wrong impression. I'm not an OS/2 advocate. I like linux > > much better. I am only trying to explain why jfs was written this way. > > I wish it wasn't. :^) > > What about JFS on AIX? How does it handle this situation? I really don't know. They were not concerned with compatibility with OS/2, so maybe they dropped the unicode completely. I'll have to ask the AIX guys. I know JFS1 was just a stream of bytes. > (We do have an AIX machine here runninf JFS2, and I've never experienced > this problem there. OTOH, it's possible that I've never used anything > but 7-bit filenames there.) > > -- John -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Problems with charset.
On Mon, 2004-06-28 at 14:16, John Goerzen wrote: > On Mon, Jun 28, 2004 at 02:04:40PM -0500, Dave Kleikamp wrote: > > Unfortunately, in Linux the locale information is not accessible by the > > kernel, so jfs needs to be told what the iocharset is. The best I could > > do was a per-mount option with iocharset. I originally decided the > > default would use CONFIG_NLS_DEFAULT. > > I still don't understand the value of the filesystem knowing what the > charset is, nor how this fits in with everything else on the system. > Why does JFS not just use a stream of bytes like the other Linux > filesystems to store filenames? The jfs filesystem was ported from OS/2. I tried to explain how it worked in OS/2 and why. Because it was ported, I tried to maintain portability between OS/2 and Linux, therefore pathnames are stored in 16-bit unicode. If jfs had been written from scratch for linux, it wouldn't work this way. > > > On the 2.4.26 system I just installed, lots of filenames came back with > > > '?' characters in them. I finally had to mount it with charset=utf8. > > > At least that way, files can be deleted. I had reported the same bug in > > > 2.6, and it appears to have been fixed there. Why not in 2.4 also? > > > > You shouldn't have inaccessible files unless either the value of > > CONFIG_NLS_DEFAULT changed in the kernels you are running, or you used a > > different iocharset mount option. > > I migrated from ext3 to JFS. ext3 has no iocharset uption. > CONFIG_NLS_DEFAULT did not change. The machine is a Samba server, and I > have no idea what people are using for characters in their filenames on > clients. I don't know how you got inaccessible files. Files created in a particular charset should be accessible in the same charset. > > In 2.6 I changed the default from using CONFIG_NLS_DEFAULT to doing no > > translation. I thought this change to the default behavior would not be > > appropriate in the more stable 2.4 kernel, since users using the default > > with CONFIG_NLS_DEFAULT set to something other than iso8859-1 would have > > trouble accessing existing files. > > Could you at least add an option, such as iocharset=none, to disable > conversion in a 2.4 kernel? Yes, that would be reasonable. > > > I don't see why JFS forces this system-wide choice. > > > > Let me know how the file system code can determine the correct locale > > for each process. I've fixed the default in 2.6 to do no translation, > > but I'm not willing to change 2.4 now. > > It can't. I still don't understand why it needs to. If it was possible (as it is in OS/2), it's a great feature. User A, using an iso8859-1 locale, can create files containing any characters in that charset. User B, using a utf-8 locale, can see those files correctly, even though they are represented as a different stream of bytes to him. Not all characters translate into all charsets, so there is a drawback. OS/2 also passed wildcards down to the file system, so it was even possible to delete files with untranslated characters such as "del portugu?s.alias". Don't get the wrong impression. I'm not an OS/2 advocate. I like linux much better. I am only trying to explain why jfs was written this way. I wish it wasn't. :^) > > -- John -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Problems with charset.
On Mon, 2004-06-28 at 11:24, John Goerzen wrote: > On Mon, Jun 28, 2004 at 10:57:25AM -0500, Dave Kleikamp wrote: > > On Sun, 2004-06-27 at 08:04, Antonio A. wrote: > > > This is a FS problem since I had a previous version of the dictionary > > > that I installed without problems. > > > > > > Doing a dmesg I get this > > > > > > jfs_strtoUCS: char2uni returned -22. > > > charset = utf8, char = 0xea > > > > iocharset=utf8 tells jfs that the pathnames are utf-8, but that is not > > the character set being used. 0xea is à in iso8859-1. It could be a > > valid byte of a utf-8 character, but the next two bytes would have to > > have the high order bit set, and I assume they don't. (I'm sure they > > are 's' and '.'.) > > I am confused about all of this. Why should the filesystem care? > With every other FS, it just stores the stream of bytes that make up the > filename. Why is JFS any different? The reason goes back to the origins of this file system on OS/2. In OS/2 the file system had access to each process's locale, so storing the path names in unicode was useful, and conversion to the user's characters set was reliable. Unfortunately, in Linux the locale information is not accessible by the kernel, so jfs needs to be told what the iocharset is. The best I could do was a per-mount option with iocharset. I originally decided the default would use CONFIG_NLS_DEFAULT. > On the 2.4.26 system I just installed, lots of filenames came back with > '?' characters in them. I finally had to mount it with charset=utf8. > At least that way, files can be deleted. I had reported the same bug in > 2.6, and it appears to have been fixed there. Why not in 2.4 also? You shouldn't have inaccessible files unless either the value of CONFIG_NLS_DEFAULT changed in the kernels you are running, or you used a different iocharset mount option. In 2.6 I changed the default from using CONFIG_NLS_DEFAULT to doing no translation. I thought this change to the default behavior would not be appropriate in the more stable 2.4 kernel, since users using the default with CONFIG_NLS_DEFAULT set to something other than iso8859-1 would have trouble accessing existing files. > > > I have the iocharset=utf8 mount option set in fstab. > > > > Are you sure your system is configured in a utf-8 locale? (What is > > $LANG?) > > That is a fallacy. There is no reason to assume that everything on the > system uses the same locale setting. For instance: > > * Some users may log in from Unicode-capable terminals, and thus prefer > to use UTF-8. Others may use a Latin-1 terminal. > > * The machine may act as a file server, serving files to machines that > are individually configured with different locales. (Heck, they may > even be physically located in different locales!) > > * LANG could be individually set per-user or even per-process for a > variety of reasons. > > I don't see why JFS forces this system-wide choice. Let me know how the file system code can determine the correct locale for each process. I've fixed the default in 2.6 to do no translation, but I'm not willing to change 2.4 now. > > -- John -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Problems with charset.
On Sun, 2004-06-27 at 08:04, Antonio A. wrote: > I can't install the aspell portuguese dictionary. When installing the > debian package it cannot stat a file /usr/lib/aspell/portuguÃs.alias > > This is a FS problem since I had a previous version of the dictionary > that I installed without problems. > > Doing a dmesg I get this > > jfs_strtoUCS: char2uni returned -22. > charset = utf8, char = 0xea iocharset=utf8 tells jfs that the pathnames are utf-8, but that is not the character set being used. 0xea is à in iso8859-1. It could be a valid byte of a utf-8 character, but the next two bytes would have to have the high order bit set, and I assume they don't. (I'm sure they are 's' and '.'.) > I have the iocharset=utf8 mount option set in fstab. Are you sure your system is configured in a utf-8 locale? (What is $LANG?) > Needless to say that the filenames in the directory are garbled. > > I'm using a 2.4.26 kernel. Is there a way around this? The way around it is to specify the iocharset that matches the systems's locale. I don't believe it is utf-8. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Kernel 2.4.26 XT_GETPAGE: xtree page corrupt
Here is a patch that should fix the oops. If you see the problem again,
you should get the XT_GETPAGE error in the error log and the filesystem
should become read-only until fsck is run against it to fix it.
Did fsck clean the filesystem, or is it still unmounted? If fsck hasn't
fixed it, I would be interested in the output of "fsck -nv" run against
it, but I image it's probably too late for that.
= fs/jfs/jfs_xtree.c 1.14 vs edited =
--- 1.14/fs/jfs/jfs_xtree.c 2004-03-24 14:11:46 -06:00
+++ edited/fs/jfs/jfs_xtree.c 2004-06-25 14:44:58 -05:00
@@ -1071,8 +1071,10 @@
*/
/* get/pin the parent page */
XT_GETPAGE(ip, parent->bn, smp, PSIZE, sp, rc);
- if (rc)
- goto errout2;
+ if (rc) {
+ XT_PUTPAGE(rcmp);
+ return rc;
+ }
/*
* The new key entry goes ONE AFTER the index of parent entry,
@@ -1106,8 +1108,10 @@
rc = (sp->header.flag & BT_ROOT) ?
xtSplitRoot(tid, ip, split, &rmp) :
xtSplitPage(tid, ip, split, &rmp, &rbn);
- if (rc)
- goto errout1;
+ if (rc) {
+ XT_PUTPAGE(smp);
+ return rc;
+ }
XT_PUTPAGE(smp);
/* keep new child page pinned */
@@ -1170,19 +1174,6 @@
XT_PUTPAGE(rmp);
return 0;
-
- /*
-* If something fails in the above loop we were already walking back
-* up the tree and the tree is now inconsistent.
-* release all pages we're holding.
-*/
- errout1:
- XT_PUTPAGE(smp);
-
- errout2:
- XT_PUTPAGE(rcmp);
-
- return rc;
}
___
Jfs-discussion mailing list
[EMAIL PROTECTED]
http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] Kernel 2.4.26 XT_GETPAGE: xtree page corrupt
On Fri, 2004-06-25 at 08:59, Guerra, Jim wrote: > Hello > > We have a redhat 9 system running the 2.4.26 kernel, jfsutil-1.1.6 and a > 1.4T jfs filesystem. > > on an appearantly quiet system we get the following error and have to > reboot to use to get the machine back > There are two problems here. jfs discovered corrupt metadata in the file. This is the XT_GETPAGE warning. The cause of this may be hard to determine. The second problem is the BUG immediately following it. This should be easy to fix if I can find out where release_metapage is being called. Can you run the oops through ksymoops? (see linux/Documentation/oops-tracing.txt. > > Thanks for your help Thanks for reporting this. Shaggy -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
Re: [Jfs-discussion] JFS and top
On Wed, 2004-06-23 at 15:55, John Goerzen wrote: > Hmm. I wonder why this is different between 2.4 and 2.6? It might have to do with jfs using nobh_prepare_write, nobh_commit_write, and nobh_truncate_page in 2.6. In 2.4, jfs calls block_prepare_write, etc. The block_* helpers still allocate buffer_heads and attach them to the pages. -- David Kleikamp IBM Linux Technology Center ___ Jfs-discussion mailing list [EMAIL PROTECTED] http://www-124.ibm.com/developerworks/oss/mailman/listinfo/jfs-discussion
