Re: ext2 inode size patch - RE: PR kern/124621
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Wed, 3 Dec 2008 17:53:43 -0500 Josh Carroll josh.carr...@gmail.com mentioned: Ok, I describe my concern once more. I do not object against the checking of the inode size. But, if inode size is changed, then some data is added to the inode, that could (and usually does, otherwise why extend it ?) change intrerpetation of the inode. Thus, we need a verification of the fact that simply ignoring added fields does not damage filesystem or cause user data corruption. Verification != testing. Let me take another crack at explaining why I think this patch is not dangerous. The inode size is determined by reading the following member: __u16 s_inode_size; of the ext2_super_block structure. I took a look at the Linux 2.6.27.7 kernel source, and indeed they do something very similar if not the same: #define EXT2_INODE_SIZE(s) (EXT2_SB(s)-s_inode_size) If you compare to what I did: #define EXT2_INODE_SIZE(s) ((s)-u.ext2_sb.s_inode_size) This is really the same thing, since EXT2_SB is defined (in the Linux kernel src as): #ifdef __KERNEL__ #include linux/ext2_fs_sb.h static inline struct ext2_sb_info *EXT2_SB(struct super_block *sb) { return sb-s_fs_info; } And struct ext2_sb_info has the following member: int s_inode_size; Essentially, the changes I made simply query the existing information from the filesystem, which is what the Linux kernel does as well. The inode size, blocks per group, etc are all defined at filesystem creation time by mke2fs and it ensures the sanity of the relationship between the inodes/blocks/block groups. The first diagram here: http://sunsite.nus.sg/LDP/LDP/tlk/node95.html Makes it clear that as long as the number of inodes per block and the blocks per group is is sane during fs creation, looking up the inode size as my patch does is fine, since the creation of the filesystem is ensures a correct by construction situation. We're simply reading the size of the inode based on the filesystem. I hope this is sufficient to convince some further thought about committing this. For those interested in the relevant Linux kernel source, you can have a look at line 105 of include/linux/ext2_fs.h from the linux-2.6.27.7 kernel source. Hi Josh! I've commited the similar patch today that should be fixing your problem. Can you check this, please? Sorry I've missed this thread in the first place. - -- Stanislav Sedov ST4096-RIPE -BEGIN PGP SIGNATURE- iEYEARECAAYFAklzYsoACgkQK/VZk+smlYF3ZwCeOyVUdzrKOdu4Pg3ztAZ0QQaY GGIAnA+oL054T0EAajbfwpYSTDRKVISC =jJFT -END PGP SIGNATURE- !DSPAM:497362c8967002668918391! ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: ext2 inode size patch - RE: PR kern/124621
FYI:The ext2 IFS driver for Windows v1.11a also appears to have the inode size issue: http://www.fs-driver.org/ I was not able to mount an ext2 filesystem with 256 byte inode size using this driver. Its installer will see that the filesystem exists, that it's ext2, but whenever you try to mount, you get nothing -- a very similar failure mode to the FreeBSD ext2fs driver. Again, it sounds like the author is actively maintaining it, so it might be worth contacting him to pool resources. cheers, BMS ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ext2 inode size patch - RE: PR kern/124621
On Thu, Dec 04, 2008 at 07:15:05AM +, Bruce M. Simpson wrote: Hi, The inode size for the ext3 filesystem which Gentoo created for my last install defaulted to 256 bytes, so I got bit by this problem. I can't speak for the write path. but the read path looks just fine to me, and the patch should go in ASAP. Josh Carroll wrote: Ok, I describe my concern once more. I do not object against the checking of the inode size. But, if inode size is changed, then some data is added to the inode, that could (and usually does, otherwise why extend it ?) change intrerpetation of the inode. Thus, we need a verification of the fact that simply ignoring added fields does not damage filesystem or cause user data corruption. Verification != testing. If folk are paranoid, then add a check for dynamic inode size and disable ext2fs writes by downgrading the mount in that case (We can do that, right? Can someone make sure Josh gets the help he needs here?) As Josh points out, the ext2 inode size is stored in the superblock. Whilst it may vary between ext2 filesystems, *the inode size itself does not appear to be something which one can modify in an existing ext2/3 filesystem*. Older ext2 filesystems may not contain the inode size field in the superblock, and the patch appears to default to 128 for that case. The double indirection thus introduced doesn't concern me, our ext2fs is not performance critical code, and the superblock is likely to sit in L2/L3 cache anyway (note: content free argument). Thanks to Josh for fixing this problem. Bruce, feel free to commit the patch. I do not want to spend time on ext2 in any form, and due to our (only partly jokingly) rule of the last committer is the owner, I do not want to analyze ext2 bug reports after. pgpGCrYMCttDh.pgp Description: PGP signature
Re: ext2 inode size patch - RE: PR kern/124621
Kostik Belousov wrote: ... Bruce, feel free to commit the patch. I do not want to spend time on ext2 in any form, and due to our (only partly jokingly) rule of the last committer is the owner, I do not want to analyze ext2 bug reports after. Yes, development resource is limited here too, and any involvement on my part here DOES NOT constitute any commitment, express or implied, to maintaining the ext2fs module beyond the change being considered right now. I find that this often has to be reiterated as people are prone to confusing the concepts open source and free, basic economics dictates infinite demand for free goods, and we've all got lives to live. As per our off-list discussion: It's a damned if we do and damned if we don't situation. Take the patch and it eats someone's lunch, and our reptuation suffers. Don't take the patch and look like patriarchal killjoys, and our reputation siffers. Your specific objection is that the testing is insufficient to exercise the patch, and there could be an area of ext2 which this patch doesn't address. That can never be said with 100% certainty, but I agree with you. Content free argument: Based on my reading of the code, the patch must be considered experimental. Whilst the scope of the patch appears to be small, the symbol space of ext2 is bigger -- a case of feeping creaturism due to ext2 itself, but hey, that's evolution for you. If folk are happy with it going in, let it go in, but remember, you get the system you apply effort to. I myself consider the patch experimental -- but HEAD is an experiment, is it not? Reality is what you can get away with. cheers BMS ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ext2 inode size patch - RE: PR kern/124621
Could you please point me to your patch and an explanation on how to apply it and test it? You can grab the patch here: http://pflog.net/~floyd/ext2fs.diff To apply it: cd /usr/src/sys/gnu/fs patch /path/to/ext2fs.diff cd /usr/src/sys/modules/ext2fs make clean make kldload ./ext2fs.ko Then umount and mount again your ext2 file systems. This should apply cleanly to RELENG_7_0, RELENG_7_1 and RELENG_7 source. I'm not sure if it'll apply cleanly to -CURRENT or not (I can provide an updated patch if you need it). Note: if you have ext2fs built into your kernel, you'll have to build and install your kernel as usual after patching, instead of building the module separately. Also, if you already have ext2fs loaded, you'll need to kldunload it first of course. If you want to update the ext2fs.ko in your installed kernel in /boot/kernel, after a make in .../modules/ext2fs, you can make install. Thanks, Josh ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ext2 inode size patch - RE: PR kern/124621
On Wednesday 03 December 2008 8:53:43 pm Josh Carroll wrote: Ok, I describe my concern once more. I do not object against the checking of the inode size. But, if inode size is changed, then some data is added to the inode, that could (and usually does, otherwise why extend it ?) change intrerpetation of the inode. Thus, we need a verification of the fact that simply ignoring added fields does not damage filesystem or cause user data corruption. Verification != testing. Let me take another crack at explaining why I think this patch is not dangerous. The inode size is determined by reading the following member: __u16 s_inode_size; of the ext2_super_block structure. I took a look at the Linux 2.6.27.7 kernel source, and indeed they do something very similar if not the same: #define EXT2_INODE_SIZE(s) (EXT2_SB(s)-s_inode_size) If you compare to what I did: #define EXT2_INODE_SIZE(s) ((s)-u.ext2_sb.s_inode_size) This is really the same thing, since EXT2_SB is defined (in the Linux kernel src as): #ifdef __KERNEL__ #include linux/ext2_fs_sb.h static inline struct ext2_sb_info *EXT2_SB(struct super_block *sb) { return sb-s_fs_info; } And struct ext2_sb_info has the following member: int s_inode_size; Essentially, the changes I made simply query the existing information from the filesystem, which is what the Linux kernel does as well. The inode size, blocks per group, etc are all defined at filesystem creation time by mke2fs and it ensures the sanity of the relationship between the inodes/blocks/block groups. The first diagram here: http://sunsite.nus.sg/LDP/LDP/tlk/node95.html Makes it clear that as long as the number of inodes per block and the blocks per group is is sane during fs creation, looking up the inode size as my patch does is fine, since the creation of the filesystem is ensures a correct by construction situation. We're simply reading the size of the inode based on the filesystem. I hope this is sufficient to convince some further thought about committing this. For those interested in the relevant Linux kernel source, you can have a look at line 105 of include/linux/ext2_fs.h from the linux-2.6.27.7 kernel source. Thanks, Josh Could you please point me to your patch and an explanation on how to apply it and test it? I've been following your las emails referencing it ( on @questions and @hackers or current i think it was ... ) and I'd like to give it a spin in here ... I've followed the can't mount ext2/3 problem for a time (since I have that problem) and would like to know to what extent for patch solves the problem. Here are some of the references: mounting linux partitions Fri May 9 18:05:26 UTC 2008 http://lists.freebsd.org/pipermail/freebsd-questions/2008-May/174588.html bad file descriptor when mounting an ext2fs. Tue Jun 10 11:08:46 UTC 2008 http://lists.freebsd.org/pipermail/freebsd-questions/2008-June/176506.html mounting ext2fs partitions on FBSD7 ( third time a charm?) Fri Jul 4 23:33:53 UTC 2008 http://lists.freebsd.org/pipermail/freebsd-questions/2008-July/178219.html My case: [EMAIL PROTECTED]:~ # ls -l /dev/ad4s ad4s1% ad4s2% ad4s3% ad4s3a% ad4s3b% ad4s3c% ad4s3d% ad4s3e% ad4s3f% ad4s4% ad4s5% ad4s6% ad4s7% ad4s8% [EMAIL PROTECTED]:~ # ls -l /dev/ad4s [EMAIL PROTECTED]:~ # tune2fs -l /dev/ad4s1 | grep Inode size Inode size: 256 [EMAIL PROTECTED]:~ # tune2fs -l /dev/ad4s6 | grep Inode size Inode size: 256 [EMAIL PROTECTED]:~ # tune2fs -l /dev/ad4s7 | grep Inode size Inode size: 256 [EMAIL PROTECTED]:~ # tune2fs -l /dev/ad4s8 | grep Inode size Inode size: 256 [EMAIL PROTECTED]:~ # tune2fs -l /dev/ad4s9 | grep Inode size BTW: I'm willing to run any tests you need me too, even if they imply a serious risk of loosing data on the 256 inode partitions. Regards -- Blessings Gonzalo Nemmi ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ext2 inode size patch - RE: PR kern/124621
Ok, I describe my concern once more. I do not object against the checking of the inode size. But, if inode size is changed, then some data is added to the inode, that could (and usually does, otherwise why extend it ?) change intrerpetation of the inode. Thus, we need a verification of the fact that simply ignoring added fields does not damage filesystem or cause user data corruption. Verification != testing. Let me take another crack at explaining why I think this patch is not dangerous. The inode size is determined by reading the following member: __u16 s_inode_size; of the ext2_super_block structure. I took a look at the Linux 2.6.27.7 kernel source, and indeed they do something very similar if not the same: #define EXT2_INODE_SIZE(s) (EXT2_SB(s)-s_inode_size) If you compare to what I did: #define EXT2_INODE_SIZE(s) ((s)-u.ext2_sb.s_inode_size) This is really the same thing, since EXT2_SB is defined (in the Linux kernel src as): #ifdef __KERNEL__ #include linux/ext2_fs_sb.h static inline struct ext2_sb_info *EXT2_SB(struct super_block *sb) { return sb-s_fs_info; } And struct ext2_sb_info has the following member: int s_inode_size; Essentially, the changes I made simply query the existing information from the filesystem, which is what the Linux kernel does as well. The inode size, blocks per group, etc are all defined at filesystem creation time by mke2fs and it ensures the sanity of the relationship between the inodes/blocks/block groups. The first diagram here: http://sunsite.nus.sg/LDP/LDP/tlk/node95.html Makes it clear that as long as the number of inodes per block and the blocks per group is is sane during fs creation, looking up the inode size as my patch does is fine, since the creation of the filesystem is ensures a correct by construction situation. We're simply reading the size of the inode based on the filesystem. I hope this is sufficient to convince some further thought about committing this. For those interested in the relevant Linux kernel source, you can have a look at line 105 of include/linux/ext2_fs.h from the linux-2.6.27.7 kernel source. Thanks, Josh ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ext2 inode size patch - RE: PR kern/124621
Hi, The inode size for the ext3 filesystem which Gentoo created for my last install defaulted to 256 bytes, so I got bit by this problem. I can't speak for the write path. but the read path looks just fine to me, and the patch should go in ASAP. Josh Carroll wrote: Ok, I describe my concern once more. I do not object against the checking of the inode size. But, if inode size is changed, then some data is added to the inode, that could (and usually does, otherwise why extend it ?) change intrerpetation of the inode. Thus, we need a verification of the fact that simply ignoring added fields does not damage filesystem or cause user data corruption. Verification != testing. If folk are paranoid, then add a check for dynamic inode size and disable ext2fs writes by downgrading the mount in that case (We can do that, right? Can someone make sure Josh gets the help he needs here?) As Josh points out, the ext2 inode size is stored in the superblock. Whilst it may vary between ext2 filesystems, *the inode size itself does not appear to be something which one can modify in an existing ext2/3 filesystem*. Older ext2 filesystems may not contain the inode size field in the superblock, and the patch appears to default to 128 for that case. The double indirection thus introduced doesn't concern me, our ext2fs is not performance critical code, and the superblock is likely to sit in L2/L3 cache anyway (note: content free argument). Thanks to Josh for fixing this problem. cheers BMS ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ext2 inode size patch - RE: PR kern/124621
On Mon, Nov 24, 2008 at 02:29:57PM -0500, Josh Carroll wrote: A while back, I submitted a patch for PR kern/124621, which allows the mounting of an ext2(3) filesystem created with an inode size other than 128. The e2fsprogs' default is now 256, so file systems created on newer Linux distributions or with the port will not be mountable. I was hopeful this would get committed in time for 7.1-RELEASE (and 6.4-RELEASE), however the PR remains open. If there is an issue with the patch itself, I would be glad to fix it. I'm posting to fs@ because hopefully some folks more experienced with file system/kernel code can have a look and see if the patch is ok to commit. I've seen a few people in ##freebsdhelp on Freenode as well as #freebsdhelp on EFnet with this problem, and have had them test this patch out with success (and no obvious adverse effects), so I was hoping it could committed in time for 7.1-RELEASE. Since 6.4 is so close to release, I'm not so sure about that. Anyway, I would appreciate it if the patch could get some review to see if it can be committed in time. I already expressed my opinion on http://lists.freebsd.org/pipermail/freebsd-hackers/2008-September/025933.html pgpyTQ1gPVpvO.pgp Description: PGP signature
Re: ext2 inode size patch - RE: PR kern/124621
I already expressed my opinion on http://lists.freebsd.org/pipermail/freebsd-hackers/2008-September/025933.html Sorry, I do not subscribe to hackers@ so I did not see that message. So what do you recommend is done to further test it? I tested simple things like copies, writes, deletes, etc on a memory disk, but nothing formal. I don't (currently) have a spare blank disk or space on an existing disk to test on physical media, but I can look into doing so once my development box is back up and running. I'm also curious what about the changes you feel are dangerous, so I can target the testing to exercise the boundary conditions or circumstances you think this patch would elicit problems. Thanks, Josh ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ext2 inode size patch - RE: PR kern/124621
On Tue, Nov 25, 2008 at 09:17:06AM -0500, Josh Carroll wrote: I already expressed my opinion on http://lists.freebsd.org/pipermail/freebsd-hackers/2008-September/025933.html Sorry, I do not subscribe to hackers@ so I did not see that message. So what do you recommend is done to further test it? I tested simple things like copies, writes, deletes, etc on a memory disk, but nothing formal. I don't (currently) have a spare blank disk or space on an existing disk to test on physical media, but I can look into doing so once my development box is back up and running. I'm also curious what about the changes you feel are dangerous, so I can target the testing to exercise the boundary conditions or circumstances you think this patch would elicit problems. I do not suggest testing. I suggest understand what inode metadata is stored in the added 128 bytes and evaluate whether this information can be ignored without dangerous consequences for filesystem consistency or user data. pgpDeHjQMpI8R.pgp Description: PGP signature
Re: ext2 inode size patch - RE: PR kern/124621
On Tue, Nov 25, 2008 at 09:57:18AM -0500, Josh Carroll wrote: I do not suggest testing. I suggest understand what inode metadata is stored in the added 128 bytes and evaluate whether this information can be ignored without dangerous consequences for filesystem consistency or user data. Well, to be clear I didn't just double the size of the inode table. It is dynamically determined based on the data structure. I'm not a file system expert (to call me a novice would probably be stretching it), so I'm hoping someone more versed can chime in. All the code does is query the data structure (specifically, the s_inode_size field of the structure) and use that value instead of blindly assuming an inode size of 128. I don't think it's a matter of what is done with the extra bits, since it's just querying the size of an already created filesystem. Ok, I describe my concern once more. I do not object against the checking of the inode size. But, if inode size is changed, then some data is added to the inode, that could (and usually does, otherwise why extend it ?) change intrerpetation of the inode. Thus, we need a verification of the fact that simply ignoring added fields does not damage filesystem or cause user data corruption. Verification != testing. Until we make this work, patch cannot go into the tree. pgpfHlW3Qa7Vi.pgp Description: PGP signature
Re: ext2 inode size patch - RE: PR kern/124621
I do not suggest testing. I suggest understand what inode metadata is stored in the added 128 bytes and evaluate whether this information can be ignored without dangerous consequences for filesystem consistency or user data. Well, to be clear I didn't just double the size of the inode table. It is dynamically determined based on the data structure. I'm not a file system expert (to call me a novice would probably be stretching it), so I'm hoping someone more versed can chime in. All the code does is query the data structure (specifically, the s_inode_size field of the structure) and use that value instead of blindly assuming an inode size of 128. I don't think it's a matter of what is done with the extra bits, since it's just querying the size of an already created filesystem. Thanks, Josh ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ext2 inode size patch - RE: PR kern/124621
Ok, I describe my concern once more. I do not object against the checking of the inode size. But, if inode size is changed, then some data is added to the inode, that could (and usually does, otherwise why extend it ?) change intrerpetation of the inode. Thus, we need a verification of the fact that simply ignoring added fields does not damage filesystem or cause user data corruption. Verification != testing. Ok, I see your point. I will do some more research into the ext2 inode structure on disk and see what happens when inode size 128. Until we make this work, patch cannot go into the tree. Understood, thanks for your attention. Regards, Josh ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ext2 inode size patch - RE: PR kern/124621
On 2008-11-25 10:11:09AM -0500, Josh Carroll wrote: Ok, I describe my concern once more. I do not object against the checking of the inode size. But, if inode size is changed, then some data is added to the inode, that could (and usually does, otherwise why extend it ?) change intrerpetation of the inode. Thus, we need a verification of the fact that simply ignoring added fields does not damage filesystem or cause user data corruption. Verification != testing. Ok, I see your point. I will do some more research into the ext2 inode structure on disk and see what happens when inode size 128. Possibly overstating the obvious, but since e2fsprogs were the ones who actually initiated the change in default inode size, maybe start digging through that to see what it actually does with the other 128 bytes (the changelog and some posts on comp.os.linux seem to suggest is that it has something to do with optimizing extended attributes/acls; basically extended acls being inaccessible to kernels that ignore the extra data, and 2.4 kernels refusing to mount those filesystems at all (presumably due to the same assumption we've been making)). -- === Peter C. Lai | Bard College at Simon's Rock Systems Administrator| 84 Alford Rd. Information Technology Svcs. | Gt. Barrington, MA 01230 USA peter AT simons-rock.edu | (413) 528-7428 === ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
ext2 inode size patch - RE: PR kern/124621
A while back, I submitted a patch for PR kern/124621, which allows the mounting of an ext2(3) filesystem created with an inode size other than 128. The e2fsprogs' default is now 256, so file systems created on newer Linux distributions or with the port will not be mountable. I was hopeful this would get committed in time for 7.1-RELEASE (and 6.4-RELEASE), however the PR remains open. If there is an issue with the patch itself, I would be glad to fix it. I'm posting to fs@ because hopefully some folks more experienced with file system/kernel code can have a look and see if the patch is ok to commit. I've seen a few people in ##freebsdhelp on Freenode as well as #freebsdhelp on EFnet with this problem, and have had them test this patch out with success (and no obvious adverse effects), so I was hoping it could committed in time for 7.1-RELEASE. Since 6.4 is so close to release, I'm not so sure about that. Anyway, I would appreciate it if the patch could get some review to see if it can be committed in time. Regards, Josh ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]