Re: [PATCH 1/2] ext4: Add support for 48 bit inode i_blocks.
Andreas Dilger wrote:
On Oct 12, 2007 10:06 +0530, Aneesh Kumar K.V wrote:
use the __le16 l_i_reserved1 field of the linux2
struct of ext4_inode to represet the higher 16
bits for i_blocks. With this change max_file size becomes
(2**48 -1 )* 512 bytes.
+static int ext4_inode_blocks_set(handle_t *handle,
+ struct ext4_inode *raw_inode,
+ struct ext4_inode_info *ei)
CodingStyle would suggest aligning the continued lines with the '('.
Will fix
+{
+ } else if (i_blocks <= 0xULL) {
+ if (!EXT4_HAS_RO_COMPAT_FEATURE(sb,
+ EXT4_FEATURE_RO_COMPAT_HUGE_FILE)) {
+
+ err = ext4_journal_get_write_access(handle,
+ EXT4_SB(sb)->s_sbh);
+ if (err)
+ goto err_out;
+ ext4_update_dynamic_rev(sb);
+ EXT4_SET_RO_COMPAT_FEATURE(sb,
+ EXT4_FEATURE_RO_COMPAT_HUGE_FILE);
+ sb->s_dirt = 1;
+ handle->h_sync = 1;
+ err = ext4_journal_dirty_metadata(handle,
+ EXT4_SB(sb)->s_sbh);
+ }
Can you please make a helper function for this, like:
int ext4_update_feature(sb, __u32 compat, __u32 rocompat, __u32 incompat)
as we have similar code in a few places already (HTREE, LARGE_FILE, EXTENTS).
That could be done in a prerequisite patch.
Will do
+ if (EXT4_HAS_RO_COMPAT_FEATURE(sb, EXT4_FEATURE_RO_COMPAT_HUGE_FILE)) {
+ /*
+* Large file size enabled file system can only be
+* mount if kernel is build with CONFIG_LSF
+*/
+ if (sizeof(root->i_blocks) < sizeof(u64) &&
+ !(sb->s_flags & MS_RDONLY)) {
+ printk(KERN_ERR "EXT4-fs: %s: Having huge file with "
+ "LSF off, you must mount filesystem "
+ "read-only.\n", sb->s_id);
What do you think about changing the wording:
"Filesystem with huge files cannot mount read-write without CONFIG_LSF."
#define EXT4_FEATURE_INCOMPAT_EXTENTS 0x0040 /* extents support */
#define EXT4_FEATURE_INCOMPAT_64BIT0x0080
+#define EXT4_FEATURE_INCOMPAT_MMP 0x0100
#define EXT4_FEATURE_INCOMPAT_FLEX_BG 0x0200
Note that it is fine to add the #define for this flag.
EXT4_FEATURE_INCOMPAT_EXTENTS| \
EXT4_FEATURE_INCOMPAT_64BIT| \
+EXT4_FEATURE_INCOMPAT_MMP|\
EXT4_FEATURE_INCOMPAT_FLEX_BG)
Note it is NOT OK to add INCOMPAT_MMP to the INCOMPAT_SUPP flags, or you
defeat the entire purpose of having the feature flag.
Will fix.
-aneesh
-
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/2] ext4: Add support for 48 bit inode i_blocks.
On Oct 12, 2007 10:06 +0530, Aneesh Kumar K.V wrote:
> use the __le16 l_i_reserved1 field of the linux2
> struct of ext4_inode to represet the higher 16
> bits for i_blocks. With this change max_file size becomes
> (2**48 -1 )* 512 bytes.
>
> +static int ext4_inode_blocks_set(handle_t *handle,
> + struct ext4_inode *raw_inode,
> + struct ext4_inode_info *ei)
CodingStyle would suggest aligning the continued lines with the '('.
> +{
> + } else if (i_blocks <= 0xULL) {
> + if (!EXT4_HAS_RO_COMPAT_FEATURE(sb,
> + EXT4_FEATURE_RO_COMPAT_HUGE_FILE)) {
> +
> + err = ext4_journal_get_write_access(handle,
> + EXT4_SB(sb)->s_sbh);
> + if (err)
> + goto err_out;
> + ext4_update_dynamic_rev(sb);
> + EXT4_SET_RO_COMPAT_FEATURE(sb,
> + EXT4_FEATURE_RO_COMPAT_HUGE_FILE);
> + sb->s_dirt = 1;
> + handle->h_sync = 1;
> + err = ext4_journal_dirty_metadata(handle,
> + EXT4_SB(sb)->s_sbh);
> + }
Can you please make a helper function for this, like:
int ext4_update_feature(sb, __u32 compat, __u32 rocompat, __u32 incompat)
as we have similar code in a few places already (HTREE, LARGE_FILE, EXTENTS).
That could be done in a prerequisite patch.
> + if (EXT4_HAS_RO_COMPAT_FEATURE(sb, EXT4_FEATURE_RO_COMPAT_HUGE_FILE)) {
> + /*
> + * Large file size enabled file system can only be
> + * mount if kernel is build with CONFIG_LSF
> + */
> + if (sizeof(root->i_blocks) < sizeof(u64) &&
> + !(sb->s_flags & MS_RDONLY)) {
> + printk(KERN_ERR "EXT4-fs: %s: Having huge file with "
> + "LSF off, you must mount filesystem "
> + "read-only.\n", sb->s_id);
What do you think about changing the wording:
"Filesystem with huge files cannot mount read-write without CONFIG_LSF."
> #define EXT4_FEATURE_INCOMPAT_EXTENTS0x0040 /* extents
> support */
> #define EXT4_FEATURE_INCOMPAT_64BIT 0x0080
> +#define EXT4_FEATURE_INCOMPAT_MMP 0x0100
> #define EXT4_FEATURE_INCOMPAT_FLEX_BG0x0200
Note that it is fine to add the #define for this flag.
>EXT4_FEATURE_INCOMPAT_EXTENTS| \
>EXT4_FEATURE_INCOMPAT_64BIT| \
> + EXT4_FEATURE_INCOMPAT_MMP|\
>EXT4_FEATURE_INCOMPAT_FLEX_BG)
Note it is NOT OK to add INCOMPAT_MMP to the INCOMPAT_SUPP flags, or you
defeat the entire purpose of having the feature flag.
Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.
-
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/2] ext4: Add support for 48 bit inode i_blocks.
use the __le16 l_i_reserved1 field of the linux2
struct of ext4_inode to represet the higher 16
bits for i_blocks. With this change max_file size becomes
(2**48 -1 )* 512 bytes.
Signed-off-by: Aneesh Kumar K.V <[EMAIL PROTECTED]>
---
fs/ext4/inode.c | 69 +-
fs/ext4/super.c | 62 ++
include/linux/ext4_fs.h | 11 +--
3 files changed, 131 insertions(+), 11 deletions(-)
diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 218eec9..41af2d3 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -2706,6 +2706,22 @@ void ext4_get_inode_flags(struct ext4_inode_info *ei)
if (flags & S_DIRSYNC)
ei->i_flags |= EXT4_DIRSYNC_FL;
}
+static blkcnt_t ext4_inode_blocks(struct ext4_inode *raw_inode,
+ struct ext4_inode_info *ei)
+{
+ blkcnt_t i_blocks ;
+ struct super_block *sb = ei->vfs_inode.i_sb;
+
+ if (EXT4_HAS_RO_COMPAT_FEATURE(sb,
+ EXT4_FEATURE_RO_COMPAT_HUGE_FILE)) {
+ /* we are using combined 48 bit field */
+ i_blocks = ((u64)le16_to_cpu(raw_inode->i_blocks_high)) << 32 |
+
le32_to_cpu(raw_inode->i_blocks_lo);
+ return i_blocks;
+ } else {
+ return le32_to_cpu(raw_inode->i_blocks_lo);
+ }
+}
void ext4_read_inode(struct inode * inode)
{
@@ -2755,8 +2771,8 @@ void ext4_read_inode(struct inode * inode)
* recovery code: that's fine, we're about to complete
* the process of deleting those. */
}
- inode->i_blocks = le32_to_cpu(raw_inode->i_blocks);
ei->i_flags = le32_to_cpu(raw_inode->i_flags);
+ inode->i_blocks = ext4_inode_blocks(raw_inode, ei);
ei->i_file_acl = le32_to_cpu(raw_inode->i_file_acl);
if (EXT4_SB(inode->i_sb)->s_es->s_creator_os !=
cpu_to_le32(EXT4_OS_HURD))
@@ -2850,6 +2866,54 @@ bad_inode:
return;
}
+static int ext4_inode_blocks_set(handle_t *handle,
+ struct ext4_inode *raw_inode,
+ struct ext4_inode_info *ei)
+{
+ struct inode *inode = &(ei->vfs_inode);
+ u64 i_blocks = inode->i_blocks;
+ struct super_block *sb = inode->i_sb;
+ int err = 0;
+
+ if (i_blocks <= ~0U) {
+ /*
+* i_blocks can be represnted in a 32 bit variable
+* as multiple of 512 bytes
+*/
+ raw_inode->i_blocks_lo = cpu_to_le32((u32)i_blocks);
+ raw_inode->i_blocks_high = 0;
+ } else if (i_blocks <= 0xULL) {
+ /*
+* i_blocks can be represented in a 48 bit variable
+* as multiple of 512 bytes
+*/
+ if (!EXT4_HAS_RO_COMPAT_FEATURE(sb,
+ EXT4_FEATURE_RO_COMPAT_HUGE_FILE)) {
+
+ err = ext4_journal_get_write_access(handle,
+ EXT4_SB(sb)->s_sbh);
+ if (err)
+ goto err_out;
+ ext4_update_dynamic_rev(sb);
+ EXT4_SET_RO_COMPAT_FEATURE(sb,
+ EXT4_FEATURE_RO_COMPAT_HUGE_FILE);
+ sb->s_dirt = 1;
+ handle->h_sync = 1;
+ err = ext4_journal_dirty_metadata(handle,
+ EXT4_SB(sb)->s_sbh);
+ }
+ /* i_block is stored in the split 48 bit fields */
+ raw_inode->i_blocks_lo = cpu_to_le32((u32)i_blocks);
+ raw_inode->i_blocks_high = cpu_to_le16(i_blocks >> 32);
+ }else {
+ ext4_error(sb, __FUNCTION__,
+ "Wrong inode i_blocks count %llu\n",
+ (unsigned long long)inode->i_blocks);
+ }
+err_out:
+ return err;
+}
+
/*
* Post the struct inode info into an on-disk inode location in the
* buffer-cache. This gobbles the caller's reference to the
@@ -2905,7 +2969,8 @@ static int ext4_do_update_inode(handle_t *handle,
EXT4_INODE_SET_XTIME(i_atime, inode, raw_inode);
EXT4_EINODE_SET_XTIME(i_crtime, ei, raw_inode);
- raw_inode->i_blocks = cpu_to_le32(inode->i_blocks);
+ if (ext4_inode_blocks_set(handle, raw_inode, ei))
+ goto out_brelse;
raw_inode->i_dtime = cpu_to_le32(ei->i_dtime);
raw_inode->i_flags = cpu_to_le32(ei->i_flags);
if (EXT4_SB(inode->i_sb)->s_es->s_creator_os !=
diff --git a/fs/ext4/super.c b/fs/ext4/super.c
index 9dc37ba..a7f3edf 100644
--- a/fs/ext4/super.c
+++ b/fs/ext4/super.c
@@ -1503,17 +1503,50 @@ static void ext4_orphan_cleanup (struct super_block *
sb,
/*
* Maximal fil
[PATCH 2/2] ext4: Support large files
This patch converts ext4_inode i_blocks to represent total
blocks occupied by the inode in file system block size.
Earlier the variable used to represent this in 512 byte
block size. This actually limited the total size of the file.
The feature is enabled transparently when we write an inode
whose i_blocks cannot be represnted as 512 byte units in a
48 bit variable. We add a RO_COMPAT feature to the super
block to indicate that some of the inode have i_blocks
represented as file system block size units. Super block
with this feature set cannot be mounted read write on a kernel
with CONFIG_LSF disabled.
Super block flag EXT4_FEATURE_RO_COMPAT_HUGE_FILE
inode flag EXT4_HUGE_FILE_FL
Signed-off-by: Aneesh Kumar K.V <[EMAIL PROTECTED]>
---
fs/ext4/inode.c | 41 +++--
fs/ext4/super.c |9 ++---
include/linux/ext4_fs.h |3 ++-
3 files changed, 43 insertions(+), 10 deletions(-)
diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 41af2d3..7fa1868 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -2710,14 +2710,20 @@ static blkcnt_t ext4_inode_blocks(struct ext4_inode
*raw_inode,
struct ext4_inode_info *ei)
{
blkcnt_t i_blocks ;
- struct super_block *sb = ei->vfs_inode.i_sb;
+ struct inode *inode = &(ei->vfs_inode);
+ struct super_block *sb = inode->i_sb;
if (EXT4_HAS_RO_COMPAT_FEATURE(sb,
EXT4_FEATURE_RO_COMPAT_HUGE_FILE)) {
/* we are using combined 48 bit field */
i_blocks = ((u64)le16_to_cpu(raw_inode->i_blocks_high)) << 32 |
le32_to_cpu(raw_inode->i_blocks_lo);
- return i_blocks;
+ if (ei->i_flags & EXT4_HUGE_FILE_FL) {
+ /* i_blocks represent file system block size */
+ return i_blocks << (inode->i_blkbits - 9);
+ } else {
+ return i_blocks;
+ }
} else {
return le32_to_cpu(raw_inode->i_blocks_lo);
}
@@ -2882,6 +2888,7 @@ static int ext4_inode_blocks_set(handle_t *handle,
*/
raw_inode->i_blocks_lo = cpu_to_le32((u32)i_blocks);
raw_inode->i_blocks_high = 0;
+ ei->i_flags &= ~EXT4_HUGE_FILE_FL;
} else if (i_blocks <= 0xULL) {
/*
* i_blocks can be represented in a 48 bit variable
@@ -2905,10 +2912,32 @@ static int ext4_inode_blocks_set(handle_t *handle,
/* i_block is stored in the split 48 bit fields */
raw_inode->i_blocks_lo = cpu_to_le32((u32)i_blocks);
raw_inode->i_blocks_high = cpu_to_le16(i_blocks >> 32);
- }else {
- ext4_error(sb, __FUNCTION__,
- "Wrong inode i_blocks count %llu\n",
- (unsigned long long)inode->i_blocks);
+ ei->i_flags &= ~EXT4_HUGE_FILE_FL;
+ } else {
+ /*
+* i_blocks should be represented in a 48 bit variable
+* as multiple of file system block size
+*/
+ if (!EXT4_HAS_RO_COMPAT_FEATURE(sb,
+ EXT4_FEATURE_RO_COMPAT_HUGE_FILE)) {
+
+ err = ext4_journal_get_write_access(handle,
+ EXT4_SB(sb)->s_sbh);
+ if (err)
+ goto err_out;
+ ext4_update_dynamic_rev(sb);
+ EXT4_SET_RO_COMPAT_FEATURE(sb,
+ EXT4_FEATURE_RO_COMPAT_HUGE_FILE);
+ sb->s_dirt = 1;
+ handle->h_sync = 1;
+ err = ext4_journal_dirty_metadata(handle,
+ EXT4_SB(sb)->s_sbh);
+ }
+ ei->i_flags |= EXT4_HUGE_FILE_FL;
+ /* i_block is stored in file system block size */
+ i_blocks = i_blocks >> (inode->i_blkbits - 9);
+ raw_inode->i_blocks_lo = cpu_to_le32((u32)i_blocks);
+ raw_inode->i_blocks_high = cpu_to_le16(i_blocks >> 32);
}
err_out:
return err;
diff --git a/fs/ext4/super.c b/fs/ext4/super.c
index a7f3edf..ea3b2c3 100644
--- a/fs/ext4/super.c
+++ b/fs/ext4/super.c
@@ -1531,11 +1531,14 @@ static loff_t ext4_max_size(int bits)
upper_limit >>= (bits - 9);
} else {
- /* We use 48 bit ext4_inode i_blocks */
+ /*
+* We use 48 bit ext4_inode i_blocks
+* With EXT4_HUGE_FILE_FL set the i_blocks
+* represent total number of blocks in
+* file system block size
+*/
upper_limit = (1LL << 48) - 1;
- /*
Re: [PATCH] ext2: Fix the max file size for ext2 file system.
Andreas Dilger wrote: On Oct 11, 2007 20:20 +0530, Aneesh Kumar K.V wrote: + /* indirect blocks */ + meta_blocks = 1; + /* double indirect blocks */ + meta_blocks += 1 + (1LL << (bits-2)); + /* tripple indirect blocks */ + meta_blocks += 1 + (1LL << (bits-2)) + (1LL << (2*(bits-2))); It may be worthwhile calculating how many indirect blocks there actually are in a file that size. The math become more confusing for that right ?. I am not sure we need to be that specific. Also note that your comments are backward - there is at most a single triple indirect block per file. What i meant by those comment was, the count of meta data blocks taken for different type of indirect blocks. -aneesh - To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
- jbd2-debug-code-cleanup.patch removed from -mm tree
The patch titled
JBD2: debug code cleanup
has been removed from the -mm tree. Its filename was
jbd2-debug-code-cleanup.patch
This patch was dropped because it was merged into mainline or a subsystem tree
--
Subject: JBD2: debug code cleanup
From: Jose R. Santos <[EMAIL PROTECTED]>
JBD2: debug code cleanup.
Mostly stolen from akpm's JBD cleanup patch.
- use `#ifdef foo' instead of `#if defined(foo)'
- Make journal_enable_debug __read_mostly just for the heck of it
- Make jbd_debugfs_dir and jbd_debug static
- debugfs_remove(NULL) is legal: remove unneeded tests
- remove unnecessary empty loops
Signed-off-by: Jose R. Santos <[EMAIL PROTECTED]>
Cc:
Signed-off-by: Andrew Morton <[EMAIL PROTECTED]>
---
diff -puN fs/jbd2/journal.c~jbd2-debug-code-cleanup fs/jbd2/journal.c
--- a/fs/jbd2/journal.c~jbd2-debug-code-cleanup
+++ a/fs/jbd2/journal.c
@@ -2199,16 +2199,14 @@ void jbd2_journal_put_journal_head(struc
/*
* debugfs tunables
*/
-#if defined(CONFIG_JBD2_DEBUG)
-u8 jbd2_journal_enable_debug;
+#ifdef CONFIG_JBD2_DEBUG
+u8 jbd2_journal_enable_debug __read_mostly;
EXPORT_SYMBOL(jbd2_journal_enable_debug);
-#endif
-
-#if defined(CONFIG_JBD2_DEBUG) && defined(CONFIG_DEBUG_FS)
#define JBD2_DEBUG_NAME "jbd2-debug"
-struct dentry *jbd2_debugfs_dir, *jbd2_debug;
+static struct dentry *jbd2_debugfs_dir;
+static struct dentry *jbd2_debug;
static void __init jbd2_create_debugfs_entry(void)
{
@@ -2221,24 +2219,18 @@ static void __init jbd2_create_debugfs_e
static void __exit jbd2_remove_debugfs_entry(void)
{
- if (jbd2_debug)
- debugfs_remove(jbd2_debug);
- if (jbd2_debugfs_dir)
- debugfs_remove(jbd2_debugfs_dir);
+ debugfs_remove(jbd2_debug);
+ debugfs_remove(jbd2_debugfs_dir);
}
#else
static void __init jbd2_create_debugfs_entry(void)
{
- do {
- } while (0);
}
static void __exit jbd2_remove_debugfs_entry(void)
{
- do {
- } while (0);
}
#endif
_
Patches currently in -mm which might be from [EMAIL PROTECTED] are
ext4-flex_bg-kernel-support-v2.patch
jbd-config_jbd_debug-cannot-create-proc-entry.patch
jbd-config_jbd_debug-cannot-create-proc-entry-fix.patch
jbd-fix-jbd-warnings-when-compiling-with-config_jbd_debug.patch
-
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
- jbd2-fix-commit-code-to-properly-abort-journal.patch removed from -mm tree
The patch titled
jbd2: fix commit code to properly abort journal
has been removed from the -mm tree. Its filename was
jbd2-fix-commit-code-to-properly-abort-journal.patch
This patch was dropped because it was merged into mainline or a subsystem tree
--
Subject: jbd2: fix commit code to properly abort journal
From: Jan Kara <[EMAIL PROTECTED]>
We should really call journal_abort() and not __journal_abort_hard() in
case of errors. The latter call does not record the error in the journal
superblock and thus filesystem won't be marked as with errors later (and
user could happily mount it without any warning).
Signed-off-by: Jan Kara <[EMAIL PROTECTED]>
Cc:
Signed-off-by: Andrew Morton <[EMAIL PROTECTED]>
---
fs/jbd2/commit.c |8
1 file changed, 4 insertions(+), 4 deletions(-)
diff -puN fs/jbd2/commit.c~jbd2-fix-commit-code-to-properly-abort-journal
fs/jbd2/commit.c
--- a/fs/jbd2/commit.c~jbd2-fix-commit-code-to-properly-abort-journal
+++ a/fs/jbd2/commit.c
@@ -539,7 +539,7 @@ void jbd2_journal_commit_transaction(jou
spin_unlock(&journal->j_list_lock);
if (err)
- __jbd2_journal_abort_hard(journal);
+ jbd2_journal_abort(journal, err);
jbd2_journal_write_revoke_records(journal, commit_transaction);
@@ -602,7 +602,7 @@ void jbd2_journal_commit_transaction(jou
descriptor =
jbd2_journal_get_descriptor_buffer(journal);
if (!descriptor) {
- __jbd2_journal_abort_hard(journal);
+ jbd2_journal_abort(journal, -EIO);
continue;
}
@@ -635,7 +635,7 @@ void jbd2_journal_commit_transaction(jou
and repeat this loop: we'll fall into the
refile-on-abort condition above. */
if (err) {
- __jbd2_journal_abort_hard(journal);
+ jbd2_journal_abort(journal, err);
continue;
}
@@ -860,7 +860,7 @@ wait_for_iobuf:
err = journal_wait_on_commit_record(cbh);
if (err)
- __jbd2_journal_abort_hard(journal);
+ jbd2_journal_abort(journal, err);
/* End of a transaction! Finally, we can do checkpoint
processing: any buffers committed as a result of this
_
Patches currently in -mm which might be from [EMAIL PROTECTED] are
send-quota-messages-via-netlink.patch
send-quota-messages-via-netlink-fix.patch
send-quota-messages-via-netlink-fix-fix.patch
udf-code-style-fixup-v3.patch
lockdep-give-each-filesystem-its-own-inode-lock-class.patch
jbd-config_jbd_debug-cannot-create-proc-entry.patch
jbd-config_jbd_debug-cannot-create-proc-entry-fix.patch
jbd-fix-commit-code-to-properly-abort-journal.patch
-
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/2] Make ext3 use bit operations to manipulate i_flags
On Oct 11, 2007 20:01 +0200, Jan Kara wrote: > #define EXT3_TOPDIR_FL 17 /* Top of directory > hierarchies*/ > +#define EXT3_RTIME_FL18 /* Update recursive mtime */ Could you please use "20" (0x10) for this flag, as there were patches to use 18 (0x4) for EXT4_HUGE_FILE_FL, and 19 (0x8) is definitely reserved for EXT4_EXTENTS_FL. I'd suggest emailing Ted to reserve a flag for this, and also adding the other reserved flags to ext2 and ext3 so there is no risk of conflicts. > @@ -572,6 +576,7 @@ static inline int ext3_valid_inum(struct > #define EXT3_FEATURE_COMPAT_EXT_ATTR 0x0008 > #define EXT3_FEATURE_COMPAT_RESIZE_INODE 0x0010 > #define EXT3_FEATURE_COMPAT_DIR_INDEX0x0020 > +#define EXT3_FEATURE_COMPAT_RTIME0x0040 Similarly, there is already #define EXT2_FEATURE_COMPAT_LAZY_BG 0x0040 in e2fsprogs so you need to use 0x0080 for your feature. Please update all of the ext*_fs.h files to have the most recent flags from e2fsprogs so that we can avoid such dangers in the future. Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc. - To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [patch 1/2] add init_ext4_proc() stub for when CONFIG_PROC_FS is not set
On Wed, 2007-10-10 at 15:00 -0400, Theodore Tso wrote: > On Wed, Oct 10, 2007 at 08:30:10AM -0700, Badari Pulavarty wrote: > > Its a good start. I think there are lots of proc handling routines that > > can be move into #ifdef CONFIG_PROC_FS also. > > > > All the code around ext4_mb_read_prealloc_table(), > > ext4_mb_write_prealloc_table(), MB_PROC_VALUE_READ(stats), > > MB_PROC_VALUE_WRITE(stats), .. can be ifdefed out. > > There's no need to ifdef them out; in include/proc_fs.h there are the > following convenience #define's if CONFIG_PROC_FS is not defined: Sorry. If I wasn't clear.. What I meant to say was the above routines are NOT needed if we don't define CONFIG_PROC_FS. They are supporting read/write to /proc files. We can #ifdef them out to reduce the text size. Thanks, Badari - To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] ext2: Fix the max file size for ext2 file system.
On Oct 11, 2007 20:20 +0530, Aneesh Kumar K.V wrote: > + /* indirect blocks */ > + meta_blocks = 1; > + /* double indirect blocks */ > + meta_blocks += 1 + (1LL << (bits-2)); > + /* tripple indirect blocks */ > + meta_blocks += 1 + (1LL << (bits-2)) + (1LL << (2*(bits-2))); It may be worthwhile calculating how many indirect blocks there actually are in a file that size. Also note that your comments are backward - there is at most a single triple indirect block per file. Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc. - To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 12/13][e2fsprogs] Add new mm_lazy test case.
From: Jose R. Santos <[EMAIL PROTECTED]> Add new mm_lazy test case. Add test case for lazy bg feature. Signed-off-by: Jose R. Santos <[EMAIL PROTECTED]> -- tests/m_lazy/expect.1 | 166 + tests/m_lazy/script |4 + 2 files changed, 170 insertions(+), 0 deletions(-) diff --git a/tests/m_lazy/expect.1 b/tests/m_lazy/expect.1 new file mode 100644 index 000..32ca764 --- /dev/null +++ b/tests/m_lazy/expect.1 @@ -0,0 +1,166 @@ +Filesystem label= +OS type: Linux +Block size=1024 (log=0) +Fragment size=1024 (log=0) +32768 inodes, 131072 blocks +6553 blocks (5.00%) reserved for the super user +First data block=1 +Maximum filesystem blocks=67371008 +16 block groups +8192 blocks per group, 8192 fragments per group +2048 inodes per group +Superblock backups stored on blocks: + 8193, 24577, 40961, 57345, 73729 + +Writing inode tables: done +Writing superblocks and filesystem accounting information: done + +Filesystem features: resize_inode dir_index lazy_bg filetype sparse_super + +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +test_filesys: 28683/32768 files (0.0% non-contiguous), 77097/131072 blocks +Exit status is 0 + +Filesystem volume name: +Last mounted on: +Filesystem magic number: 0xEF53 +Filesystem revision #:1 (dynamic) +Filesystem features: resize_inode dir_index lazy_bg filetype sparse_super +Default mount options:(none) +Filesystem state: clean +Errors behavior: Continue +Filesystem OS type: Linux +Inode count: 32768 +Block count: 131072 +Reserved block count: 6553 +Free blocks: 53975 +Free inodes: 4085 +First block: 1 +Block size: 1024 +Fragment size:1024 +Reserved GDT blocks: 256 +Blocks per group: 8192 +Fragments per group: 8192 +Inodes per group: 2048 +Inode blocks per group: 256 +Mount count: 0 +Check interval: 15552000 (6 months) +Reserved blocks uid: 0 +Reserved blocks gid: 0 +First inode: 11 +Inode size: 128 +Default directory hash: tea + + +Group 0: (Blocks 1-8192) + Primary superblock at 1, Group descriptors at 2-2 + Reserved GDT blocks at 3-258 + Block bitmap at 259 (+258), Inode bitmap at 260 (+259) + Inode table at 261-516 (+260) + 7662 free blocks, 2037 free inodes, 2 directories + Free blocks: 531-8192 + Free inodes: 12-2048 +Group 1: (Blocks 8193-16384) [Inode not init] + Backup superblock at 8193, Group descriptors at 8194-8194 + Reserved GDT blocks at 8195-8450 + Block bitmap at 8451 (+258), Inode bitmap at 8452 (+259) + Inode table at 8453-8708 (+260) + 7676 free blocks, 0 free inodes, 0 directories + Free blocks: 8709-16384 + Free inodes: +Group 2: (Blocks 16385-24576) [Inode not init, Block not init] + Block bitmap at 16385 (+0), Inode bitmap at 16386 (+1) + Inode table at 16387-16642 (+2) + 0 free blocks, 0 free inodes, 0 directories + Free blocks: + Free inodes: +Group 3: (Blocks 24577-32768) [Inode not init] + Backup superblock at 24577, Group descriptors at 24578-24578 + Reserved GDT blocks at 24579-24834 + Block bitmap at 24835 (+258), Inode bitmap at 24836 (+259) + Inode table at 24837-25092 (+260) + 7676 free blocks, 0 free inodes, 0 directories + Free blocks: 25093-32768 + Free inodes: +Group 4: (Blocks 32769-40960) [Inode not init, Block not init] + Block bitmap at 32769 (+0), Inode bitmap at 32770 (+1) + Inode table at 32771-33026 (+2) + 0 free blocks, 0 free inodes, 0 directories + Free blocks: + Free inodes: +Group 5: (Blocks 40961-49152) [Inode not init] + Backup superblock at 40961, Group descriptors at 40962-40962 + Reserved GDT blocks at 40963-41218 + Block bitmap at 41219 (+258), Inode bitmap at 41220 (+259) + Inode table at 41221-41476 (+260) + 7676 free blocks, 0 free inodes, 0 directories + Free blocks: 41477-49152 + Free inodes: +Group 6: (Blocks 49153-57344) [Inode not init, Block not init] + Block bitmap at 49153 (+0), Inode bitmap at 49154 (+1) + Inode table at 49155-49410 (+2) + 0 free blocks, 0 free inodes, 0 directories + Free blocks: + Free inodes: +Group 7: (Blocks 57345-65536) [Inode not init] + Backup superblock at 57345, Group descriptors at 57346-57346 + Reserved GDT blocks at 57347-57602 + Block bitmap at 57603 (+258), Inode bitmap at 57604 (+259) + Inode table at 57605-57860 (+260) + 7676 free blocks, 0 free inodes, 0 directories + Free blocks: 57861-65536 + Free inodes: +Group 8: (Blocks 65537-73728) [Inode not init, Block not init] + Block bitmap at 65537 (+0), Inode bitmap at 65538 (+1) + Inode table at 65539-65794 (+2) + 0 free blocks, 0 free inodes, 0 directories + Free blocks: + Free inodes: +Group 9:
[PATCH 13/13][e2fsprogs] Add m_uninit test case.
From: Jose R. Santos <[EMAIL PROTECTED]> Add m_uninit test case. Add test case to test for uninit block groups. Signed-off-by: Jose R. Santos <[EMAIL PROTECTED]> -- tests/m_uninit/expect.1 | 166 +++ tests/m_uninit/script |4 + 2 files changed, 170 insertions(+), 0 deletions(-) diff --git a/tests/m_uninit/expect.1 b/tests/m_uninit/expect.1 new file mode 100644 index 000..4167ff5 --- /dev/null +++ b/tests/m_uninit/expect.1 @@ -0,0 +1,166 @@ +Filesystem label= +OS type: Linux +Block size=1024 (log=0) +Fragment size=1024 (log=0) +32768 inodes, 131072 blocks +6553 blocks (5.00%) reserved for the super user +First data block=1 +Maximum filesystem blocks=67371008 +16 block groups +8192 blocks per group, 8192 fragments per group +2048 inodes per group +Superblock backups stored on blocks: + 8193, 24577, 40961, 57345, 73729 + +Writing inode tables: done +Writing superblocks and filesystem accounting information: done + +Filesystem features: resize_inode dir_index filetype sparse_super uninit_groups + +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +test_filesys: 11/32768 files (9.1% non-contiguous), 5691/131072 blocks +Exit status is 0 + +Filesystem volume name: +Last mounted on: +Filesystem magic number: 0xEF53 +Filesystem revision #:1 (dynamic) +Filesystem features: resize_inode dir_index filetype sparse_super uninit_groups +Default mount options:(none) +Filesystem state: clean +Errors behavior: Continue +Filesystem OS type: Linux +Inode count: 32768 +Block count: 131072 +Reserved block count: 6553 +Free blocks: 125381 +Free inodes: 32757 +First block: 1 +Block size: 1024 +Fragment size:1024 +Reserved GDT blocks: 256 +Blocks per group: 8192 +Fragments per group: 8192 +Inodes per group: 2048 +Inode blocks per group: 256 +Mount count: 0 +Check interval: 15552000 (6 months) +Reserved blocks uid: 0 +Reserved blocks gid: 0 +First inode: 11 +Inode size: 128 +Default directory hash: tea + + +Group 0: (Blocks 1-8192) + Primary superblock at 1, Group descriptors at 2-2 + Reserved GDT blocks at 3-258 + Block bitmap at 259 (+258), Inode bitmap at 260 (+259) + Inode table at 261-516 (+260) + 7662 free blocks, 2037 free inodes, 2 directories, 2037 unused inodes + Free blocks: 531-8192 + Free inodes: 12-2048 +Group 1: (Blocks 8193-16384) [Inode not init] + Backup superblock at 8193, Group descriptors at 8194-8194 + Reserved GDT blocks at 8195-8450 + Block bitmap at 8451 (+258), Inode bitmap at 8452 (+259) + Inode table at 8453-8708 (+260) + 7676 free blocks, 2048 free inodes, 0 directories, 2048 unused inodes + Free blocks: 8709-16384 + Free inodes: +Group 2: (Blocks 16385-24576) [Inode not init, Block not init] + Block bitmap at 16385 (+0), Inode bitmap at 16386 (+1) + Inode table at 16387-16642 (+2) + 7934 free blocks, 2048 free inodes, 0 directories, 2048 unused inodes + Free blocks: + Free inodes: +Group 3: (Blocks 24577-32768) [Inode not init] + Backup superblock at 24577, Group descriptors at 24578-24578 + Reserved GDT blocks at 24579-24834 + Block bitmap at 24835 (+258), Inode bitmap at 24836 (+259) + Inode table at 24837-25092 (+260) + 7676 free blocks, 2048 free inodes, 0 directories, 2048 unused inodes + Free blocks: 25093-32768 + Free inodes: +Group 4: (Blocks 32769-40960) [Inode not init, Block not init] + Block bitmap at 32769 (+0), Inode bitmap at 32770 (+1) + Inode table at 32771-33026 (+2) + 7934 free blocks, 2048 free inodes, 0 directories, 2048 unused inodes + Free blocks: + Free inodes: +Group 5: (Blocks 40961-49152) [Inode not init] + Backup superblock at 40961, Group descriptors at 40962-40962 + Reserved GDT blocks at 40963-41218 + Block bitmap at 41219 (+258), Inode bitmap at 41220 (+259) + Inode table at 41221-41476 (+260) + 7676 free blocks, 2048 free inodes, 0 directories, 2048 unused inodes + Free blocks: 41477-49152 + Free inodes: +Group 6: (Blocks 49153-57344) [Inode not init, Block not init] + Block bitmap at 49153 (+0), Inode bitmap at 49154 (+1) + Inode table at 49155-49410 (+2) + 7934 free blocks, 2048 free inodes, 0 directories, 2048 unused inodes + Free blocks: + Free inodes: +Group 7: (Blocks 57345-65536) [Inode not init] + Backup superblock at 57345, Group descriptors at 57346-57346 + Reserved GDT blocks at 57347-57602 + Block bitmap at 57603 (+258), Inode bitmap at 57604 (+259) + Inode table at 57605-57860 (+260) + 7676 free blocks, 2048 free inodes, 0 directories, 2048 unused inodes + Free blocks: 57861-65536 + Free inodes: +Group 8: (Blocks 65537-73728
[PATCH 09/13][e2fsprogs] Make e2fsck uninit block group aware.
From: Jose R. Santos <[EMAIL PROTECTED]>
Make e2fsck uninit block group aware.
This patch has all the necesary pieces to open and fix filesystems created
with the uninit block group feature.
Signed-off-by: Jose R. Santos <[EMAIL PROTECTED]>
--
e2fsck/e2fsck.h |2 +
e2fsck/journal.c |2 +
e2fsck/pass2.c | 77 --
e2fsck/pass5.c | 61 +++
e2fsck/problem.c | 42 +
e2fsck/problem.h | 26 ++
e2fsck/super.c | 40
e2fsck/unix.c| 11 ++--
e2fsck/util.c| 61 +++
9 files changed, 292 insertions(+), 30 deletions(-)
diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h
index 9ccffd8..a67322d 100644
--- a/e2fsck/e2fsck.h
+++ b/e2fsck/e2fsck.h
@@ -468,6 +468,8 @@ extern void e2fsck_read_bitmaps(e2fsck_t ctx);
extern void e2fsck_write_bitmaps(e2fsck_t ctx);
extern void preenhalt(e2fsck_t ctx);
extern char *string_copy(e2fsck_t ctx, const char *str, int len);
+extern errcode_t e2fsck_zero_blocks(ext2_filsys fs, blk_t blk, int num,
+ blk_t *ret_blk, int *ret_count);
#ifdef RESOURCE_TRACK
extern void print_resource_track(const char *desc,
struct resource_track *track,
diff --git a/e2fsck/journal.c b/e2fsck/journal.c
index f5f4647..ceade93 100644
--- a/e2fsck/journal.c
+++ b/e2fsck/journal.c
@@ -988,6 +988,8 @@ void e2fsck_move_ext3_journal(e2fsck_t ctx)
ext2fs_unmark_inode_bitmap(fs->inode_map, ino);
ext2fs_mark_ib_dirty(fs);
fs->group_desc[group].bg_free_inodes_count++;
+ fs->group_desc[group].bg_checksum =
+ ext2fs_group_desc_csum(fs->super, group,&fs->group_desc[group]);
fs->super->s_free_inodes_count++;
return;
diff --git a/e2fsck/pass2.c b/e2fsck/pass2.c
index 27f7136..047b5ca 100644
--- a/e2fsck/pass2.c
+++ b/e2fsck/pass2.c
@@ -151,7 +151,7 @@ void e2fsck_pass2(e2fsck_t ctx)
cd.pctx.errcode = ext2fs_dblist_iterate(fs->dblist, check_dir_block,
&cd);
- if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+ if (ctx->flags & E2F_FLAG_SIGNAL_MASK || ctx->flags & E2F_FLAG_RESTART)
return;
if (cd.pctx.errcode) {
fix_problem(ctx, PR_2_DBLIST_ITERATE, &cd.pctx);
@@ -736,7 +736,7 @@ static int check_dir_block(ext2_filsys fs,
buf = cd->buf;
ctx = cd->ctx;
- if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+ if (ctx->flags & E2F_FLAG_SIGNAL_MASK || ctx->flags & E2F_FLAG_RESTART)
return DIRENT_ABORT;
if (ctx->progress && (ctx->progress)(ctx, 2, cd->count++, cd->max))
@@ -833,6 +833,9 @@ static int check_dir_block(ext2_filsys fs,
dict_init(&de_dict, DICTCOUNT_T_MAX, dict_de_cmp);
prev = 0;
do {
+ int group;
+ ext2_ino_t first_unused_inode;
+
problem = 0;
dirent = (struct ext2_dir_entry *) (buf + offset);
cd->pctx.dirent = dirent;
@@ -882,12 +885,6 @@ static int check_dir_block(ext2_filsys fs,
(dirent->inode < EXT2_FIRST_INODE(fs->super))) ||
(dirent->inode > fs->super->s_inodes_count)) {
problem = PR_2_BAD_INO;
- } else if (!(ext2fs_test_inode_bitmap(ctx->inode_used_map,
- dirent->inode))) {
- /*
-* If the inode is unused, offer to clear it.
-*/
- problem = PR_2_UNUSED_INODE;
} else if (ctx->inode_bb_map &&
(ext2fs_test_inode_bitmap(ctx->inode_bb_map,
dirent->inode))) {
@@ -964,6 +961,67 @@ static int check_dir_block(ext2_filsys fs,
return DIRENT_ABORT;
}
+ group = ext2fs_group_of_ino(fs, dirent->inode);
+ first_unused_inode = group * fs->super->s_inodes_per_group +
+ 1 + fs->super->s_inodes_per_group -
+ fs->group_desc[group].bg_itable_unused;
+ cd->pctx.group = group;
+
+ /*
+* Check if the inode was missed out because _INODE_UNINIT
+* flag was set or bg_itable_unused was incorrect.
+* If that is the case restart e2fsck.
+* XXX Optimisations TODO:
+* 1. only restart e2fsck once
+* 2. only exposed inodes are checked again.
+*/
+ if (fs->group_desc[group].bg_flags & EXT2_BG_INODE_UNINIT) {
+ if (fix_problem(ctx, PR_2_INOREF_BG_INO_UNINIT,
+ &
[PATCH 11/13][e2fsprogs] Fix test cases.
From: Jose R. Santos <[EMAIL PROTECTED]> Fix test cases. Some of the tools outputs have changed, so this patch fixes what to expect from the outputs of the f_dupfsblks and m_raid_opt test cases. Signed-off-by: Jose R. Santos <[EMAIL PROTECTED]> -- tests/f_dupfsblks/expect.1 |3 ++- tests/m_raid_opt/expect.1 | 33 ++--- 2 files changed, 24 insertions(+), 12 deletions(-) diff --git a/tests/f_dupfsblks/expect.1 b/tests/f_dupfsblks/expect.1 index 661e164..32ce89b 100644 --- a/tests/f_dupfsblks/expect.1 +++ b/tests/f_dupfsblks/expect.1 @@ -44,7 +44,8 @@ Salvage? yes Directory inode 12, block 3, offset 0: directory corrupted Salvage? yes -Entry '' in ??? (12) has deleted/unused inode 32. Clear? yes +Entry '' in ??? (12) has a zero-length name. +Clear? yes Directory inode 12, block 4, offset 100: directory corrupted Salvage? yes diff --git a/tests/m_raid_opt/expect.1 b/tests/m_raid_opt/expect.1 index 44c5b46..f5abc37 100644 --- a/tests/m_raid_opt/expect.1 +++ b/tests/m_raid_opt/expect.1 @@ -46,57 +46,68 @@ Setting filetype for entry '..' in ??? (11) to 2. Directory inode 11, block 1, offset 0: directory corrupted Salvage? yes -Entry '' in ??? (11) has deleted/unused inode 1063. Clear? yes +Entry '' in ??? (11) has a zero-length name. +Clear? yes Directory inode 11, block 2, offset 0: directory corrupted Salvage? yes -Entry '' in ??? (11) has deleted/unused inode 1064. Clear? yes +Entry '' in ??? (11) has a zero-length name. +Clear? yes Directory inode 11, block 3, offset 0: directory corrupted Salvage? yes -Entry '' in ??? (11) has deleted/unused inode 1065. Clear? yes +Entry '' in ??? (11) has a zero-length name. +Clear? yes Directory inode 11, block 4, offset 0: directory corrupted Salvage? yes -Entry '' in ??? (11) has deleted/unused inode 1066. Clear? yes +Entry '' in ??? (11) has a zero-length name. +Clear? yes Directory inode 11, block 5, offset 0: directory corrupted Salvage? yes -Entry '' in ??? (11) has deleted/unused inode 1067. Clear? yes +Entry '' in ??? (11) has a zero-length name. +Clear? yes Directory inode 11, block 6, offset 0: directory corrupted Salvage? yes -Entry '' in ??? (11) has deleted/unused inode 1068. Clear? yes +Entry '' in ??? (11) has a zero-length name. +Clear? yes Directory inode 11, block 7, offset 0: directory corrupted Salvage? yes -Entry '' in ??? (11) has deleted/unused inode 1069. Clear? yes +Entry '' in ??? (11) has a zero-length name. +Clear? yes Directory inode 11, block 8, offset 0: directory corrupted Salvage? yes -Entry '' in ??? (11) has deleted/unused inode 1070. Clear? yes +Entry '' in ??? (11) has a zero-length name. +Clear? yes Directory inode 11, block 9, offset 0: directory corrupted Salvage? yes -Entry '' in ??? (11) has deleted/unused inode 1071. Clear? yes +Entry '' in ??? (11) has a zero-length name. +Clear? yes Directory inode 11, block 10, offset 0: directory corrupted Salvage? yes -Entry '' in ??? (11) has deleted/unused inode 1072. Clear? yes +Entry '' in ??? (11) has a zero-length name. +Clear? yes Directory inode 11, block 11, offset 0: directory corrupted Salvage? yes -Entry '' in ??? (11) has deleted/unused inode 1073. Clear? yes +Entry '' in ??? (11) has a zero-length name. +Clear? yes Pass 3: Checking directory connectivity '..' in / (2) is (0), should be / (2). - To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 08/13][e2fsprogs] Make debugfs uninit block group aware.
From: Jose R. Santos <[EMAIL PROTECTED]>
Make debugfs uninit block group aware.
Signed-off-by: Jose R. Santos <[EMAIL PROTECTED]>
--
debugfs/debugfs.c | 18 +++---
1 files changed, 15 insertions(+), 3 deletions(-)
diff --git a/debugfs/debugfs.c b/debugfs/debugfs.c
index 190c4b7..c802b63 100644
--- a/debugfs/debugfs.c
+++ b/debugfs/debugfs.c
@@ -286,7 +286,10 @@ void do_show_super_stats(int argc, char *argv[])
FILE*out;
struct ext2_group_desc *gdp;
int c, header_only = 0;
- int numdirs = 0, first;
+ int numdirs = 0, first, gdt_csum;
+
+ gdt_csum = EXT2_HAS_RO_COMPAT_FEATURE(current_fs->super,
+ EXT4_FEATURE_RO_COMPAT_GDT_CSUM);
reset_getopt();
while ((c = getopt (argc, argv, "h")) != EOF) {
@@ -322,7 +325,7 @@ void do_show_super_stats(int argc, char *argv[])
"inode table at %u\n"
" %d free %s, "
"%d free %s, "
- "%d used %s\n",
+ "%d used %s%s",
i, gdp->bg_block_bitmap,
gdp->bg_inode_bitmap, gdp->bg_inode_table,
gdp->bg_free_blocks_count,
@@ -331,12 +334,21 @@ void do_show_super_stats(int argc, char *argv[])
gdp->bg_free_inodes_count != 1 ? "inodes" : "inode",
gdp->bg_used_dirs_count,
gdp->bg_used_dirs_count != 1 ? "directories"
- : "directory");
+ : "directory", gdt_csum ? ", " : "\n");
+ if (gdt_csum)
+ fprintf(out, "%d unused %s\n",
+ gdp->bg_itable_unused,
+ gdp->bg_itable_unused != 1 ? "inodes":"inode");
first = 1;
print_bg_opts(gdp, EXT2_BG_INODE_UNINIT, "Inode not init",
&first, out);
print_bg_opts(gdp, EXT2_BG_BLOCK_UNINIT, "Block not init",
&first, out);
+ if (gdt_csum) {
+ fprintf(out, "%sChecksum 0x%04x",
+ first ? " [":", ", gdp->bg_checksum);
+ first = 0;
+ }
if (!first)
fputs("]\n", out);
}
-
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 10/13][e2fsprogs] Update uninit block group documetation for some of the utilities.
From: Jose R. Santos <[EMAIL PROTECTED]> Update uninit block group documetation for some of the utilities. Upadates documentation man pages for mke2fs(8) and tune2fs(8) Signed-off-by: Jose R. Santos <[EMAIL PROTECTED]> -- misc/mke2fs.8.in |9 - misc/tune2fs.8.in | 12 ++-- 2 files changed, 18 insertions(+), 3 deletions(-) diff --git a/misc/mke2fs.8.in b/misc/mke2fs.8.in index a3dc4a1..171df5b 100644 --- a/misc/mke2fs.8.in +++ b/misc/mke2fs.8.in @@ -210,7 +210,7 @@ for the filesystem. (For administrators who are creating filesystems on RAID arrays, it is preferable to use the .I stride RAID parameter as part of the -.B \-R +.B \-E option rather than manipulating the number of blocks per group.) This option is generally used by developers who are developing test cases. @@ -406,6 +406,13 @@ Store file type information in directory entries. .TP .B has_journal Create an ext3 journal (as if using the +.TP +.B uninit_groups +Create a filesystem without initializing all of the groups. This speeds +up filesystem creation time noticably, and can also reduce +.BR e2fsck time +dramatically. This feature causes the filesystem to be read-only in +older kernels is not supported in most Linux kernels, use with caution. .B \-j option). @[EMAIL PROTECTED] diff --git a/misc/tune2fs.8.in b/misc/tune2fs.8.in index 2e617db..5ab1bd0 100644 --- a/misc/tune2fs.8.in +++ b/misc/tune2fs.8.in @@ -392,10 +392,16 @@ option. .TP .B sparse_super Limit the number of backup superblocks to save space on large filesystems. +.TP +.B uninit_groups +Allow the kernel to initialize bitmaps and inode tables and keep a high +watermark for the unused inodes in a filesystem, to reduce +.BR e2fsck (8) +time. .RE .IP After setting or clearing -.B sparse_super +.BR sparse_super , " uninit_groups" , and .B filetype filesystem features, @@ -414,7 +420,9 @@ can be run to convert existing directories to the hashed B-tree format. Linux kernels before 2.0.39 and many 2.1 series kernels do not support the filesystems that use any of these features. Enabling certain filesystem features may prevent the filesystem from -being mounted by kernels which do not support those features. +being mounted by kernels which do not support those features. The +.B uninit_groups +feature is not yet supported by any released kernel. .TP .BI \-r " reserved-blocks-count" Set the number of reserved filesystem blocks. - To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 05/13][e2fsprogs] Make tune2fs uninit block group aware.
From: Jose R. Santos <[EMAIL PROTECTED]>
Make tune2fs uninit block group aware.
Signed-off-by: Jose R. Santos <[EMAIL PROTECTED]>
--
misc/tune2fs.c | 12 ++--
1 files changed, 10 insertions(+), 2 deletions(-)
diff --git a/misc/tune2fs.c b/misc/tune2fs.c
index 833b994..e2ebc08 100644
--- a/misc/tune2fs.c
+++ b/misc/tune2fs.c
@@ -98,7 +98,8 @@ static __u32 ok_features[3] = {
EXT3_FEATURE_COMPAT_HAS_JOURNAL |
EXT2_FEATURE_COMPAT_DIR_INDEX, /* Compat */
EXT2_FEATURE_INCOMPAT_FILETYPE, /* Incompat */
- EXT2_FEATURE_RO_COMPAT_SPARSE_SUPER /* R/O compat */
+ EXT2_FEATURE_RO_COMPAT_SPARSE_SUPER | /* R/O compat */
+ EXT4_FEATURE_RO_COMPAT_GDT_CSUM
};
/*
@@ -213,6 +214,8 @@ static int release_blocks_proc(ext2_filsys fs, blk_t
*blocknr,
ext2fs_unmark_block_bitmap(fs->block_map,block);
group = ext2fs_group_of_blk(fs, block);
fs->group_desc[group].bg_free_blocks_count++;
+ fs->group_desc[group].bg_checksum =
+ ext2fs_group_desc_csum(fs->super, group,&fs->group_desc[group]);
fs->super->s_free_blocks_count++;
return 0;
}
@@ -282,7 +285,7 @@ static void update_mntopts(ext2_filsys fs, char *mntopts)
static void update_feature_set(ext2_filsys fs, char *features)
{
int sparse, old_sparse, filetype, old_filetype;
- int journal, old_journal, dxdir, old_dxdir;
+ int journal, old_journal, dxdir, old_dxdir, uninit, old_uninit;
struct ext2_super_block *sb= fs->super;
__u32 old_compat, old_incompat, old_ro_compat;
@@ -298,6 +301,8 @@ static void update_feature_set(ext2_filsys fs, char
*features)
EXT3_FEATURE_COMPAT_HAS_JOURNAL;
old_dxdir = sb->s_feature_compat &
EXT2_FEATURE_COMPAT_DIR_INDEX;
+ old_uninit = sb->s_feature_ro_compat &
+ EXT4_FEATURE_RO_COMPAT_GDT_CSUM;
if (e2p_edit_feature(features, &sb->s_feature_compat,
ok_features)) {
fprintf(stderr, _("Invalid filesystem option set: %s\n"),
@@ -312,6 +317,8 @@ static void update_feature_set(ext2_filsys fs, char
*features)
EXT3_FEATURE_COMPAT_HAS_JOURNAL;
dxdir = sb->s_feature_compat &
EXT2_FEATURE_COMPAT_DIR_INDEX;
+ old_uninit = sb->s_feature_ro_compat &
+ EXT4_FEATURE_RO_COMPAT_GDT_CSUM;
if (old_journal && !journal) {
if ((mount_flags & EXT2_MF_MOUNTED) &&
!(mount_flags & EXT2_MF_READONLY)) {
@@ -358,6 +365,7 @@ static void update_feature_set(ext2_filsys fs, char
*features)
sb->s_feature_incompat))
ext2fs_update_dynamic_rev(fs);
if ((sparse != old_sparse) ||
+ (uninit != old_uninit) ||
(filetype != old_filetype)) {
sb->s_state &= ~EXT2_VALID_FS;
printf("\n%s\n", _(please_fsck));
-
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 06/13][e2fsprogs] Make dumpe2fs uninit block group aware.
From: Jose R. Santos <[EMAIL PROTECTED]>
Make dumpe2fs uninit block group aware.
Signed-off-by: Jose R. Santos <[EMAIL PROTECTED]>
--
misc/dumpe2fs.c | 13 +
1 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/misc/dumpe2fs.c b/misc/dumpe2fs.c
index d4d95bb..c08528a 100644
--- a/misc/dumpe2fs.c
+++ b/misc/dumpe2fs.c
@@ -112,7 +112,8 @@ static void print_bg_opts(ext2_filsys fs, dgrp_t i)
{
int first = 1, bg_flags;
- if (fs->super->s_feature_compat & EXT2_FEATURE_COMPAT_LAZY_BG)
+ if (fs->super->s_feature_compat & EXT2_FEATURE_COMPAT_LAZY_BG ||
+ fs->super->s_feature_ro_compat & EXT4_FEATURE_RO_COMPAT_GDT_CSUM)
bg_flags = fs->group_desc[i].bg_flags;
else
bg_flags = 0;
@@ -210,11 +211,15 @@ static void list_desc (ext2_filsys fs)
diff = fs->group_desc[i].bg_inode_table - first_block;
if (diff > 0)
printf(" (+%ld)", diff);
- printf (_("\n %d free blocks, %d free inodes, "
- "%d directories\n"),
+ printf (_("\n %u free blocks, %u free inodes, "
+ "%u directories%s"),
fs->group_desc[i].bg_free_blocks_count,
fs->group_desc[i].bg_free_inodes_count,
- fs->group_desc[i].bg_used_dirs_count);
+ fs->group_desc[i].bg_used_dirs_count,
+ fs->group_desc[i].bg_itable_unused ? "" : "\n");
+ if (fs->group_desc[i].bg_itable_unused)
+ printf (_(", %u unused inodes\n"),
+ fs->group_desc[i].bg_itable_unused);
if (block_bitmap) {
fputs(_(" Free blocks: "), stdout);
ext2fs_get_block_bitmap_range(fs->block_map,
-
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 04/13][e2fsprogs] Add support for creating filesystems using uninit block group.
From: Jose R. Santos <[EMAIL PROTECTED]>
Add support for creating filesystems using uninit block group.
Signed-off-by: Jose R. Santos <[EMAIL PROTECTED]>
--
misc/mke2fs.c | 44
1 files changed, 32 insertions(+), 12 deletions(-)
diff --git a/misc/mke2fs.c b/misc/mke2fs.c
index 4a6cace..8360c51 100644
--- a/misc/mke2fs.c
+++ b/misc/mke2fs.c
@@ -432,6 +432,8 @@ static void write_inode_tables(ext2_filsys fs)
num, blk, error_message(retval));
exit(1);
}
+ /* The kernel doesn't need to zero the itable blocks */
+ fs->group_desc[i].bg_flags |= EXT2_BG_INODE_ZEROED;
}
if (sync_kludge) {
if (sync_kludge == 1)
@@ -447,34 +449,49 @@ static void write_inode_tables(ext2_filsys fs)
static void setup_lazy_bg(ext2_filsys fs)
{
dgrp_t i;
- int blks;
+ int blks, csum_flag;
struct ext2_super_block *sb = fs->super;
struct ext2_group_desc *bg = fs->group_desc;
- if (EXT2_HAS_COMPAT_FEATURE(fs->super,
- EXT2_FEATURE_COMPAT_LAZY_BG)) {
+ csum_flag = EXT2_HAS_RO_COMPAT_FEATURE(fs->super,
+ EXT4_FEATURE_RO_COMPAT_GDT_CSUM);
+ if (EXT2_HAS_COMPAT_FEATURE(fs->super, EXT2_FEATURE_COMPAT_LAZY_BG) ||
+ csum_flag) {
for (i = 0; i < fs->group_desc_count; i++, bg++) {
if ((i == 0) ||
- (i == fs->group_desc_count-1))
+ (i == fs->group_desc_count - 1 && !csum_flag))
continue;
if (bg->bg_free_inodes_count ==
sb->s_inodes_per_group) {
- bg->bg_free_inodes_count = 0;
bg->bg_flags |= EXT2_BG_INODE_UNINIT;
- sb->s_free_inodes_count -=
- sb->s_inodes_per_group;
+ if (!csum_flag) {
+ bg->bg_free_inodes_count = 0;
+ sb->s_free_inodes_count -=
+ sb->s_inodes_per_group;
+ }
}
+
+ /* Skip groups with GDT backups because the resize
+* inode has blocks allocated in them, and the last
+* group because it needs block bitmap padding. */
+ if ((ext2fs_bg_has_super(fs, i) &&
+sb->s_reserved_gdt_blocks) ||
+ i == fs->group_desc_count - 1)
+ continue;
+
blks = ext2fs_super_and_bgd_loc(fs, i, 0, 0, 0, 0);
- if (bg->bg_free_blocks_count == blks) {
- bg->bg_free_blocks_count = 0;
+ if (bg->bg_free_blocks_count == blks &&
+ bg->bg_flags & EXT2_BG_INODE_UNINIT) {
bg->bg_flags |= EXT2_BG_BLOCK_UNINIT;
- sb->s_free_blocks_count -= blks;
+ if (!csum_flag) {
+ bg->bg_free_blocks_count = 0;
+ sb->s_free_blocks_count -= blks;
+ }
}
}
}
}
-
static void create_root_dir(ext2_filsys fs)
{
errcode_t retval;
@@ -874,7 +891,8 @@ static __u32 ok_features[3] = {
EXT2_FEATURE_INCOMPAT_FILETYPE| /* Incompat */
EXT3_FEATURE_INCOMPAT_JOURNAL_DEV|
EXT2_FEATURE_INCOMPAT_META_BG,
- EXT2_FEATURE_RO_COMPAT_SPARSE_SUPER /* R/O compat */
+ EXT2_FEATURE_RO_COMPAT_SPARSE_SUPER|/* R/O compat */
+ EXT4_FEATURE_RO_COMPAT_GDT_CSUM
};
@@ -1750,6 +1768,8 @@ int main (int argc, char *argv[])
}
no_journal:
+ if (!super_only)
+ ext2fs_set_gdt_csum(fs);
if (!quiet)
printf(_("Writing superblocks and "
"filesystem accounting information: "));
-
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 07/13][e2fsprogs] Make resize2fs uninit block group aware.
From: Jose R. Santos <[EMAIL PROTECTED]>
Make resize2fs uninit block group aware.
Signed-off-by: Jose R. Santos <[EMAIL PROTECTED]>
--
resize/main.c |7 +++
resize/resize2fs.c | 29 -
2 files changed, 31 insertions(+), 5 deletions(-)
diff --git a/resize/main.c b/resize/main.c
index 7db4ebc..7c1d0c1 100644
--- a/resize/main.c
+++ b/resize/main.c
@@ -298,6 +298,13 @@ int main (int argc, char ** argv)
printf (_("Couldn't find valid filesystem superblock.\n"));
exit (1);
}
+
+ if (fs->super->s_feature_ro_compat & EXT4_FEATURE_RO_COMPAT_GDT_CSUM) {
+ com_err(program_name, EXT2_ET_RO_UNSUPP_FEATURE,
+ ":- uninit_groups");
+ exit(1);
+ }
+
/*
* Check for compatibility with the feature sets. We need to
* be more stringent than ext2fs_open().
diff --git a/resize/resize2fs.c b/resize/resize2fs.c
index 0d6a082..ce0111c 100644
--- a/resize/resize2fs.c
+++ b/resize/resize2fs.c
@@ -339,7 +339,9 @@ retry:
numblocks = fs->super->s_blocks_per_group;
i = old_fs->group_desc_count - 1;
fs->group_desc[i].bg_free_blocks_count += (numblocks-old_numblocks);
-
+ fs->group_desc[i].bg_checksum =
+ ext2fs_group_desc_csum(fs->super, i, &fs->group_desc[i]);
+
/*
* If the number of block groups is staying the same, we're
* done and can exit now. (If the number block groups is
@@ -415,6 +417,8 @@ retry:
fs->group_desc[i].bg_free_inodes_count =
fs->super->s_inodes_per_group;
fs->group_desc[i].bg_used_dirs_count = 0;
+ fs->group_desc[i].bg_checksum =
+ ext2fs_group_desc_csum(fs->super, i,&fs->group_desc[i]);
retval = ext2fs_allocate_group_table(fs, i, 0);
if (retval) goto errout;
@@ -1223,9 +1227,13 @@ static errcode_t inode_scan_and_fix(ext2_resize_t rfs)
if (retval) goto errout;
group = (new_inode-1) /
EXT2_INODES_PER_GROUP(rfs->new_fs->super);
- if (LINUX_S_ISDIR(inode.i_mode))
+ if (LINUX_S_ISDIR(inode.i_mode)) {
rfs->new_fs->group_desc[group].bg_used_dirs_count++;
-
+ rfs->new_fs->group_desc[group].bg_checksum =
+ ext2fs_group_desc_csum(rfs->new_fs->super,group,
+ &rfs->new_fs->group_desc[group]);
+ }
+
#ifdef RESIZE2FS_DEBUG
if (rfs->flags & RESIZE_DEBUG_INODEMAP)
printf("Inode moved %u->%u\n", ino, new_inode);
@@ -1478,6 +1486,9 @@ static errcode_t move_itables(ext2_resize_t rfs)
ext2fs_unmark_block_bitmap(fs->block_map, blk);
rfs->old_fs->group_desc[i].bg_inode_table = new_blk;
+ rfs->old_fs->group_desc[i].bg_checksum =
+ ext2fs_group_desc_csum(rfs->old_fs->super, i,
+ &rfs->old_fs->group_desc[i]);
ext2fs_mark_super_dirty(rfs->old_fs);
ext2fs_flush(rfs->old_fs);
@@ -1575,8 +1586,12 @@ static errcode_t
ext2fs_calculate_summary_stats(ext2_filsys fs)
count++;
if ((count == fs->super->s_blocks_per_group) ||
(blk == fs->super->s_blocks_count-1)) {
- fs->group_desc[group++].bg_free_blocks_count =
+ fs->group_desc[group].bg_free_blocks_count =
group_free;
+ fs->group_desc[group].bg_checksum =
+ ext2fs_group_desc_csum(fs->super, group,
+ &fs->group_desc[group]);
+ group++;
count = 0;
group_free = 0;
}
@@ -1600,8 +1615,12 @@ static errcode_t
ext2fs_calculate_summary_stats(ext2_filsys fs)
count++;
if ((count == fs->super->s_inodes_per_group) ||
(ino == fs->super->s_inodes_count)) {
- fs->group_desc[group++].bg_free_inodes_count =
+ fs->group_desc[group].bg_free_inodes_count =
group_free;
+ fs->group_desc[group].bg_checksum =
+ ext2fs_group_desc_csum(fs->super, group,
+ &fs->group_desc[group]);
+ group++;
count = 0;
group_free = 0;
}
-
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 00/13][e2fsprogs] Uninit block group break down
The Uninit block group patch is painful to review since it's quite a large patch. Since simple patches are easier to review, I decided to break down the patch into smaller logical pieces that should be easier to review, clean and fix. The patch is based from the latest e2fsprogs git tree and aside from a compile test (which fails on PPC64 because of missing ext2fs_swab_group_desc()), I have not fully tested these patches. My intent here is to get some of the back log of patches in better shape for eventual inclussion into Ted's tree. Aside from the breakdown, this is essentially the same patch that Avantika submitted to the mailing list a couple of weeks ago with some minor changer to allow it to apply on the latest git tree. Let me know what you folks think. -JRS - To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 02/13][e2fsprogs] Add uninit block group support on libe2fs.
From: Jose R. Santos <[EMAIL PROTECTED]>
Add uninit block group support on libe2fs.
Signed-off-by: Jose R. Santos <[EMAIL PROTECTED]>
--
lib/ext2fs/alloc_stats.c | 25 +
lib/ext2fs/alloc_tables.c |5 ++---
lib/ext2fs/ext2_fs.h |1 +
lib/ext2fs/ext2fs.h |4 +++-
lib/ext2fs/initialize.c |2 ++
lib/ext2fs/inode.c| 29 +++--
lib/ext2fs/openfs.c | 16
lib/ext2fs/rw_bitmaps.c | 14 ++
8 files changed, 82 insertions(+), 14 deletions(-)
diff --git a/lib/ext2fs/alloc_stats.c b/lib/ext2fs/alloc_stats.c
index 4088f7b..ee4a1e4 100644
--- a/lib/ext2fs/alloc_stats.c
+++ b/lib/ext2fs/alloc_stats.c
@@ -27,6 +27,27 @@ void ext2fs_inode_alloc_stats2(ext2_filsys fs, ext2_ino_t
ino,
fs->group_desc[group].bg_free_inodes_count -= inuse;
if (isdir)
fs->group_desc[group].bg_used_dirs_count += inuse;
+
+ /* We don't strictly need to be clearing these if inuse < 0
+* (i.e. freeing inodes) but it also means something is bad. */
+ fs->group_desc[group].bg_flags &= ~(EXT2_BG_INODE_UNINIT |
+ EXT2_BG_BLOCK_UNINIT);
+ if (EXT2_HAS_RO_COMPAT_FEATURE(fs->super,
+ EXT4_FEATURE_RO_COMPAT_GDT_CSUM)) {
+ ext2_ino_t first_unused_inode = fs->super->s_inodes_per_group -
+ fs->group_desc[group].bg_itable_unused +
+ group * fs->super->s_inodes_per_group + 1;
+
+ if (ino >= first_unused_inode)
+ fs->group_desc[group].bg_itable_unused =
+ group * fs->super->s_inodes_per_group +
+ fs->super->s_inodes_per_group - ino;
+
+ fs->group_desc[group].bg_checksum =
+ ext2fs_group_desc_csum(fs->super, group,
+ &fs->group_desc[group]);
+ }
+
fs->super->s_free_inodes_count -= inuse;
ext2fs_mark_super_dirty(fs);
ext2fs_mark_ib_dirty(fs);
@@ -46,6 +67,10 @@ void ext2fs_block_alloc_stats(ext2_filsys fs, blk_t blk, int
inuse)
else
ext2fs_unmark_block_bitmap(fs->block_map, blk);
fs->group_desc[group].bg_free_blocks_count -= inuse;
+ fs->group_desc[group].bg_flags &= ~EXT2_BG_BLOCK_UNINIT;
+ fs->group_desc[group].bg_checksum =
+ ext2fs_group_desc_csum(fs->super, group,&fs->group_desc[group]);
+
fs->super->s_free_blocks_count -= inuse;
ext2fs_mark_super_dirty(fs);
ext2fs_mark_bb_dirty(fs);
diff --git a/lib/ext2fs/alloc_tables.c b/lib/ext2fs/alloc_tables.c
index 4ad2ba9..290e54b 100644
--- a/lib/ext2fs/alloc_tables.c
+++ b/lib/ext2fs/alloc_tables.c
@@ -95,13 +95,12 @@ errcode_t ext2fs_allocate_group_table(ext2_filsys fs,
dgrp_t group,
ext2fs_mark_block_bitmap(bmap, blk);
fs->group_desc[group].bg_inode_table = new_blk;
}
+ fs->group_desc[group].bg_checksum =
+ ext2fs_group_desc_csum(fs->super, group,&fs->group_desc[group]);
-
return 0;
}
-
-
errcode_t ext2fs_allocate_tables(ext2_filsys fs)
{
errcode_t retval;
diff --git a/lib/ext2fs/ext2_fs.h b/lib/ext2fs/ext2_fs.h
index a316665..7b63828 100644
--- a/lib/ext2fs/ext2_fs.h
+++ b/lib/ext2fs/ext2_fs.h
@@ -173,6 +173,7 @@ struct ext4_group_desc
#define EXT2_BG_INODE_UNINIT 0x0001 /* Inode table/bitmap not initialized */
#define EXT2_BG_BLOCK_UNINIT 0x0002 /* Block bitmap not initialized */
+#define EXT2_BG_INODE_ZEROED 0x0004 /* On-disk itable initialized to zero */
/*
* Data structures used by the directory indexing feature
diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
index 1267ee8..c47536b 100644
--- a/lib/ext2fs/ext2fs.h
+++ b/lib/ext2fs/ext2fs.h
@@ -321,6 +321,7 @@ typedef struct ext2_struct_inode_scan *ext2_inode_scan;
#define EXT2_SF_BAD_EXTRA_BYTES0x0004
#define EXT2_SF_SKIP_MISSING_ITABLE0x0008
#define EXT2_SF_DO_LAZY0x0010
+#define EXT2_SF_DO_CSUM0x0020
/*
* ext2fs_check_if_mounted flags
@@ -440,7 +441,8 @@ typedef struct ext2_icount *ext2_icount_t;
EXT3_FEATURE_INCOMPAT_RECOVER)
#endif
#define EXT2_LIB_FEATURE_RO_COMPAT_SUPP
(EXT2_FEATURE_RO_COMPAT_SPARSE_SUPER|\
-EXT2_FEATURE_RO_COMPAT_LARGE_FILE)
+EXT2_FEATURE_RO_COMPAT_LARGE_FILE|\
+EXT4_FEATURE_RO_COMPAT_GDT_CSUM)
/*
* These features are only allowed if EXT2_FLAG_SOFTSUPP_FEATURES is passed
diff --git a/lib/ext2fs/initialize.c b/lib/ext2fs/initialize.c
index 16e9eaa..5710f04 100644
--- a/lib/ext2fs/initialize.c
+++ b/lib/ext2fs/initialize.c
@@ -374,6 +374,8 @@ ipg_retry:
fs->gro
[PATCH 03/13][e2fsprogs] Rename feature name from gdt_checksum to uninit_groups.
From: Jose R. Santos <[EMAIL PROTECTED]>
Rename feature name from gdt_checksum to uninit_groups.
This name is a more intuitive option when running mke2fs.
Signed-off-by: Jose R. Santos <[EMAIL PROTECTED]>
--
lib/e2p/feature.c |2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/lib/e2p/feature.c b/lib/e2p/feature.c
index fe7e65a..7c25736 100644
--- a/lib/e2p/feature.c
+++ b/lib/e2p/feature.c
@@ -45,7 +45,7 @@ static struct feature feature_list[] = {
{ E2P_FEATURE_RO_INCOMPAT, EXT4_FEATURE_RO_COMPAT_HUGE_FILE,
"huge_file" },
{ E2P_FEATURE_RO_INCOMPAT, EXT4_FEATURE_RO_COMPAT_GDT_CSUM,
- "gdt_checksum" },
+ "uninit_groups" },
{ E2P_FEATURE_RO_INCOMPAT, EXT4_FEATURE_RO_COMPAT_DIR_NLINK,
"dir_nlink" },
{ E2P_FEATURE_RO_INCOMPAT, EXT4_FEATURE_RO_COMPAT_EXTRA_ISIZE,
-
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 01/13][e2fsprogs] Add initial checksum support.
From: Jose R. Santos <[EMAIL PROTECTED]>
Add initial checksum support.
- Add support for computing CRC-16 value.
- Add call to check/verify/set csum on block_groups.
- Add a test program to verify csum operations.
Signed-off-by: Jose R. Santos <[EMAIL PROTECTED]>
--
lib/ext2fs/Makefile.in | 23 ++-
lib/ext2fs/crc16.c | 60 +++
lib/ext2fs/crc16.h | 29 +
lib/ext2fs/csum.c | 149
lib/ext2fs/ext2fs.h|7 ++
lib/ext2fs/tst_csum.c | 113
6 files changed, 378 insertions(+), 3 deletions(-)
diff --git a/lib/ext2fs/Makefile.in b/lib/ext2fs/Makefile.in
index 03ce131..847fe23 100644
--- a/lib/ext2fs/Makefile.in
+++ b/lib/ext2fs/Makefile.in
@@ -66,7 +66,9 @@ OBJS= $(DEBUGFS_LIB_OBJS) $(RESIZE_LIB_OBJS)
$(E2IMAGE_LIB_OBJS) \
unix_io.o \
unlink.o \
valid_blk.o \
- version.o
+ version.o \
+ crc16.o \
+ csum.o
SRCS= ext2_err.c \
$(srcdir)/alloc.c \
@@ -132,7 +134,10 @@ SRCS= ext2_err.c \
$(srcdir)/tst_bitops.c \
$(srcdir)/tst_byteswap.c \
$(srcdir)/tst_getsize.c \
- $(srcdir)/tst_iscan.c
+ $(srcdir)/tst_iscan.c \
+ $(srcdir)/tst_csum.c \
+ $(srcdir)/crc16.c \
+ $(srcdir)/csum.c
HFILES= bitops.h ext2fs.h ext2_io.h ext2_fs.h ext2_ext_attr.h ext3_extents.h \
tdb.h
@@ -239,17 +244,23 @@ ext2_tdbtool: tdbtool.o
@echo " LD $@"
@$(CC) -o ext2_tdbtool tdbtool.o tdb.o
+tst_csum: tst_csum.o csum.o crc16.o $(STATIC_LIBEXT2FS)
+ @echo " LD $@"
+ @$(CC) -o tst_csum csum.o tst_csum.o crc16.o $(STATIC_LIBEXT2FS) \
+ $(LIBCOM_ERR)
+
mkjournal: mkjournal.c $(STATIC_LIBEXT2FS)
@echo " LD $@"
@$(CC) -o mkjournal $(srcdir)/mkjournal.c -DDEBUG $(STATIC_LIBEXT2FS)
$(LIBCOM_ERR) $(ALL_CFLAGS)
-check:: tst_bitops tst_badblocks tst_iscan tst_types tst_icount tst_super_size
+check:: tst_bitops tst_badblocks tst_iscan tst_types tst_icount tst_super_size
tst_types tst_csum
LD_LIBRARY_PATH=$(LIB) DYLD_LIBRARY_PATH=$(LIB) ./tst_bitops
LD_LIBRARY_PATH=$(LIB) DYLD_LIBRARY_PATH=$(LIB) ./tst_badblocks
LD_LIBRARY_PATH=$(LIB) DYLD_LIBRARY_PATH=$(LIB) ./tst_iscan
LD_LIBRARY_PATH=$(LIB) DYLD_LIBRARY_PATH=$(LIB) ./tst_types
LD_LIBRARY_PATH=$(LIB) DYLD_LIBRARY_PATH=$(LIB) ./tst_icount
LD_LIBRARY_PATH=$(LIB) DYLD_LIBRARY_PATH=$(LIB) ./tst_super_size
+ LD_LIBRARY_PATH=$(LIB) DYLD_LIBRARY_PATH=$(LIB) ./tst_csum
installdirs::
@echo " MKINSTALLDIRS $(libdir) $(includedir)/ext2fs"
@@ -357,6 +368,10 @@ closefs.o: $(srcdir)/closefs.c $(srcdir)/ext2_fs.h \
$(srcdir)/ext2fs.h $(srcdir)/ext2_fs.h $(srcdir)/ext3_extents.h \
$(top_srcdir)/lib/et/com_err.h $(srcdir)/ext2_io.h \
$(top_builddir)/lib/ext2fs/ext2_err.h $(srcdir)/bitops.h
+crc16.o: $(srcdir)/crc16.c $(srcdir)/ext2_fs.h $(srcdir)/crc16.h \
+ $(top_builddir)/lib/ext2fs/ext2_types.h $(srcdir)/ext2fs.h $(srcdir)/ext2_fs.h
+csum.o: $(srcdir)/csum.c $(srcdir)/ext2_fs.h \
+ $(top_builddir)/lib/ext2fs/ext2_types.h $(srcdir)/ext2fs.h $(srcdir)/ext2_fs.h
dblist.o: $(srcdir)/dblist.c $(srcdir)/ext2_fs.h \
$(top_builddir)/lib/ext2fs/ext2_types.h $(srcdir)/ext2fsP.h \
$(srcdir)/ext2fs.h $(srcdir)/ext2_fs.h $(srcdir)/ext3_extents.h \
@@ -571,3 +586,5 @@ tst_iscan.o: $(srcdir)/tst_iscan.c $(srcdir)/ext2_fs.h \
$(top_builddir)/lib/ext2fs/ext2_types.h $(srcdir)/ext2fs.h \
$(srcdir)/ext2_fs.h $(srcdir)/ext3_extents.h $(top_srcdir)/lib/et/com_err.h \
$(srcdir)/ext2_io.h $(top_builddir)/lib/ext2fs/ext2_err.h $(srcdir)/bitops.h
+tst_csum.o: $(srcdir)/tst_csum.c $(srcdir)/ext2_fs.h \
+ $(top_builddir)/lib/ext2fs/ext2_types.h $(srcdir)/ext2fs.h
diff --git a/lib/ext2fs/crc16.c b/lib/ext2fs/crc16.c
new file mode 100644
index 000..c3d07e1
--- /dev/null
+++ b/lib/ext2fs/crc16.c
@@ -0,0 +1,60 @@
+/*
+ * crc16.c
+ *
+ * This source code is licensed under the GNU General Public License,
+ * Version 2. See the file COPYING for more details.
+ */
+
+#include
+#include "crc16.h"
+
+/** CRC table for the CRC-16. The poly is 0x8005 (x16 + x15 + x2 + 1) */
+uint16_t const crc16_table[256] = {
+ 0x, 0xC0C1, 0xC181, 0x0140, 0xC301, 0x03C0, 0x0280, 0xC241,
+ 0xC601, 0x06C0, 0x0780, 0xC741, 0x0500, 0xC5C1, 0xC481, 0x0440,
+ 0xCC01, 0x0CC0, 0x0D80, 0xCD41, 0x0F00, 0xCFC1, 0xCE81, 0x0E40,
+ 0x0A00, 0xCAC1, 0xCB81, 0x0B40, 0xC901, 0x09C0, 0x0880, 0xC841,
+ 0xD801, 0x18C0, 0x1980, 0xD941, 0x1B00, 0xDBC1, 0xDA81, 0x1A40,
+ 0x1E00, 0xDEC1, 0xDF81, 0x1F40, 0xDD01, 0x1DC0, 0x1C80, 0xDC41,
+ 0x1400, 0xD4C1, 0xD581, 0x1540, 0xD701, 0x17C0, 0x1680, 0xD641,
+ 0xD201, 0x12C0, 0x1380, 0xD341, 0x1100, 0xD1C1, 0xD081, 0x1040,
+ 0xF001, 0x30C0, 0x3180, 0xF141, 0x3300, 0xF3C1, 0xF281, 0x3240,
+ 0x3600, 0xF6C1, 0xF781, 0x3740, 0xF501, 0x35C0, 0x3480, 0xF441,
+ 0x3C00, 0xFCC1, 0x
Re: [PATCH 1/2] Make ext3 use bit operations to manipulate i_flags
Hello, attached patch implements recursive mtime feature for ext3. Things like rsync or updatedb can be much faster with it... More detailed description is in the patch. BTW: Hardlinks are currently unsolved - change is propagated to just the first parent. Currently it's upto userspace to handle this - if you have some idea how to make hardlinked files easier for userspace, please speak up :). Any comments welcome. Honza Implement recursive mtime (rtime) feature for ext3. The feature works as follows: In each inode we keep a flag EXT3_RTIME_FL (modifiable by user) whether rtime should be updated. In case the inode is modified and when the flag is set, inode's rtime is updated, the flag is cleared, and we move to the parent. If the flag is set there, we clear it, update rtime and continue upwards upto the root of the filesystem. Because the flag is always cleared after updating rtime, we have constant amortized complexity of rtime updates. Intended use case is that application which wants to watch any modification in a subtree scans the subtree and sets flags for all inodes there. Next time, it just needs to recurse in directories having rtime newer than the start of the previous scan. There it can handle modifications and set the flag again. Signed-off-by: Jan Kara <[EMAIL PROTECTED]> diff -rupX /home/jack/.kerndiffexclude linux-2.6.23-1-ext3_iflags_locking/fs/ext3/ialloc.c linux-2.6.23-2-ext3_recursive_mtime/fs/ext3/ialloc.c --- linux-2.6.23-1-ext3_iflags_locking/fs/ext3/ialloc.c 2007-10-11 18:05:48.0 +0200 +++ linux-2.6.23-2-ext3_recursive_mtime/fs/ext3/ialloc.c2007-10-11 18:14:03.0 +0200 @@ -580,6 +580,7 @@ got: ei->i_file_acl = 0; ei->i_dir_acl = 0; ei->i_dtime = 0; + ei->i_rtime = inode->i_mtime.tv_sec; ei->i_block_alloc_info = NULL; ei->i_block_group = group; diff -rupX /home/jack/.kerndiffexclude linux-2.6.23-1-ext3_iflags_locking/fs/ext3/inode.c linux-2.6.23-2-ext3_recursive_mtime/fs/ext3/inode.c --- linux-2.6.23-1-ext3_iflags_locking/fs/ext3/inode.c 2007-10-11 18:13:04.0 +0200 +++ linux-2.6.23-2-ext3_recursive_mtime/fs/ext3/inode.c 2007-10-11 18:57:50.0 +0200 @@ -1232,6 +1232,8 @@ static int ext3_ordered_commit_write(str ret2 = ext3_journal_stop(handle); if (!ret) ret = ret2; + if (!ret) + ext3_update_rtimes(inode); return ret; } @@ -1255,6 +1257,8 @@ static int ext3_writeback_commit_write(s ret2 = ext3_journal_stop(handle); if (!ret) ret = ret2; + if (!ret) + ext3_update_rtimes(inode); return ret; } @@ -1288,6 +1292,8 @@ static int ext3_journalled_commit_write( ret2 = ext3_journal_stop(handle); if (!ret) ret = ret2; + if (!ret) + ext3_update_rtimes(inode); return ret; } @@ -2386,6 +2392,10 @@ out_stop: ext3_orphan_del(handle, inode); ext3_journal_stop(handle); + /* We update time only for linked inodes. Unlinked ones already +* notified parent during unlink... */ + if (inode->i_nlink) + ext3_update_rtimes(inode); } static ext3_fsblk_t ext3_get_inode_block(struct super_block *sb, @@ -2628,6 +2638,8 @@ void ext3_read_inode(struct inode * inod inode->i_ctime.tv_sec = (signed)le32_to_cpu(raw_inode->i_ctime); inode->i_mtime.tv_sec = (signed)le32_to_cpu(raw_inode->i_mtime); inode->i_atime.tv_nsec = inode->i_ctime.tv_nsec = inode->i_mtime.tv_nsec = 0; + if (EXT3_HAS_COMPAT_FEATURE(inode->i_sb, EXT3_FEATURE_COMPAT_RTIME)) + ei->i_rtime = le32_to_cpu(raw_inode->i_rtime); ei->i_state = 0; ei->i_dir_start_lookup = 0; @@ -2785,6 +2797,8 @@ static int ext3_do_update_inode(handle_t raw_inode->i_atime = cpu_to_le32(inode->i_atime.tv_sec); raw_inode->i_ctime = cpu_to_le32(inode->i_ctime.tv_sec); raw_inode->i_mtime = cpu_to_le32(inode->i_mtime.tv_sec); + if (EXT3_HAS_COMPAT_FEATURE(inode->i_sb, EXT3_FEATURE_COMPAT_RTIME)) + raw_inode->i_rtime = cpu_to_le32(ei->i_rtime); raw_inode->i_blocks = cpu_to_le32(inode->i_blocks); raw_inode->i_dtime = cpu_to_le32(ei->i_dtime); raw_inode->i_flags = cpu_to_le32(ei->i_flags); @@ -3137,6 +3151,7 @@ void ext3_dirty_inode(struct inode *inod handle_t *current_handle = ext3_journal_current_handle(); handle_t *handle; + /* Reserve 2 blocks for inode and superblock */ handle = ext3_journal_start(inode, 2); if (IS_ERR(handle)) goto out; diff -rupX /home/jack/.kerndiffexclude linux-2.6.23-1-ext3_iflags_locking/fs/ext3/ioctl.c linux-2.6.23-2-ext3_recursive_mtime/fs/ext3/ioctl.c --- linux-2.6.23-1-ext3_iflags_locking/fs/ext3/ioctl.c 2007-10-11 16:54:03.0 +0200 ++
[PATCH 1/2] Make ext3 use bit operations to manipulate i_flags
Hello,
attached patch makes ext3 use bit operations to manipulate its
EXT3_I->i_flags. So far i_flags modifications were usually guarded
by i_mutex but my next patch needs to modify i_flags without i_mutex.
BTW: Is there any type + functions, which allow both bit operations
and things like atomic_read() and atomic_set()? It would be nice to
use them here - currently I just assign to the variable but I'm not sure if
this cannot result in some garbage on some strange architecture.
Honza
--
Jan Kara <[EMAIL PROTECTED]>
SUSE Labs, CR
Use set_bit(), clear_bit() and test_bit() to manipulate EXT3_I->i_flags.
This allows concurrent updates to EXT3_I->i_flags. At several places
we need to copy EXT3_I->i_flags as a whole - we do it non-atomically
and hope we don't get any garbage.
Signed-off-by: Jan Kara <[EMAIL PROTECTED]>
diff -rupX /home/jack/.kerndiffexclude linux-2.6.23/fs/ext3/dir.c
linux-2.6.23-1-ext3_iflags_locking/fs/ext3/dir.c
--- linux-2.6.23/fs/ext3/dir.c 2007-10-11 12:01:23.0 +0200
+++ linux-2.6.23-1-ext3_iflags_locking/fs/ext3/dir.c2007-10-11
18:03:19.0 +0200
@@ -110,7 +110,7 @@ static int ext3_readdir(struct file * fi
#ifdef CONFIG_EXT3_INDEX
if (EXT3_HAS_COMPAT_FEATURE(inode->i_sb,
EXT3_FEATURE_COMPAT_DIR_INDEX) &&
- ((EXT3_I(inode)->i_flags & EXT3_INDEX_FL) ||
+ (test_bit(EXT3_INDEX_FL, &EXT3_I(inode)->i_flags) ||
((inode->i_size >> sb->s_blocksize_bits) == 1))) {
err = ext3_dx_readdir(filp, dirent, filldir);
if (err != ERR_BAD_DX_DIR) {
@@ -121,7 +121,7 @@ static int ext3_readdir(struct file * fi
* We don't set the inode dirty flag since it's not
* critical that it get flushed back to the disk.
*/
- EXT3_I(filp->f_path.dentry->d_inode)->i_flags &= ~EXT3_INDEX_FL;
+ clear_bit(EXT3_INDEX_FL,
&EXT3_I(filp->f_path.dentry->d_inode)->i_flags);
}
#endif
stored = 0;
diff -rupX /home/jack/.kerndiffexclude linux-2.6.23/fs/ext3/ialloc.c
linux-2.6.23-1-ext3_iflags_locking/fs/ext3/ialloc.c
--- linux-2.6.23/fs/ext3/ialloc.c 2006-11-29 22:57:37.0 +0100
+++ linux-2.6.23-1-ext3_iflags_locking/fs/ext3/ialloc.c 2007-10-11
18:05:48.0 +0200
@@ -278,7 +278,7 @@ static int find_group_orlov(struct super
ndirs = percpu_counter_read_positive(&sbi->s_dirs_counter);
if ((parent == sb->s_root->d_inode) ||
- (EXT3_I(parent)->i_flags & EXT3_TOPDIR_FL)) {
+ test_bit(EXT3_TOPDIR_FL, &EXT3_I(parent)->i_flags)) {
int best_ndir = inodes_per_group;
int best_group = -1;
diff -rupX /home/jack/.kerndiffexclude linux-2.6.23/fs/ext3/inode.c
linux-2.6.23-1-ext3_iflags_locking/fs/ext3/inode.c
--- linux-2.6.23/fs/ext3/inode.c2007-10-11 12:01:23.0 +0200
+++ linux-2.6.23-1-ext3_iflags_locking/fs/ext3/inode.c 2007-10-11
18:13:04.0 +0200
@@ -2557,18 +2557,16 @@ int ext3_get_inode_loc(struct inode *ino
void ext3_set_inode_flags(struct inode *inode)
{
- unsigned int flags = EXT3_I(inode)->i_flags;
-
inode->i_flags &= ~(S_SYNC|S_APPEND|S_IMMUTABLE|S_NOATIME|S_DIRSYNC);
- if (flags & EXT3_SYNC_FL)
+ if (test_bit(EXT3_SYNC_FL, &EXT3_I(inode)->i_flags))
inode->i_flags |= S_SYNC;
- if (flags & EXT3_APPEND_FL)
+ if (test_bit(EXT3_APPEND_FL, &EXT3_I(inode)->i_flags))
inode->i_flags |= S_APPEND;
- if (flags & EXT3_IMMUTABLE_FL)
+ if (test_bit(EXT3_IMMUTABLE_FL, &EXT3_I(inode)->i_flags))
inode->i_flags |= S_IMMUTABLE;
- if (flags & EXT3_NOATIME_FL)
+ if (test_bit(EXT3_NOATIME_FL, &EXT3_I(inode)->i_flags))
inode->i_flags |= S_NOATIME;
- if (flags & EXT3_DIRSYNC_FL)
+ if (test_bit(EXT3_DIRSYNC_FL, &EXT3_I(inode)->i_flags))
inode->i_flags |= S_DIRSYNC;
}
@@ -2577,18 +2575,26 @@ void ext3_get_inode_flags(struct ext3_in
{
unsigned int flags = ei->vfs_inode.i_flags;
- ei->i_flags &= ~(EXT3_SYNC_FL|EXT3_APPEND_FL|
- EXT3_IMMUTABLE_FL|EXT3_NOATIME_FL|EXT3_DIRSYNC_FL);
if (flags & S_SYNC)
- ei->i_flags |= EXT3_SYNC_FL;
+ set_bit(EXT3_SYNC_FL, &ei->i_flags);
+ else
+ clear_bit(EXT3_SYNC_FL, &ei->i_flags);
if (flags & S_APPEND)
- ei->i_flags |= EXT3_APPEND_FL;
+ set_bit(EXT3_APPEND_FL, &ei->i_flags);
+ else
+ clear_bit(EXT3_APPEND_FL, &ei->i_flags);
if (flags & S_IMMUTABLE)
- ei->i_flags |= EXT3_IMMUTABLE_FL;
+ set_bit(EXT3_IMMUTABLE_FL, &ei->i_flags);
+ else
+ clear_bit(EXT3_IMMUTABLE_FL, &ei->i_flags);
if (flags & S_NOATIME)
- ei->i_flags |= EXT3_NOATIME_FL;
+
When to set EXT4_FEATURE_RO_COMPAT_LARGE_FILE ?
Right now we do this in ext4_do_update_inode
if (ei->i_disksize > 0x7fffULL) {
.
.
EXT4_SET_RO_COMPAT_FEATURE(sb, EXT4_FEATURE_RO_COMPAT_LARGE_FILE);
That is if i_disksize is greater that 2**31 -1 we set the flag
But in ext4_inode the size of i_size is 32 bits
__le32 i_size; /* Size in bytes */
That means it should be able to store ULL.
Why did we decided to enable the EXT4_FEATURE_RO_COMPAT_LARGE_FILE when we
cross 2**31 -1 ?
NOTE: With EXT4_FEATURE_RO_COMPAT_LARGE_FILE we store the higer order 32 bits
of i_size in i_dir_acl.
-aneesh
-
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Threaded readahead strawman
Hello
Andreas Dilger wrote:
> On Oct 10, 2007 20:09 -0700, Valerie Henson wrote:
>> I need to get started on a mergeable version of the threaded readahead
>> patch for e2fsck. I intend for it to be compatible with Andreas'
>> sys_readahead() for block devices that support it. Here's a first
>> draft proposal - your thoughts? Note that it's not really that
>> anything is being read *ahead* per se, but that it's being read
>> simultaneously. Single-threaded readahead doesn't go any faster.
>
> We've been fiddling with this as well. I'd attach some patches but
> bugzilla is down as I write this :(. I also asked Vladimir (working on
> these patches) to forward them to you and the linux-ext4 mailing list.
>
The patch is attached.
If an application can foresee what it is going to read in future - it
can call io_channel_readahead for those data forehand. Even if
io_channel_readahead is called right before the data are actually needed
- it may make positive effect for multi disk devices because of parallel
reading.
For example, using io_channel_readahead to readahead coming inode tables
in done_group callback of ext2_inode_scan changes inode table scan in my
local quick test from 34 seconds to 26 (on 2 two ide disk raid0)
> We added a "readahead" method to the io_manager interface (no-op for
> Win/DOS) that can be used generically. This is currently done via
> posix_fadvise(POSIX_FADV_WILLNEED). We haven't done any multi-threading
> yet, but there is some hope that the block layer could sort it out?
> It would still be beneficial to have multiple user-space threads do
> the reading of the data, to get parallel memcpy() into userspace.
>
>> The major global parameters to the system are:
>>
>> 1. Optimal number of concurrent requests - number of underlying read
>> heads times some N of best number of outstanding requests. Default to
>> one.
>>
>> 2. Stripe size, or more generally which areas can be read concurrently
>> and which cannot.
>
> There are new parameters in the superblock (s_raid_stride and
> s_raid_stripe_width) but as yet only s_raid_stride is initialized by
> mke2fs. There is a library in xfstools (libdisk or somesuch) that
> can get a lot more disk geometry info and it would be good to leverage
> that for mke2fs also.
>
>> 3. Maximum memory to use. We have to keep the readahead from
>> outrunning the actual processing (though so far, that hasn't been a
>> problem) and having bits of our buffer cache kicked out before they
>> are used. This can be set to some percentage of available memory by
>> default.
>
> Agreed. I'd proposed in the past that fsck could call fsck.{fstype}
> with a parameter like --expected-memory to determine the expected memory
> usage of fsck.{fstype} based on the filesystem geometry, and it could
> also supply --max-memory so we don't have parallel fscks stomping on
> each other.
>
>> I see two main ways to do this: One is a straightforward offset plus
>> size, telling it what to read. The other is to make libext2 do all
>> the interpretation of ondisk format, and design the interface in terms
>> of kinds of metadata to read. Given that libext2 functions like
>> ext2fs_get_next_inode_full() should be aware of what's going on in
>> readahead. This argues for a metadata aware, in-library
>> implementation. Something like:
>>
>> /* Creates the threads, sets some variables. Returns a handle. */
>> handle = ext2fs_readahead_init(concurrent_requests, stripe_size, max_memory);
>>
>> /* Readahead inode tables and inode indirect blocks - can't really be
>> separated */
>> ext2fs_readahead_inodes(handle, fs);
>
> Well, there's something to be said for allowing the inode tables and
> corresponding bitmaps to be read in a single shot. Also, not all users
> require the indirect blocks, so I would make that an option.
>
>> /* Read the directory block list (pass 2) */
>> ext2fs_readahead_dblist(handle, fs);
>
> We're working on this as part of e2scan (in bug 13108 above), not sure if
> there is a patch available or not.
>
>> /* Read bitmaps (pass 5) */
>> ext2fs_readahead_bitmaps(handle, fs);
>
> This is a big one, because of the many seeks for small data read. Using
> the FLEX_BG feature (which is really a tiny kernel patch) could improve
> this many times.
>
> Cheers, Andreas
> --
> Andreas Dilger
> Principal Software Engineer
> Cluster File Systems, Inc.
>
>
This patch adds a "readahead" method to the io_manager interface
Signed-off-by: Vladimir V. Saveliev [EMAIL PROTECTED]
Index: e2fsprogs-1.40.2/lib/ext2fs/ext2_io.h
===
--- e2fsprogs-1.40.2.orig/lib/ext2fs/ext2_io.h
+++ e2fsprogs-1.40.2/lib/ext2fs/ext2_io.h
@@ -68,6 +68,8 @@ struct struct_io_manager {
errcode_t (*set_blksize)(io_channel channel, int blksize);
errcode_t (*read_blk)(io_channel channel, unsigned long block,
int count, void *data);
+ errcode_t (*readahead)(io_channel channel, unsigned long block,
+
[PATCH] ext2: Fix the max file size for ext2 file system.
The max file size for ext2 file system is now calculated
with hardcoded 4K block size. The patch fixes it to be
calculated with the right block size.
Signed-off-by: Aneesh Kumar K.V <[EMAIL PROTECTED]>
---
fs/ext2/super.c | 32
1 files changed, 28 insertions(+), 4 deletions(-)
diff --git a/fs/ext2/super.c b/fs/ext2/super.c
index 639a32c..a433a53 100644
--- a/fs/ext2/super.c
+++ b/fs/ext2/super.c
@@ -603,11 +603,31 @@ static int ext2_check_descriptors (struct super_block *
sb)
static loff_t ext2_max_size(int bits)
{
loff_t res = EXT2_NDIR_BLOCKS;
- /* This constant is calculated to be the largest file size for a
-* dense, 4k-blocksize file such that the total number of
+ int meta_blocks;
+ loff_t upper_limit;
+
+ /* This is calculated to be the largest file size for a
+* dense, file such that the total number of
* sectors in the file, including data and all indirect blocks,
-* does not exceed 2^32. */
- const loff_t upper_limit = 0x1ff7fffd000LL;
+* does not exceed 2^32 -1
+* __u32 i_blocks representing the total number of
+* 512 bytes blocks of the file
+*/
+ upper_limit = (1LL << 32) - 1;
+
+ /* total blocks in file system block size */
+ upper_limit >>= (bits - 9);
+
+
+ /* indirect blocks */
+ meta_blocks = 1;
+ /* double indirect blocks */
+ meta_blocks += 1 + (1LL << (bits-2));
+ /* tripple indirect blocks */
+ meta_blocks += 1 + (1LL << (bits-2)) + (1LL << (2*(bits-2)));
+
+ upper_limit -= meta_blocks;
+ upper_limit <<= bits;
res += 1LL << (bits-2);
res += 1LL << (2*(bits-2));
@@ -615,6 +635,10 @@ static loff_t ext2_max_size(int bits)
res <<= bits;
if (res > upper_limit)
res = upper_limit;
+
+ if (res > MAX_LFS_FILESIZE)
+ res = MAX_LFS_FILESIZE;
+
return res;
}
--
1.5.3.4.206.g58ba4-dirty
-
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] ext3: Fix the max file size for ext3 file system.
The max file size for ext3 file system is now calculated
with hardcoded 4K block size. The patch fixes it to be
calculated with the right block size.
Signed-off-by: Aneesh Kumar K.V <[EMAIL PROTECTED]>
---
fs/ext3/super.c | 32
1 files changed, 28 insertions(+), 4 deletions(-)
diff --git a/fs/ext3/super.c b/fs/ext3/super.c
index 9537316..4f0de11 100644
--- a/fs/ext3/super.c
+++ b/fs/ext3/super.c
@@ -1362,11 +1362,31 @@ static void ext3_orphan_cleanup (struct super_block *
sb,
static loff_t ext3_max_size(int bits)
{
loff_t res = EXT3_NDIR_BLOCKS;
- /* This constant is calculated to be the largest file size for a
-* dense, 4k-blocksize file such that the total number of
+ int meta_blocks;
+ loff_t upper_limit;
+
+ /* This is calculated to be the largest file size for a
+* dense, file such that the total number of
* sectors in the file, including data and all indirect blocks,
-* does not exceed 2^32. */
- const loff_t upper_limit = 0x1ff7fffd000LL;
+* does not exceed 2^32 -1
+* __u32 i_blocks representing the total number of
+* 512 bytes blocks of the file
+*/
+ upper_limit = (1LL << 32) - 1;
+
+ /* total blocks in file system block size */
+ upper_limit >>= (bits - 9);
+
+
+ /* indirect blocks */
+ meta_blocks = 1;
+ /* double indirect blocks */
+ meta_blocks += 1 + (1LL << (bits-2));
+ /* tripple indirect blocks */
+ meta_blocks += 1 + (1LL << (bits-2)) + (1LL << (2*(bits-2)));
+
+ upper_limit -= meta_blocks;
+ upper_limit <<= bits;
res += 1LL << (bits-2);
res += 1LL << (2*(bits-2));
@@ -1374,6 +1394,10 @@ static loff_t ext3_max_size(int bits)
res <<= bits;
if (res > upper_limit)
res = upper_limit;
+
+ if (res > MAX_LFS_FILESIZE)
+ res = MAX_LFS_FILESIZE;
+
return res;
}
--
1.5.3.4.206.g58ba4-dirty
-
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2/2] ext2: Avoid rec_len overflow with 64KB block size
On Thu 04-10-07 13:12:07, Andrew Morton wrote:
> On Mon, 01 Oct 2007 17:35:46 -0700
> Mingming Cao <[EMAIL PROTECTED]> wrote:
>
> > ext2: Avoid rec_len overflow with 64KB block size
> >
> > From: Jan Kara <[EMAIL PROTECTED]>
> >
> > With 64KB blocksize, a directory entry can have size 64KB which does not fit
> > into 16 bits we have for entry lenght. So we store 0x instead and
> > convert
> > value when read from / written to disk.
>
> This patch clashes in non-trivial ways with
> ext2-convert-to-new-aops-fix.patch and perhaps other things which are
> already queued for 2.6.24 inclusion, so I'll need to ask for an updated
> patch, please.
>
> Also, I'm planing on merging the ext2 reservations code into 2.6.24, so if
> we're aiming for complete support of 64k blocksize in 2.6.24's ext2,
> additional testing and checking will be needed.
OK, attached is a patch diffed against 2.6.23-rc9-mm2 - does that work
fine for you?
Honza
--
Jan Kara <[EMAIL PROTECTED]>
SUSE Labs, CR
--
With 64KB blocksize, a directory entry can have size 64KB which does not fit
into 16 bits we have for entry lenght. So we store 0x instead and convert
value when read from / written to disk.
Signed-off-by: Jan Kara <[EMAIL PROTECTED]>
diff -rupX /home/jack/.kerndiffexclude linux-2.6.23-mm/fs/ext2/dir.c
linux-2.6.23-mm-1-ext2_64k_rec_len/fs/ext2/dir.c
--- linux-2.6.23-mm/fs/ext2/dir.c 2007-10-11 12:08:16.0 +0200
+++ linux-2.6.23-mm-1-ext2_64k_rec_len/fs/ext2/dir.c2007-10-11
12:14:24.0 +0200
@@ -28,6 +28,24 @@
typedef struct ext2_dir_entry_2 ext2_dirent;
+static inline unsigned ext2_rec_len_from_disk(__le16 dlen)
+{
+ unsigned len = le16_to_cpu(dlen);
+
+ if (len == EXT2_MAX_REC_LEN)
+ return 1 << 16;
+ return len;
+}
+
+static inline __le16 ext2_rec_len_to_disk(unsigned len)
+{
+ if (len == (1 << 16))
+ return cpu_to_le16(EXT2_MAX_REC_LEN);
+ else if (len > (1 << 16))
+ BUG();
+ return cpu_to_le16(len);
+}
+
/*
* ext2 uses block-sized chunks. Arguably, sector-sized ones would be
* more robust, but we have what we have
@@ -106,7 +124,7 @@ static void ext2_check_page(struct page
}
for (offs = 0; offs <= limit - EXT2_DIR_REC_LEN(1); offs += rec_len) {
p = (ext2_dirent *)(kaddr + offs);
- rec_len = le16_to_cpu(p->rec_len);
+ rec_len = ext2_rec_len_from_disk(p->rec_len);
if (rec_len < EXT2_DIR_REC_LEN(1))
goto Eshort;
@@ -204,7 +222,8 @@ static inline int ext2_match (int len, c
*/
static inline ext2_dirent *ext2_next_entry(ext2_dirent *p)
{
- return (ext2_dirent *)((char*)p + le16_to_cpu(p->rec_len));
+ return (ext2_dirent *)((char*)p +
+ ext2_rec_len_from_disk(p->rec_len));
}
static inline unsigned
@@ -316,7 +335,7 @@ ext2_readdir (struct file * filp, void *
return 0;
}
}
- filp->f_pos += le16_to_cpu(de->rec_len);
+ filp->f_pos += ext2_rec_len_from_disk(de->rec_len);
}
ext2_put_page(page);
}
@@ -425,7 +444,7 @@ void ext2_set_link(struct inode *dir, st
{
loff_t pos = page_offset(page) +
(char *) de - (char *) page_address(page);
- unsigned len = le16_to_cpu(de->rec_len);
+ unsigned len = ext2_rec_len_from_disk(de->rec_len);
int err;
lock_page(page);
@@ -482,7 +501,7 @@ int ext2_add_link (struct dentry *dentry
/* We hit i_size */
name_len = 0;
rec_len = chunk_size;
- de->rec_len = cpu_to_le16(chunk_size);
+ de->rec_len = ext2_rec_len_to_disk(chunk_size);
de->inode = 0;
goto got_it;
}
@@ -496,7 +515,7 @@ int ext2_add_link (struct dentry *dentry
if (ext2_match (namelen, name, de))
goto out_unlock;
name_len = EXT2_DIR_REC_LEN(de->name_len);
- rec_len = le16_to_cpu(de->rec_len);
+ rec_len = ext2_rec_len_from_disk(de->rec_len);
if (!de->inode && rec_len >= reclen)
goto got_it;
if (rec_len >= name_len + reclen)
@@ -518,8 +537,8 @@ got_it:
goto out_unlock;
if (de->inode) {
ext2_dirent *de1 = (ext2_dirent *) ((char *) de + name_len);
- de1->rec_len = cpu_to_le16(rec_len - name_len);
- de->rec_len = cpu_to_le16(name_len);
+ de1->rec_len = ext2
+ ext3-support-large-blocksize-up-to-pagesize.patch added to -mm tree
The patch titled
ext3: support large blocksize up to PAGESIZE
has been added to the -mm tree. Its filename is
ext3-support-large-blocksize-up-to-pagesize.patch
*** Remember to use Documentation/SubmitChecklist when testing your code ***
See http://www.zip.com.au/~akpm/linux/patches/stuff/added-to-mm.txt to find
out what to do about this
--
Subject: ext3: support large blocksize up to PAGESIZE
From: Takashi Sato <[EMAIL PROTECTED]>
This patch set supports large block size(>4k, <=64k) in ext3 just enlarging
the block size limit. But it is NOT possible to have 64kB blocksize on
ext3 without some changes to the directory handling code. The reason is
that an empty 64kB directory block would have a rec_len == (__u16)2^16 ==
0, and this would cause an error to be hit in the filesystem. The proposed
solution is treat 64k rec_len with a an impossible value like rec_len =
0x to handle this.
The Patch-set consists of the following 2 patches.
[1/2] ext3: enlarge blocksize
- Allow blocksize up to pagesize
[2/2] ext3: fix rec_len overflow
- prevent rec_len from overflow with 64KB blocksize
Now on 64k page ppc64 box runs with this patch set we could create a 64k
block size ext3, and able to handle empty directory block.
Signed-off-by: Takashi Sato <[EMAIL PROTECTED]>
Signed-off-by: Mingming Cao <[EMAIL PROTECTED]>
Cc:
Acked-by: Christoph Lameter <[EMAIL PROTECTED]>
Signed-off-by: Andrew Morton <[EMAIL PROTECTED]>
---
fs/ext3/super.c |6 +-
include/linux/ext3_fs.h |4 ++--
2 files changed, 7 insertions(+), 3 deletions(-)
diff -puN fs/ext3/super.c~ext3-support-large-blocksize-up-to-pagesize
fs/ext3/super.c
--- a/fs/ext3/super.c~ext3-support-large-blocksize-up-to-pagesize
+++ a/fs/ext3/super.c
@@ -1625,7 +1625,11 @@ static int ext3_fill_super (struct super
}
brelse (bh);
- sb_set_blocksize(sb, blocksize);
+ if (!sb_set_blocksize(sb, blocksize)) {
+ printk(KERN_ERR "EXT3-fs: bad blocksize %d.\n",
+ blocksize);
+ goto out_fail;
+ }
logic_sb_block = (sb_block * EXT3_MIN_BLOCK_SIZE) / blocksize;
offset = (sb_block * EXT3_MIN_BLOCK_SIZE) % blocksize;
bh = sb_bread(sb, logic_sb_block);
diff -puN include/linux/ext3_fs.h~ext3-support-large-blocksize-up-to-pagesize
include/linux/ext3_fs.h
--- a/include/linux/ext3_fs.h~ext3-support-large-blocksize-up-to-pagesize
+++ a/include/linux/ext3_fs.h
@@ -72,8 +72,8 @@
* Macro-instructions used to manage several block sizes
*/
#define EXT3_MIN_BLOCK_SIZE1024
-#defineEXT3_MAX_BLOCK_SIZE 4096
-#define EXT3_MIN_BLOCK_LOG_SIZE 10
+#defineEXT3_MAX_BLOCK_SIZE 65536
+#define EXT3_MIN_BLOCK_LOG_SIZE10
#ifdef __KERNEL__
# define EXT3_BLOCK_SIZE(s)((s)->s_blocksize)
#else
_
Patches currently in -mm which might be from [EMAIL PROTECTED] are
ext3-support-large-blocksize-up-to-pagesize.patch
-
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2/2] ext2: Avoid rec_len overflow with 64KB block size
On Thu, 11 Oct 2007 12:30:03 +0200 Jan Kara <[EMAIL PROTECTED]> wrote: > On Thu 04-10-07 16:11:21, Andrew Morton wrote: > > On Thu, 4 Oct 2007 16:40:44 -0600 > > Andreas Dilger <[EMAIL PROTECTED]> wrote: > > > > > On Oct 04, 2007 13:12 -0700, Andrew Morton wrote: > > > > On Mon, 01 Oct 2007 17:35:46 -0700 > > > > > ext2: Avoid rec_len overflow with 64KB block size > > > > > > > > > > into 16 bits we have for entry lenght. So we store 0x instead and > > > > > convert value when read from / written to disk. > > > > > > > > This patch clashes in non-trivial ways with > > > > ext2-convert-to-new-aops-fix.patch and perhaps other things which are > > > > already queued for 2.6.24 inclusion, so I'll need to ask for an updated > > > > patch, please. > > > > > > If the rel_len overflow patch isn't going to make it, then we also need > > > to revert the EXT*_MAX_BLOCK_SIZE change to 65536. It would be possible > > > to allow this to be up to 32768 w/o the rec_len overflow fix however. > > > > > > > Ok, thanks, I dropped ext3-support-large-blocksize-up-to-pagesize.patch and > > ext2-support-large-blocksize-up-to-pagesize.patch. > Sorry, for the delayed answer but I had some urgent bugs to fix... You exceeded my memory span. > Why did you drom ext3-support-large-blocksize-up-to-pagesize.patch? I forget. I'll bring it back and see what happens. > As far > as I understand your previous email (and also as I've checked against > 2.6.23-rc8-mm2), the patch fixing rec_len overflow clashes only for ext2... > I'll send you an updated patch for ext2 in a moment. ok.. I'm basically not applying anything any more - the whole thing is a teetering wreck. I need to go through the input queue delicately adding things which look important or relatively non-injurious. - To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2/2] ext2: Avoid rec_len overflow with 64KB block size
On Thu 04-10-07 16:11:21, Andrew Morton wrote: > On Thu, 4 Oct 2007 16:40:44 -0600 > Andreas Dilger <[EMAIL PROTECTED]> wrote: > > > On Oct 04, 2007 13:12 -0700, Andrew Morton wrote: > > > On Mon, 01 Oct 2007 17:35:46 -0700 > > > > ext2: Avoid rec_len overflow with 64KB block size > > > > > > > > into 16 bits we have for entry lenght. So we store 0x instead and > > > > convert value when read from / written to disk. > > > > > > This patch clashes in non-trivial ways with > > > ext2-convert-to-new-aops-fix.patch and perhaps other things which are > > > already queued for 2.6.24 inclusion, so I'll need to ask for an updated > > > patch, please. > > > > If the rel_len overflow patch isn't going to make it, then we also need > > to revert the EXT*_MAX_BLOCK_SIZE change to 65536. It would be possible > > to allow this to be up to 32768 w/o the rec_len overflow fix however. > > > > Ok, thanks, I dropped ext3-support-large-blocksize-up-to-pagesize.patch and > ext2-support-large-blocksize-up-to-pagesize.patch. Sorry, for the delayed answer but I had some urgent bugs to fix... Why did you drom ext3-support-large-blocksize-up-to-pagesize.patch? As far as I understand your previous email (and also as I've checked against 2.6.23-rc8-mm2), the patch fixing rec_len overflow clashes only for ext2... I'll send you an updated patch for ext2 in a moment. Honza -- Jan Kara <[EMAIL PROTECTED]> SUSE Labs, CR - To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ext4_inode difference between e2fsprogs and ext4
On Oct 11, 2007 14:30 +0530, Aneesh Kumar K.V wrote:
> In ext4 we have
>
> union {
>struct {
>__le16 l_i_reserved1; /* Obsoleted fragment
>number/size which are removed in ext4 */
>__le16 l_i_file_acl_high;
>__le16 l_i_uid_high; /* these 2 fields */
>__le16 l_i_gid_high; /* were reserved2[0] */
>__u32 l_i_reserved2;
>} linux2;
>
>
> Last week we were discussing about taking that l_i_reserved1 and using that
> for making file_acl 64 bit and using the lower l_i_reserved2 for 64 bit
> dir_acl.
We have l_i_file_acl_high already to give us 48-bit i_file_acl, which is
sufficient, IMHO. There is no need to have a larger i_dir_acl since this
field is only really used for i_size_high and should be renamed as such
instead of just being a macro.
> Now where will i put l_i_blocks_hi ?
Just where it is now - in l_i_reserved1...
Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.
-
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
ext4_inode difference between e2fsprogs and ext4
in e2fsprogs we have
union {
struct {
__u16 l_i_blocks_hi;
__u16 i_pad1;
__u16 l_i_uid_high; /* these 2 fields*/
__u16 l_i_gid_high; /* were reserved2[0] */
__u32 l_i_reserved2;
} linux2;
In ext4 we have
union {
struct {
__le16 l_i_reserved1; /* Obsoleted fragment
number/size which are removed in ext4 */
__le16 l_i_file_acl_high;
__le16 l_i_uid_high; /* these 2 fields */
__le16 l_i_gid_high; /* were reserved2[0] */
__u32 l_i_reserved2;
} linux2;
Last week we were discussing about taking that l_i_reserved1 and using that for
making file_acl 64 bit and using the
lower l_i_reserved2 for 64 bit dir_acl.
Now where will i put l_i_blocks_hi ?
-aneesh
-
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: uninit block group/mballoc and fsstress
On Oct 11, 2007 12:26 +0530, Aneesh Kumar K.V wrote: > Andreas Dilger wrote: > >On Oct 10, 2007 22:07 +0530, Aneesh Kumar K.V wrote: > >>I am hitting this with the below config > >> > >>mke2fs -j -I 256 -O lazy_bg $dev > > > >Note that "-O lazy_bg" has nothing really to do with the uninit_groups > >feature. Also, if you don't have the CFS e2fsprogs patches for > >uninit_groups then mke2fs creates an invalid filesystem if the journal is > >large (this was an unrelated bug we fixed while testing uninit_groups). > >Try running e2fsck right after the above mke2fs and you will likely > >see errors. > > > > e2fsprogs found at > > ftp://ftp.clusterfs.com/pub/lustre/other/e2fsprogs > > sets unint with -O lazy_bg. Is there any other option that i can use to set > the uninit block group feature ? Yes, "-O uninit_groups". Note that while "-O lazy_bg" does set the UNINIT flags on the groups (this is a COMPAT feature that marks all of the groups full in the group descriptors), it does NOT handle the inode high watermark or in-kernel initialization of the bitmaps. Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc. - To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
