There's a patch for ext2 below for a quick and dirty implementation of
block bitmap readahead. But since I'm looking for comments, feedback,
etc., rather than asking for the patch to be added to ext2 in the near
future, I'll start at the beginning:>
I'm working on what amounts to a data aquisition application. It
needs the highest file write bandwidth it can get; ideally, this would
be the same as the disk write bandwidth. (But I need a file system for
convenience when processing the data.)
I'm currently using ext2, and most of the time it works pretty well.
However, I run into problems when the file being written to fills one
block group causing the block bitmap for another block group to be
read (which happens every 6 seconds or so with the SCSI disk I'm using
for development/testing). Because the elevator is loaded down with
writes, it takes a long time for the block bitmap reads to complete
(about 0.05 seconds -- long compared with the seek time). While the
read of the relevant block bitmap is in progress, writes to the end of
the file stall.
(The part of my application which does the writing happens to be
in-kernel, though I expect that the same effect can be observed from
user space. The numbers are taken from measurements I made with
2.4.1-pre9.)
So when the block bitmap read is going on, my application has to
buffer data. The buffering requirements across a block bitmap read
are much greater than the requirements in other circumstances. And
because it's working close to the disk write bandwidth, it can take a
couple of seconds for the buffer to drain.
So while trying to work out whether this can be avoided, I came up
with the patch below. It reads-ahead the next block bitmap when the
current block bitmap seems to be getting full (based on a simple
heuristic). For my application, at least, it works very nicely. The
seek to do the bitmap read still takes place, but with the patch it
happens long before the dependent writes so that they do not get
stalled.
The patch below spits out a message whenever it does a readahead.
Typical use of a desktop machine seems to incur few readaheads. I
haven't tried it for server workloads yet.
It still needs some work. With my data aquisition application, it
will often do multiple readaheads for the same block bitmap. I can
probably get rid of those by tuning the heuristic, but I want to
understand more about what causes them first.
Is this something worth including in ext2? It only makes a difference
to a very specific workload, but for that workload it gives a
significant improvement, and it doesn't involve much extra code.
One possibility is that ext2 is the simply the wrong filesystem for my
application, and any other applications that needs guaranteed
sustained write bandwidth. The obvious alternative is XFS. However,
early releases of Linux XFS will not feature the guaranteed rate I/O
features, so that's not likely to be an option for quite a while yet.
David Wragg
(This applies to 2.4.0-pre11 onwards)
diff -rua linux-2.4.0test11pre6/fs/ext2/balloc.c
linux-2.4.0test11pre6.e2ra/fs/ext2/balloc.c
--- linux-2.4.0test11pre6/fs/ext2/balloc.c Wed Oct 11 19:55:32 2000
+++ linux-2.4.0test11pre6.e2ra/fs/ext2/balloc.c Sun Nov 19 21:59:52 2000
@@ -107,6 +107,43 @@
}
/*
+ * Find the next group after block_group with free blocks, and arrange
+ * for its bitmap to be loaded.
+ */
+static void readahead_block_bitmap (struct super_block * sb,
+ unsigned int block_group)
+{
+ int i;
+ struct ext2_group_desc * gdp;
+ struct buffer_head * bh;
+ unsigned int obg = block_group;
+
+ for (i = 0; i < sb->u.ext2_sb.s_groups_count; i++) {
+ block_group++;
+ if (block_group >= sb->u.ext2_sb.s_groups_count)
+ block_group = 0;
+ gdp = ext2_get_group_desc (sb, block_group, NULL);
+ if (!gdp)
+ return;
+ if (le16_to_cpu(gdp->bg_free_blocks_count) > 0)
+ break;
+ }
+ if (i >= sb->u.ext2_sb.s_groups_count)
+ return;
+
+ bh = getblk(sb->s_dev, le32_to_cpu(gdp->bg_block_bitmap),
+ sb->s_blocksize);
+ if (!buffer_uptodate(bh)) {
+ ext2_warning(sb, "readahead_block_bitmap",
+ "from %u to %u", obg, block_group);
+ ll_rw_block(READA, 1, &bh);
+ }
+
+ brelse(bh);
+}
+
+
+/*
* load_block_bitmap loads the block bitmap for a blocks group
*
* It maintains a cache for the last bitmaps loaded. This cache is managed
@@ -358,7 +395,7 @@
struct buffer_head * bh;
struct buffer_head * bh2;
char * p, * r;
- int i, j, k, tmp;
+ int i, j, k, tmp, fbc;
int bitmap_nr;
struct super_block * sb;
struct ext2_group_desc * gdp;
@@ -551,9 +588,12 @@
*prealloc_block = next_block;
/* Writer: end */
+
+ fbc = le16_to_cpu(gdp->bg_free_blocks_count);
+
for (k = 1;
k < prealloc_goal && (j + k) < EXT2_BLOCKS_PER_GROUP(sb);
- k++, next_block++) {
+ k++, next_block++, fbc--) {
if (DQUOT_PREALLOC_BLOCK(sb, inode, 1))
break;
/* Writer: ->i_prealloc* */
@@ -565,14 +605,15 @@
}
(*prealloc_count)++;
/* Writer: end */
+
+ if ((fbc & (fbc - 1)) == 0)
+ readahead_block_bitmap(sb, i);
}
/*
* As soon as we go for per-group spinlocks we'll need these
* done inside the loop above.
*/
- gdp->bg_free_blocks_count =
- cpu_to_le16(le16_to_cpu(gdp->bg_free_blocks_count) -
- (k - 1));
+ gdp->bg_free_blocks_count = cpu_to_le16(fbc);
es->s_free_blocks_count =
cpu_to_le32(le32_to_cpu(es->s_free_blocks_count) -
(k - 1));
@@ -600,7 +641,19 @@
ext2_debug ("allocating block %d. "
"Goal hits %d of %d.\n", j, goal_hits, goal_attempts);
- gdp->bg_free_blocks_count = cpu_to_le16(le16_to_cpu(gdp->bg_free_blocks_count)
- 1);
+ fbc = le16_to_cpu(gdp->bg_free_blocks_count);
+
+ /*
+ * If the free blocks count is a power of two, read in the
+ * next bitmap. This is fairly arbitrary, but the intention
+ * is that as the group gets fuller, we are more likely to
+ * fill the group in the near future, as so a bitmap readahead
+ * is more likely to be profitable.
+ */
+ if ((fbc & (fbc - 1)) == 0)
+ readahead_block_bitmap(sb, i);
+
+ gdp->bg_free_blocks_count = cpu_to_le16(fbc - 1);
mark_buffer_dirty(bh2);
es->s_free_blocks_count = cpu_to_le32(le32_to_cpu(es->s_free_blocks_count) -
1);
mark_buffer_dirty(sb->u.ext2_sb.s_sbh);
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to [EMAIL PROTECTED]