On Fri, Apr 27, 2007 at 11:06:47AM -0400, Jeff Dike wrote:
> On Thu, Apr 26, 2007 at 09:58:25PM -0700, Valerie Henson wrote:
> > Here's an example, spelled out:
> >
> > Allocate file 1 in chunk A.
> > Grow file 1.
> > Chunk A fills up.
> > Allocate continuation inode for file 1 in chunk B.
> >
On Fri, Apr 27, 2007 at 11:06:47AM -0400, Jeff Dike wrote:
On Thu, Apr 26, 2007 at 09:58:25PM -0700, Valerie Henson wrote:
Here's an example, spelled out:
Allocate file 1 in chunk A.
Grow file 1.
Chunk A fills up.
Allocate continuation inode for file 1 in chunk B.
Chunk A gets some
On Fri, 27 April 2007 23:50:06 -0700, Valerie Henson wrote:
>
> Any mapping structure will have to be pre-allocated.
How much space would you allocate for it then? The required size surely
depends on the file size and fragmentation.
> So in my secret heart of hearts, I do indeed hope that
On Fri, Apr 27, 2007 at 12:53:34PM +0200, J??rn Engel wrote:
>
> All this would get easier if continuation inodes were known to be rare.
> You can ditch the doubly-linked list in favor of a pointer to the main
> inode then - traversing the list again is cheap, after all. And you can
> just try
On Fri, Apr 27, 2007 at 12:53:34PM +0200, J??rn Engel wrote:
All this would get easier if continuation inodes were known to be rare.
You can ditch the doubly-linked list in favor of a pointer to the main
inode then - traversing the list again is cheap, after all. And you can
just try to
On Fri, 27 April 2007 23:50:06 -0700, Valerie Henson wrote:
Any mapping structure will have to be pre-allocated.
How much space would you allocate for it then? The required size surely
depends on the file size and fragmentation.
So in my secret heart of hearts, I do indeed hope that cnodes
On Thu, Apr 26, 2007 at 09:58:25PM -0700, Valerie Henson wrote:
> Here's an example, spelled out:
>
> Allocate file 1 in chunk A.
> Grow file 1.
> Chunk A fills up.
> Allocate continuation inode for file 1 in chunk B.
> Chunk A gets some free space.
> Chunk B fills up.
> Pick chunk A for
On Thu, 26 April 2007 22:07:43 -0700, Valerie Henson wrote:
>
> What's important is that each continuation inode have a
> back pointer to the parent and that there is some structure for
> quickly looking up the continuation inode for a given file offset.
> Suggestions for data
On Thu, 26 April 2007 22:07:43 -0700, Valerie Henson wrote:
What's important is that each continuation inode have a
back pointer to the parent and that there is some structure for
quickly looking up the continuation inode for a given file offset.
Suggestions for data structures
On Thu, Apr 26, 2007 at 09:58:25PM -0700, Valerie Henson wrote:
Here's an example, spelled out:
Allocate file 1 in chunk A.
Grow file 1.
Chunk A fills up.
Allocate continuation inode for file 1 in chunk B.
Chunk A gets some free space.
Chunk B fills up.
Pick chunk A for allocating next
On Thu, Apr 26, 2007 at 10:47:38AM +0200, Jan Kara wrote:
> Do I get it right that you just have in each cnode a pointer to the
> previous & next cnode? But then if two consecutive cnodes get corrupted,
> you have no way to connect the chain, do you? If each cnode contained
> some unique
On Thu, Apr 26, 2007 at 12:05:04PM -0400, Jeff Dike wrote:
>
> No, I'm referring to a different file. The scenario is that you have
> a growing file in a nearly full disk with files being deleted (and
> thus space being freed) such that allocations for the growing file
> bounce back and forth
On Thu, 26 April 2007 10:47:40 +1000, David Chinner wrote:
>
> This assumes that you know a chunk has been corrupted, though.
> How do you find that out?
Option 1: you notice something odd while serving userspace.
Option 2: a checking/scrubbing daemon of some sorts.
The first will obviously
Jeff Dike wrote:
On Thu, Apr 26, 2007 at 10:53:16AM -0500, Amit Gud wrote:
Jeff Dike wrote:
How about this case:
Growing file starts in chunk A.
Overflows into chunk B.
Delete file in chunk A.
Growing file overflows chunk B and spots new free space in
chunk A
Alan Cox wrote:
Preventive measures are taken to limit only one continuation inode per
file per chunk. This can be done easily in the chunk allocation
algorithm for disk space. Although I'm not quite sure what you mean by
How are you handling the allocation in this situation, are you
> Preventive measures are taken to limit only one continuation inode per
> file per chunk. This can be done easily in the chunk allocation
> algorithm for disk space. Although I'm not quite sure what you mean by
How are you handling the allocation in this situation, are you assuming
that a
On Thu, Apr 26, 2007 at 10:53:16AM -0500, Amit Gud wrote:
> Jeff Dike wrote:
> >How about this case:
> >
> > Growing file starts in chunk A.
> > Overflows into chunk B.
> > Delete file in chunk A.
> > Growing file overflows chunk B and spots new free space in
> >chunk A (and
Jeff Dike wrote:
How about this case:
Growing file starts in chunk A.
Overflows into chunk B.
Delete file in chunk A.
Growing file overflows chunk B and spots new free space in
chunk A (and nothing anywhere else)
Overflows into chunk A
Delete file
On Wed, Apr 25, 2007 at 03:47:10PM -0700, Valerie Henson wrote:
> Actually, there is an upper limit on the number of continuation
> inodes. Each file can have a maximum of one continuation inode per
> chunk. (This is why we need to support sparse files.)
How about this case:
Growing
> On Wed, Apr 25, 2007 at 08:54:34PM +1000, David Chinner wrote:
> > On Tue, Apr 24, 2007 at 04:53:11PM -0500, Amit Gud wrote:
> > > -- --
> > > | cnode 0 |-->| cnode 0 |--> to another cnode or NULL
> > > -- --
> > > |
On Wed, Apr 25, 2007 at 08:54:34PM +1000, David Chinner wrote:
On Tue, Apr 24, 2007 at 04:53:11PM -0500, Amit Gud wrote:
-- --
| cnode 0 |--| cnode 0 |-- to another cnode or NULL
-- --
| cnode 1 |-
On Wed, Apr 25, 2007 at 03:47:10PM -0700, Valerie Henson wrote:
Actually, there is an upper limit on the number of continuation
inodes. Each file can have a maximum of one continuation inode per
chunk. (This is why we need to support sparse files.)
How about this case:
Growing file
Jeff Dike wrote:
How about this case:
Growing file starts in chunk A.
Overflows into chunk B.
Delete file in chunk A.
Growing file overflows chunk B and spots new free space in
chunk A (and nothing anywhere else)
Overflows into chunk A
Delete file
On Thu, Apr 26, 2007 at 10:53:16AM -0500, Amit Gud wrote:
Jeff Dike wrote:
How about this case:
Growing file starts in chunk A.
Overflows into chunk B.
Delete file in chunk A.
Growing file overflows chunk B and spots new free space in
chunk A (and nothing anywhere
Preventive measures are taken to limit only one continuation inode per
file per chunk. This can be done easily in the chunk allocation
algorithm for disk space. Although I'm not quite sure what you mean by
How are you handling the allocation in this situation, are you assuming
that a chunk
Alan Cox wrote:
Preventive measures are taken to limit only one continuation inode per
file per chunk. This can be done easily in the chunk allocation
algorithm for disk space. Although I'm not quite sure what you mean by
How are you handling the allocation in this situation, are you
Jeff Dike wrote:
On Thu, Apr 26, 2007 at 10:53:16AM -0500, Amit Gud wrote:
Jeff Dike wrote:
How about this case:
Growing file starts in chunk A.
Overflows into chunk B.
Delete file in chunk A.
Growing file overflows chunk B and spots new free space in
chunk A
On Thu, 26 April 2007 10:47:40 +1000, David Chinner wrote:
This assumes that you know a chunk has been corrupted, though.
How do you find that out?
Option 1: you notice something odd while serving userspace.
Option 2: a checking/scrubbing daemon of some sorts.
The first will obviously miss
On Thu, Apr 26, 2007 at 12:05:04PM -0400, Jeff Dike wrote:
No, I'm referring to a different file. The scenario is that you have
a growing file in a nearly full disk with files being deleted (and
thus space being freed) such that allocations for the growing file
bounce back and forth between
On Thu, Apr 26, 2007 at 10:47:38AM +0200, Jan Kara wrote:
Do I get it right that you just have in each cnode a pointer to the
previous next cnode? But then if two consecutive cnodes get corrupted,
you have no way to connect the chain, do you? If each cnode contained
some unique identifier
On Wed, Apr 25, 2007 at 04:03:44PM -0700, Valerie Henson wrote:
> On Wed, Apr 25, 2007 at 08:54:34PM +1000, David Chinner wrote:
> > On Tue, Apr 24, 2007 at 04:53:11PM -0500, Amit Gud wrote:
> > >
> > > The structure looks like this:
> > >
> > > -- --
> > > | cnode
On Wed, Apr 25, 2007 at 05:38:34AM -0600, Andreas Dilger wrote:
>
> The case where only a fsck of the corrupt chunk is done would not find the
> cnode references. Maybe there needs to be per-chunk info which contains
> a list/bitmap of other chunks that have cnodes shared with each chunk?
Yes,
On Wed, Apr 25, 2007 at 08:54:34PM +1000, David Chinner wrote:
> On Tue, Apr 24, 2007 at 04:53:11PM -0500, Amit Gud wrote:
> >
> > The structure looks like this:
> >
> > -- --
> > | cnode 0 |-->| cnode 0 |--> to another cnode or NULL
> > --
On Wed, Apr 25, 2007 at 03:34:03PM +0400, Nikita Danilov wrote:
>
> What is more important, design puts (as far as I can see) no upper limit
> on the number of continuation inodes, and hence, even if _average_ fsck
> time is greatly reduced, occasionally it can take more time than ext2 of
> the
On Tue, Apr 24, 2007 at 11:34:48PM +0400, Nikita Danilov wrote:
>
> Maybe I failed to describe the problem presicely.
>
> Suppose that all chunks have been checked. After that, for every inode
> I0 having continuations I1, I2, ... In, one has to check that every
> logical block is presented in
Andreas Dilger wrote:
How do you recover if fsfuzzer takes out a cnode in the chain? The
chunk is marked clean, but clearly corrupted and needs fixing and
you don't know what it was pointing at. Hence you have a pointer to
a trashed cnode *somewhere* that you need to find and fix, and a
bunch
On Wed, 25 Apr 2007, Nikita Danilov wrote:
David Lang writes:
> On Tue, 24 Apr 2007, Nikita Danilov wrote:
>
> > David Lang writes:
> > > On Tue, 24 Apr 2007, Nikita Danilov wrote:
> > >
> > > > Amit Gud writes:
> > > >
> > > > Hello,
> > > >
> > > > >
> > > > > This is an initial
On Apr 25, 2007 20:54 +1000, David Chinner wrote:
> On Tue, Apr 24, 2007 at 04:53:11PM -0500, Amit Gud wrote:
> > Right now, there is no distinction between an inode and continuation
> > inode (also referred to as 'cnode' below), except for the
> > EXT2_IS_CONT_FL flag. Every inode holds a list
David Lang writes:
> On Tue, 24 Apr 2007, Nikita Danilov wrote:
>
> > David Lang writes:
> > > On Tue, 24 Apr 2007, Nikita Danilov wrote:
> > >
> > > > Amit Gud writes:
> > > >
> > > > Hello,
> > > >
> > > > >
> > > > > This is an initial implementation of ChunkFS technique, briefly
On Tue, Apr 24, 2007 at 04:53:11PM -0500, Amit Gud wrote:
> Nikita Danilov wrote:
> >Maybe I failed to describe the problem presicely.
> >
> >Suppose that all chunks have been checked. After that, for every inode
> >I0 having continuations I1, I2, ... In, one has to check that every
> >logical
On Tue, Apr 24, 2007 at 04:53:11PM -0500, Amit Gud wrote:
Nikita Danilov wrote:
Maybe I failed to describe the problem presicely.
Suppose that all chunks have been checked. After that, for every inode
I0 having continuations I1, I2, ... In, one has to check that every
logical block is
David Lang writes:
On Tue, 24 Apr 2007, Nikita Danilov wrote:
David Lang writes:
On Tue, 24 Apr 2007, Nikita Danilov wrote:
Amit Gud writes:
Hello,
This is an initial implementation of ChunkFS technique, briefly
discussed
at:
On Apr 25, 2007 20:54 +1000, David Chinner wrote:
On Tue, Apr 24, 2007 at 04:53:11PM -0500, Amit Gud wrote:
Right now, there is no distinction between an inode and continuation
inode (also referred to as 'cnode' below), except for the
EXT2_IS_CONT_FL flag. Every inode holds a list of
On Wed, 25 Apr 2007, Nikita Danilov wrote:
David Lang writes:
On Tue, 24 Apr 2007, Nikita Danilov wrote:
David Lang writes:
On Tue, 24 Apr 2007, Nikita Danilov wrote:
Amit Gud writes:
Hello,
This is an initial implementation of ChunkFS technique, briefly
Andreas Dilger wrote:
How do you recover if fsfuzzer takes out a cnode in the chain? The
chunk is marked clean, but clearly corrupted and needs fixing and
you don't know what it was pointing at. Hence you have a pointer to
a trashed cnode *somewhere* that you need to find and fix, and a
bunch
On Tue, Apr 24, 2007 at 11:34:48PM +0400, Nikita Danilov wrote:
Maybe I failed to describe the problem presicely.
Suppose that all chunks have been checked. After that, for every inode
I0 having continuations I1, I2, ... In, one has to check that every
logical block is presented in at most
On Wed, Apr 25, 2007 at 03:34:03PM +0400, Nikita Danilov wrote:
What is more important, design puts (as far as I can see) no upper limit
on the number of continuation inodes, and hence, even if _average_ fsck
time is greatly reduced, occasionally it can take more time than ext2 of
the same
On Wed, Apr 25, 2007 at 08:54:34PM +1000, David Chinner wrote:
On Tue, Apr 24, 2007 at 04:53:11PM -0500, Amit Gud wrote:
The structure looks like this:
-- --
| cnode 0 |--| cnode 0 |-- to another cnode or NULL
-- --
On Wed, Apr 25, 2007 at 05:38:34AM -0600, Andreas Dilger wrote:
The case where only a fsck of the corrupt chunk is done would not find the
cnode references. Maybe there needs to be per-chunk info which contains
a list/bitmap of other chunks that have cnodes shared with each chunk?
Yes,
On Wed, Apr 25, 2007 at 04:03:44PM -0700, Valerie Henson wrote:
On Wed, Apr 25, 2007 at 08:54:34PM +1000, David Chinner wrote:
On Tue, Apr 24, 2007 at 04:53:11PM -0500, Amit Gud wrote:
The structure looks like this:
-- --
| cnode 0 |--|
Nikita Danilov wrote:
Maybe I failed to describe the problem presicely.
Suppose that all chunks have been checked. After that, for every inode
I0 having continuations I1, I2, ... In, one has to check that every
logical block is presented in at most one of these inodes. For this one
has to read
On Tue, 24 Apr 2007, Nikita Danilov wrote:
David Lang writes:
> On Tue, 24 Apr 2007, Nikita Danilov wrote:
>
> > Amit Gud writes:
> >
> > Hello,
> >
> > >
> > > This is an initial implementation of ChunkFS technique, briefly discussed
> > > at: http://lwn.net/Articles/190222 and
> > >
David Lang writes:
> On Tue, 24 Apr 2007, Nikita Danilov wrote:
>
> > Amit Gud writes:
> >
> > Hello,
> >
> > >
> > > This is an initial implementation of ChunkFS technique, briefly discussed
> > > at: http://lwn.net/Articles/190222 and
> > >
On Tue, 24 Apr 2007, Nikita Danilov wrote:
Amit Gud writes:
Hello,
>
> This is an initial implementation of ChunkFS technique, briefly discussed
> at: http://lwn.net/Articles/190222 and
> http://cis.ksu.edu/~gud/docs/chunkfs-hotdep-val-arjan-gud-zach.pdf
I have a couple of questions about
Amit Gud writes:
Hello,
>
> This is an initial implementation of ChunkFS technique, briefly discussed
> at: http://lwn.net/Articles/190222 and
> http://cis.ksu.edu/~gud/docs/chunkfs-hotdep-val-arjan-gud-zach.pdf
I have a couple of questions about chunkfs repair process.
First, as I
Amit Gud writes:
Hello,
This is an initial implementation of ChunkFS technique, briefly discussed
at: http://lwn.net/Articles/190222 and
http://cis.ksu.edu/~gud/docs/chunkfs-hotdep-val-arjan-gud-zach.pdf
I have a couple of questions about chunkfs repair process.
First, as I
On Tue, 24 Apr 2007, Nikita Danilov wrote:
Amit Gud writes:
Hello,
This is an initial implementation of ChunkFS technique, briefly discussed
at: http://lwn.net/Articles/190222 and
http://cis.ksu.edu/~gud/docs/chunkfs-hotdep-val-arjan-gud-zach.pdf
I have a couple of questions about
David Lang writes:
On Tue, 24 Apr 2007, Nikita Danilov wrote:
Amit Gud writes:
Hello,
This is an initial implementation of ChunkFS technique, briefly discussed
at: http://lwn.net/Articles/190222 and
http://cis.ksu.edu/~gud/docs/chunkfs-hotdep-val-arjan-gud-zach.pdf
On Tue, 24 Apr 2007, Nikita Danilov wrote:
David Lang writes:
On Tue, 24 Apr 2007, Nikita Danilov wrote:
Amit Gud writes:
Hello,
This is an initial implementation of ChunkFS technique, briefly discussed
at: http://lwn.net/Articles/190222 and
Nikita Danilov wrote:
Maybe I failed to describe the problem presicely.
Suppose that all chunks have been checked. After that, for every inode
I0 having continuations I1, I2, ... In, one has to check that every
logical block is presented in at most one of these inodes. For this one
has to read
Suparna Bhattacharya wrote:
Could you send this out as a patch to ext2 codebase, so we can just look
at the changes for chunkfs ? That might also make it small enough
to inline your patch in email for review.
What kind of results are you planning to gather to evaluate/optimize this ?
On Mon, Apr 23, 2007 at 09:58:49PM +0530, Suparna Bhattacharya wrote:
> On Mon, Apr 23, 2007 at 06:21:34AM -0500, Amit Gud wrote:
> >
> > This is an initial implementation of ChunkFS technique, briefly discussed
> > at: http://lwn.net/Articles/190222 and
> >
On Mon, Apr 23, 2007 at 06:21:34AM -0500, Amit Gud wrote:
>
> This is an initial implementation of ChunkFS technique, briefly discussed
> at: http://lwn.net/Articles/190222 and
> http://cis.ksu.edu/~gud/docs/chunkfs-hotdep-val-arjan-gud-zach.pdf
>
> This implementation is done within ext2
This is an initial implementation of ChunkFS technique, briefly discussed
at: http://lwn.net/Articles/190222 and
http://cis.ksu.edu/~gud/docs/chunkfs-hotdep-val-arjan-gud-zach.pdf
This implementation is done within ext2 driver. Every chunk is an
independent ext2 file system. The knowledge
This is an initial implementation of ChunkFS technique, briefly discussed
at: http://lwn.net/Articles/190222 and
http://cis.ksu.edu/~gud/docs/chunkfs-hotdep-val-arjan-gud-zach.pdf
This implementation is done within ext2 driver. Every chunk is an
independent ext2 file system. The knowledge
On Mon, Apr 23, 2007 at 06:21:34AM -0500, Amit Gud wrote:
This is an initial implementation of ChunkFS technique, briefly discussed
at: http://lwn.net/Articles/190222 and
http://cis.ksu.edu/~gud/docs/chunkfs-hotdep-val-arjan-gud-zach.pdf
This implementation is done within ext2 driver.
On Mon, Apr 23, 2007 at 09:58:49PM +0530, Suparna Bhattacharya wrote:
On Mon, Apr 23, 2007 at 06:21:34AM -0500, Amit Gud wrote:
This is an initial implementation of ChunkFS technique, briefly discussed
at: http://lwn.net/Articles/190222 and
Suparna Bhattacharya wrote:
Could you send this out as a patch to ext2 codebase, so we can just look
at the changes for chunkfs ? That might also make it small enough
to inline your patch in email for review.
What kind of results are you planning to gather to evaluate/optimize this ?
68 matches
Mail list logo
