Re: [RFC PATCH 0/5] Union Mount: A Directory listing approach with lseek support

2007-12-06 Thread sfjro
Bharata B Rao: > - The cache can grow arbitrarily large in size for big directories thereby > consuming lots of memory. Pruning individual cache entries is out of question > as entire cache is needed for subsequent readdirs for duplicate elimination. Additionally, the memory usage may be a

Re: [RFC PATCH 0/5] Union Mount: A Directory listing approach with lseek support

2007-12-06 Thread Dave Hansen
On Thu, 2007-12-06 at 11:01 +0100, Jan Blunck wrote: > > Rather than give each _dirent_ an offset, could we give each sub-mount > > an offset? Let's say we have three members comprising a union mount > > directory. The first has 100 dirents, the second 200, and the third > > 10,000. When the

Re: [RFC PATCH 0/5] Union Mount: A Directory listing approach with lseek support

2007-12-06 Thread Bharata B Rao
On Thu, Dec 06, 2007 at 11:01:18AM +0100, Jan Blunck wrote: > On Wed, Dec 05, Dave Hansen wrote: > > > I think the key here is what kind of consistency we're trying to > > provide. If a directory is being changed underneath a reader, what > > kinds of guarantees do they get about the contents of

Re: [RFC PATCH 0/5] Union Mount: A Directory listing approach with lseek support

2007-12-06 Thread Jan Blunck
On Wed, Dec 05, Dave Hansen wrote: > I think the key here is what kind of consistency we're trying to > provide. If a directory is being changed underneath a reader, what > kinds of guarantees do they get about the contents of their directory > read? When do those guarantees start? Are there

Re: [RFC PATCH 0/5] Union Mount: A Directory listing approach with lseek support

2007-12-06 Thread Jan Blunck
On Wed, Dec 05, Dave Hansen wrote: I think the key here is what kind of consistency we're trying to provide. If a directory is being changed underneath a reader, what kinds of guarantees do they get about the contents of their directory read? When do those guarantees start? Are there any

Re: [RFC PATCH 0/5] Union Mount: A Directory listing approach with lseek support

2007-12-06 Thread Bharata B Rao
On Thu, Dec 06, 2007 at 11:01:18AM +0100, Jan Blunck wrote: On Wed, Dec 05, Dave Hansen wrote: I think the key here is what kind of consistency we're trying to provide. If a directory is being changed underneath a reader, what kinds of guarantees do they get about the contents of their

Re: [RFC PATCH 0/5] Union Mount: A Directory listing approach with lseek support

2007-12-06 Thread Dave Hansen
On Thu, 2007-12-06 at 11:01 +0100, Jan Blunck wrote: Rather than give each _dirent_ an offset, could we give each sub-mount an offset? Let's say we have three members comprising a union mount directory. The first has 100 dirents, the second 200, and the third 10,000. When the first

Re: [RFC PATCH 0/5] Union Mount: A Directory listing approach with lseek support

2007-12-06 Thread sfjro
Bharata B Rao: - The cache can grow arbitrarily large in size for big directories thereby consuming lots of memory. Pruning individual cache entries is out of question as entire cache is needed for subsequent readdirs for duplicate elimination. Additionally, the memory usage may be a problem

Re: [RFC PATCH 0/5] Union Mount: A Directory listing approach with lseek support

2007-12-05 Thread Dave Hansen
On Wed, 2007-12-05 at 20:07 +0530, Bharata B Rao wrote: > In this approach, the cached dirents are given offsets in the form of > linearly increasing indices/cookies (like 0, 1, 2,...). This helps us to > uniformly define offsets across all the directories of the union > irrespective of the type

[RFC PATCH 0/5] Union Mount: A Directory listing approach with lseek support

2007-12-05 Thread Bharata B Rao
Hi, In Union Mount, the merged view of directories of the union is obtained by enhancing readdir(2)/getdents(2) to read and merge the entries of all the directories by eliminating the duplicates. While we have tried a few approaches for this, none of them could perfectly solve all the problems.

[RFC PATCH 0/5] Union Mount: A Directory listing approach with lseek support

2007-12-05 Thread Bharata B Rao
Hi, In Union Mount, the merged view of directories of the union is obtained by enhancing readdir(2)/getdents(2) to read and merge the entries of all the directories by eliminating the duplicates. While we have tried a few approaches for this, none of them could perfectly solve all the problems.

Re: [RFC PATCH 0/5] Union Mount: A Directory listing approach with lseek support

2007-12-05 Thread Dave Hansen
On Wed, 2007-12-05 at 20:07 +0530, Bharata B Rao wrote: In this approach, the cached dirents are given offsets in the form of linearly increasing indices/cookies (like 0, 1, 2,...). This helps us to uniformly define offsets across all the directories of the union irrespective of the type of