Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-07-02 Thread Badari Pulavarty
On Thu, 2007-06-14 at 13:06 -0700, Andrew Morton wrote: > On Thu, 14 Jun 2007 12:38:39 -0700 > [EMAIL PROTECTED] wrote: > > > This patchset cleans up the page cache handling by replacing > > open coded shifts and adds through inline function calls. > Some of us (crazy) people are trying to suppo

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-17 Thread William Lee Irwin III
On Sun, 17 Jun 2007, Matt Mackall wrote: >> Is it? Last I looked it had reverted to handing out reverse-contiguous >> pages. On Sun, Jun 17, 2007 at 07:08:41PM -0700, Christoph Lameter wrote: > I thought that was fixed? Bill Irwin was working on it. > But the contiguous pages usually only work sho

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-17 Thread Arjan van de Ven
On Sun, 2007-06-17 at 19:08 -0700, Christoph Lameter wrote: > On Sun, 17 Jun 2007, Matt Mackall wrote: > > > Is it? Last I looked it had reverted to handing out reverse-contiguous > > pages. > > I thought that was fixed? Bill Irwin was working on it. > > But the contiguous pages usually only wor

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-17 Thread Christoph Lameter
On Sun, 17 Jun 2007, Matt Mackall wrote: > Is it? Last I looked it had reverted to handing out reverse-contiguous > pages. I thought that was fixed? Bill Irwin was working on it. But the contiguous pages usually only work shortly after boot. After awhile memory gets sufficiently scrambled that

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-16 Thread Matt Mackall
On Sat, Jun 16, 2007 at 06:25:00PM -0700, Arjan van de Ven wrote: > > > You: conceptully-new add-on which benefits 0.25% of the user base, provided > > they select the right config options and filesystem. > > > > Me: simpler enhancement which benefits 100% of the user base (ie: includes > > 4k bl

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-16 Thread Arjan van de Ven
> You: conceptully-new add-on which benefits 0.25% of the user base, provided > they select the right config options and filesystem. > > Me: simpler enhancement which benefits 100% of the user base (ie: includes > 4k blocksize, 4k pagesize) and which also fixes your performance problem > with tha

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-15 Thread Dave Kleikamp
On Thu, 2007-06-14 at 15:04 -0700, Andrew Morton wrote: > > On Thu, 14 Jun 2007 14:37:33 -0700 (PDT) Christoph Lameter <[EMAIL > > PROTECTED]> wrote: > > On Thu, 14 Jun 2007, Andrew Morton wrote: > > > > > We want the 100% case. > > > > Yes that is what we intend to do. Universal support for lar

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-15 Thread David Chinner
On Thu, Jun 14, 2007 at 07:23:40PM -0700, Andrew Morton wrote: > On Thu, 14 Jun 2007 19:04:27 -0700 (PDT) Christoph Lameter <[EMAIL > PROTECTED]> wrote: > > > > Of course there is. The seeks are reduced since there are an factor > > > > of 16 less metadata blocks. fsck does not read files. It jus

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-14 Thread Christoph Lameter
On Thu, 14 Jun 2007, Andrew Morton wrote: > > The metadata needs to refer to 1/16th of the earlier pages that need to be > > tracked. metadata is shrunk significantly. > > Only if the filesystems are altered to use larger blocksizes and if the > operator then chooses to use that feature. Then t

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-14 Thread Andrew Morton
On Thu, 14 Jun 2007 19:04:27 -0700 (PDT) Christoph Lameter <[EMAIL PROTECTED]> wrote: > > > Of course there is. The seeks are reduced since there are an factor > > > of 16 less metadata blocks. fsck does not read files. It just reads > > > metadata structures. And the larger contiguous areas th

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-14 Thread Christoph Lameter
On Thu, 14 Jun 2007, Andrew Morton wrote: > There will be files which should use 64k but which instead end up using 4k. > > There will be files which should use 4k but which instead end up using 64k. > > Because determining which size to use requires either operator intervention > or kernel heur

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-14 Thread Andrew Morton
On Thu, 14 Jun 2007 17:45:43 -0700 (PDT) Christoph Lameter <[EMAIL PROTECTED]> wrote: > On Thu, 14 Jun 2007, Andrew Morton wrote: > > > > I do not think that the 100% users will do kernel compiles all day like > > > we do. We likely would prefer 4k page size for our small text files. > > > > T

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-14 Thread Christoph Lameter
On Thu, 14 Jun 2007, Andrew Morton wrote: > > I do not think that the 100% users will do kernel compiles all day like > > we do. We likely would prefer 4k page size for our small text files. > > There are many, many applications which use small files. There is no problem with them using 4k page

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-14 Thread David Chinner
On Thu, Jun 14, 2007 at 04:41:18PM -0700, Andrew Morton wrote: > > On Fri, 15 Jun 2007 09:30:02 +1000 David Chinner <[EMAIL PROTECTED]> wrote: > > xfs_repair also uses direct I/O and does it's own userspace block > > caching and so avoids the problems involved with low memory, context > > unaware c

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-14 Thread David Chinner
On Thu, Jun 14, 2007 at 01:06:45PM -0700, Andrew Morton wrote: > On Thu, 14 Jun 2007 12:38:39 -0700 > [EMAIL PROTECTED] wrote: > > > This patchset cleans up the page cache handling by replacing > > open coded shifts and adds through inline function calls. > > If we never inflict variable PAGE_CAC

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-14 Thread Andrew Morton
> On Fri, 15 Jun 2007 09:30:02 +1000 David Chinner <[EMAIL PROTECTED]> wrote: > On Thu, Jun 14, 2007 at 03:04:17PM -0700, Andrew Morton wrote: > > fsck is single-threaded (hence no locking issues) and operates against the > > blockdev pagecache and does a _lot_ of small reads (indirect blocks, > >

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-14 Thread David Chinner
On Thu, Jun 14, 2007 at 03:04:17PM -0700, Andrew Morton wrote: > fsck is single-threaded (hence no locking issues) and operates against the > blockdev pagecache and does a _lot_ of small reads (indirect blocks, > especially). Commenting purely about the above statement (and not on large pages or b

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-14 Thread Andrew Morton
> On Thu, 14 Jun 2007 15:22:46 -0700 (PDT) Christoph Lameter <[EMAIL > PROTECTED]> wrote: > On Thu, 14 Jun 2007, Andrew Morton wrote: > > > With 64k pagesize the amount of memory required to hold a kernel tree (say) > > will go from 270MB to 1400MB. This is not an optimisation. > > I do not th

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-14 Thread Christoph Lameter
On Thu, 14 Jun 2007, Andrew Morton wrote: > With 64k pagesize the amount of memory required to hold a kernel tree (say) > will go from 270MB to 1400MB. This is not an optimisation. I do not think that the 100% users will do kernel compiles all day like we do. We likely would prefer 4k page siz

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-14 Thread Andrew Morton
> On Thu, 14 Jun 2007 14:37:33 -0700 (PDT) Christoph Lameter <[EMAIL > PROTECTED]> wrote: > On Thu, 14 Jun 2007, Andrew Morton wrote: > > > We want the 100% case. > > Yes that is what we intend to do. Universal support for larger blocksize. > I.e. your desktop filesystem will use 64k page size

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-14 Thread Christoph Lameter
On Thu, 14 Jun 2007, Andrew Morton wrote: > We want the 100% case. Yes that is what we intend to do. Universal support for larger blocksize. I.e. your desktop filesystem will use 64k page size and server platforms likely much larger. fsck times etc etc are becoming an issue for desktop systems

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-14 Thread Andrew Morton
> On Thu, 14 Jun 2007 14:20:04 -0700 (PDT) Christoph Lameter <[EMAIL > PROTECTED]> wrote: > > I think the best way to proceed would be to investigate that _general_ > > optimisation and then, based upon the results of that work, decide whether > > further _specialised_ changes such as variable PAG

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-14 Thread Dave McCracken
On Thursday 14 June 2007, Christoph Hellwig wrote: > Christophs patches are an extremly useful cleanup and can stand on their > own.  Right now PAGE_CACHE_SIZE and friends are in there and now one can > keep them distinct because their useage is not clear at all.  By making > the macros per-mapping

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-14 Thread Christoph Lameter
On Thu, 14 Jun 2007, Andrew Morton wrote: > If we never inflict variable PAGE_CACHE_SIZE upon the kernel, these changes > become pointless obfuscation. But there is no such resonable scenario that I am aware of unless we continue to add workarounds for the issues covered here to the VM. And it

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-14 Thread Christoph Hellwig
On Thu, Jun 14, 2007 at 01:06:45PM -0700, Andrew Morton wrote: > On Thu, 14 Jun 2007 12:38:39 -0700 > [EMAIL PROTECTED] wrote: > > > This patchset cleans up the page cache handling by replacing > > open coded shifts and adds through inline function calls. > > If we never inflict variable PAGE_CAC

Re: [patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-14 Thread Andrew Morton
On Thu, 14 Jun 2007 12:38:39 -0700 [EMAIL PROTECTED] wrote: > This patchset cleans up the page cache handling by replacing > open coded shifts and adds through inline function calls. If we never inflict variable PAGE_CACHE_SIZE upon the kernel, these changes become pointless obfuscation. Let's p

[patch 00/14] Page cache cleanup in anticipation of Large Blocksize support

2007-06-14 Thread clameter
This patchset cleans up the page cache handling by replacing open coded shifts and adds through inline function calls. The ultimate goal is to replace all uses of PAGE_CACHE_xxx in the kernel through the use of these functions. All the functions take a mapping parameter. This is in anticipation of