Re: [REGRESSION] 2.6.19/2.6.20-rc3 buffered write slowdown

2007-01-11 Thread Christoph Lameter
On Fri, 12 Jan 2007, Nick Piggin wrote: > Ah yes... Can't you force it on if you have a NUMA complied kernel? But it wont do anything since it only comes into action if you have an off node allocation. If you run a NUMA kernel on an SMP system then you only have one node. There is no way that a

Re: [REGRESSION] 2.6.19/2.6.20-rc3 buffered write slowdown

2007-01-11 Thread Nick Piggin
Christoph Lameter wrote: On Thu, 11 Jan 2007, Nick Piggin wrote: You're not turning on zone_reclaim, by any chance, are you? It is not a NUMA system so zone reclaim is not available. Ah yes... Can't you force it on if you have a NUMA complied kernel? zone reclaim was already in 2.6.16.

Re: [REGRESSION] 2.6.19/2.6.20-rc3 buffered write slowdown

2007-01-11 Thread Christoph Lameter
On Thu, 11 Jan 2007, Nick Piggin wrote: > You're not turning on zone_reclaim, by any chance, are you? It is not a NUMA system so zone reclaim is not available. zone reclaim was already in 2.6.16. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message t

Re: [REGRESSION] 2.6.19/2.6.20-rc3 buffered write slowdown

2007-01-11 Thread Nick Piggin
David Chinner wrote: On Thu, Jan 11, 2007 at 12:08:10PM +1100, Nick Piggin wrote: So, what I've attached is three files which have both 'vmstat 5' output and 'iostat 5 |grep dm-' output in them. Ahh, sorry to be unclear, I meant: cat /proc/vmstat > pre run_test cat /proc/vmstat > post

Re: [REGRESSION] 2.6.19/2.6.20-rc3 buffered write slowdown

2007-01-11 Thread Nick Piggin
Thanks. BTW. You didn't cc this to the list, so I won't either in case you want it kept private. David Chinner wrote: On Thu, Jan 11, 2007 at 12:08:10PM +1100, Nick Piggin wrote: Ahh, sorry to be unclear, I meant: cat /proc/vmstat > pre run_test cat /proc/vmstat > post 6 files attached

Re: [REGRESSION] 2.6.19/2.6.20-rc3 buffered write slowdown

2007-01-10 Thread David Chinner
On Wed, Jan 10, 2007 at 05:40:26PM -0800, Christoph Lameter wrote: > On Thu, 11 Jan 2007, David Chinner wrote: > > > On Wed, Jan 10, 2007 at 04:43:36PM -0800, Christoph Lameter wrote: > > > You are comparing a debian 2.6.18 standard kernel with your tuned version > > > of 2.6.20-rc3. There may be

Re: [REGRESSION] 2.6.19/2.6.20-rc3 buffered write slowdown

2007-01-10 Thread Christoph Lameter
On Thu, 11 Jan 2007, David Chinner wrote: > On Wed, Jan 10, 2007 at 04:43:36PM -0800, Christoph Lameter wrote: > > You are comparing a debian 2.6.18 standard kernel with your tuned version > > of 2.6.20-rc3. There may be a lot of differences. Could you get us the > > config? Or use the same conf

Re: [REGRESSION] 2.6.19/2.6.20-rc3 buffered write slowdown

2007-01-10 Thread David Chinner
On Thu, Jan 11, 2007 at 12:08:10PM +1100, Nick Piggin wrote: > David Chinner wrote: > >Sure, but that doesn't really show the how erratic the per-filesystem > >throughput is because the test I'm running is PCI-X bus limited in > >it's throughput at about 750MB/s. Each dm device is capable of about

Re: [REGRESSION] 2.6.19/2.6.20-rc3 buffered write slowdown

2007-01-10 Thread David Chinner
On Thu, Jan 11, 2007 at 10:08:55AM +1100, David Chinner wrote: > On Wed, Jan 10, 2007 at 03:04:15PM -0800, Christoph Lameter wrote: > > On Thu, 11 Jan 2007, David Chinner wrote: > > > > > The performance and smoothness is fully restored on 2.6.20-rc3 > > > by setting dirty_ratio down to 10 (from t

Re: [REGRESSION] 2.6.19/2.6.20-rc3 buffered write slowdown

2007-01-10 Thread Nick Piggin
David Chinner wrote: On Thu, Jan 11, 2007 at 10:13:55AM +1100, Nick Piggin wrote: David Chinner wrote: On Wed, Jan 10, 2007 at 03:04:15PM -0800, Christoph Lameter wrote: On Thu, 11 Jan 2007, David Chinner wrote: The performance and smoothness is fully restored on 2.6.20-rc3 by setting

Re: [REGRESSION] 2.6.19/2.6.20-rc3 buffered write slowdown

2007-01-10 Thread David Chinner
On Wed, Jan 10, 2007 at 04:43:36PM -0800, Christoph Lameter wrote: > You are comparing a debian 2.6.18 standard kernel with your tuned version > of 2.6.20-rc3. There may be a lot of differences. Could you get us the > config? Or use the same config file and build 2.6.20/18 the same way. I took t

Re: [REGRESSION] 2.6.19/2.6.20-rc3 buffered write slowdown

2007-01-10 Thread Christoph Lameter
You are comparing a debian 2.6.18 standard kernel with your tuned version of 2.6.20-rc3. There may be a lot of differences. Could you get us the config? Or use the same config file and build 2.6.20/18 the same way. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the

Re: [REGRESSION] 2.6.19/2.6.20-rc3 buffered write slowdown

2007-01-10 Thread David Chinner
On Thu, Jan 11, 2007 at 10:13:55AM +1100, Nick Piggin wrote: > David Chinner wrote: > >On Wed, Jan 10, 2007 at 03:04:15PM -0800, Christoph Lameter wrote: > > > >>On Thu, 11 Jan 2007, David Chinner wrote: > >> > >> > >>>The performance and smoothness is fully restored on 2.6.20-rc3 > >>>by setting d

Re: [REGRESSION] 2.6.19/2.6.20-rc3 buffered write slowdown

2007-01-10 Thread David Chinner
On Wed, Jan 10, 2007 at 03:12:02PM -0800, Christoph Lameter wrote: > On Thu, 11 Jan 2007, David Chinner wrote: > > > Well, pdflush appears to be doing very little on both 2.6.18 and > > 2.6.20-rc3. In both cases kswapd is consuming 10-20% of a CPU and > > all of the pdflush threads combined (I've

Re: [REGRESSION] 2.6.19/2.6.20-rc3 buffered write slowdown

2007-01-10 Thread Nick Piggin
David Chinner wrote: On Wed, Jan 10, 2007 at 03:04:15PM -0800, Christoph Lameter wrote: On Thu, 11 Jan 2007, David Chinner wrote: The performance and smoothness is fully restored on 2.6.20-rc3 by setting dirty_ratio down to 10 (from the default 40), so something in the VM is not working as w

Re: [REGRESSION] 2.6.19/2.6.20-rc3 buffered write slowdown

2007-01-10 Thread Christoph Lameter
On Thu, 11 Jan 2007, David Chinner wrote: > Well, pdflush appears to be doing very little on both 2.6.18 and > 2.6.20-rc3. In both cases kswapd is consuming 10-20% of a CPU and > all of the pdflush threads combined (I've seen up to 7 active at > once) use maybe 1-2% of cpu time. This occurs regard

Re: [REGRESSION] 2.6.19/2.6.20-rc3 buffered write slowdown

2007-01-10 Thread David Chinner
On Wed, Jan 10, 2007 at 03:04:15PM -0800, Christoph Lameter wrote: > On Thu, 11 Jan 2007, David Chinner wrote: > > > The performance and smoothness is fully restored on 2.6.20-rc3 > > by setting dirty_ratio down to 10 (from the default 40), so > > something in the VM is not working as well as it u

Re: [REGRESSION] 2.6.19/2.6.20-rc3 buffered write slowdown

2007-01-10 Thread Christoph Lameter
On Thu, 11 Jan 2007, David Chinner wrote: > The performance and smoothness is fully restored on 2.6.20-rc3 > by setting dirty_ratio down to 10 (from the default 40), so > something in the VM is not working as well as it used to dirty_background_ratio is left as is at 10? So you gain performan

[REGRESSION] 2.6.19/2.6.20-rc3 buffered write slowdown

2007-01-10 Thread David Chinner
Discussion thread: http://oss.sgi.com/archives/xfs/2007-01/msg00052.html Short story is that buffered writes slowed down by 20-30% between 2.6.18 and 2.6.19 and became a lot more erratic. Writing a single file to a single filesystem doesn't appear to have major problems, but when writing a file