Re: [PATCH] mm: swap prefetch improvements

2007-05-23 Thread Ash Milsted
On Wed, 23 May 2007 08:50:01 +1000 Con Kolivas <[EMAIL PROTECTED]> wrote: > On Wednesday 23 May 2007 06:42, Ash Milsted wrote: > > Hi. I just did some video encoding on my desktop and I was noticing > > (for the first time in a while) that running apps had to hit swap quite > > a lot when I switch

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Con Kolivas
On Wednesday 23 May 2007 06:42, Ash Milsted wrote: > Hi. I just did some video encoding on my desktop and I was noticing > (for the first time in a while) that running apps had to hit swap quite > a lot when I switched to them (the encoding was going at full blast for > most of the day, and most of

Re: [ck] Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Ash Milsted
On Tue, 22 May 2007 20:37:54 +1000 Con Kolivas <[EMAIL PROTECTED]> wrote: > On Tuesday 22 May 2007 20:25, Ingo Molnar wrote: > > * Con Kolivas <[EMAIL PROTECTED]> wrote: > > > > > there was nothing else running on the system - so i suspect the > > > > > swapin activity flagged 'itself' as some 'ot

Re: [ck] Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Ingo Molnar
* Michael Chang <[EMAIL PROTECTED]> wrote: > > It clearly should not consider 'itself' as IO activity. This > > suggests some bug in the 'detect activity' mechanism, agreed? I'm > > wondering whether you are seeing the same problem, or is all > > swap-prefetch IO on your system continuous unti

Re: [ck] Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Michael Chang
On 5/22/07, Ingo Molnar <[EMAIL PROTECTED]> wrote: * Con Kolivas <[EMAIL PROTECTED]> wrote: > On Tuesday 22 May 2007 20:25, Ingo Molnar wrote: > > * Con Kolivas <[EMAIL PROTECTED]> wrote: > > > > > there was nothing else running on the system - so i suspect the > > > > > swapin activity flagged

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Ingo Molnar
* Con Kolivas <[EMAIL PROTECTED]> wrote: > > hm, what do you call 'journal writeout' here that would be happening > > on my system? > > Not really sure what you have in terms of fs, but here even with > nothing going on, ext3 writes to disk every 5 seconds with kjournald. i have ext3, but it

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Con Kolivas
On Tuesday 22 May 2007 20:57, Ingo Molnar wrote: > * Con Kolivas <[EMAIL PROTECTED]> wrote: > > On Tuesday 22 May 2007 20:46, Ingo Molnar wrote: > > > It clearly should not consider 'itself' as IO activity. This > > > suggests some bug in the 'detect activity' mechanism, agreed? I'm > > > wondering

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Ingo Molnar
* Con Kolivas <[EMAIL PROTECTED]> wrote: > On Tuesday 22 May 2007 20:46, Ingo Molnar wrote: > > It clearly should not consider 'itself' as IO activity. This > > suggests some bug in the 'detect activity' mechanism, agreed? I'm > > wondering whether you are seeing the same problem, or is all >

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Con Kolivas
On Tuesday 22 May 2007 20:46, Ingo Molnar wrote: > It clearly should not consider 'itself' as IO activity. This suggests > some bug in the 'detect activity' mechanism, agreed? I'm wondering > whether you are seeing the same problem, or is all swap-prefetch IO on > your system continuous until it's

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Ingo Molnar
* Con Kolivas <[EMAIL PROTECTED]> wrote: > On Tuesday 22 May 2007 20:25, Ingo Molnar wrote: > > * Con Kolivas <[EMAIL PROTECTED]> wrote: > > > > > there was nothing else running on the system - so i suspect the > > > > > swapin activity flagged 'itself' as some 'other' activity and > > > > > stop

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Con Kolivas
On Tuesday 22 May 2007 20:25, Ingo Molnar wrote: > * Con Kolivas <[EMAIL PROTECTED]> wrote: > > > > there was nothing else running on the system - so i suspect the > > > > swapin activity flagged 'itself' as some 'other' activity and > > > > stopped? The swapins happened in 4 bursts, separated by 5

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Ingo Molnar
* Con Kolivas <[EMAIL PROTECTED]> wrote: > > > there was nothing else running on the system - so i suspect the > > > swapin activity flagged 'itself' as some 'other' activity and > > > stopped? The swapins happened in 4 bursts, separated by 5 seconds > > > total idleness. > > > > I've noted bu

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Con Kolivas
On Tuesday 22 May 2007 20:15, Antonino Ingargiola wrote: > 2007/5/21, Ingo Molnar <[EMAIL PROTECTED]>: > > * Con Kolivas <[EMAIL PROTECTED]> wrote: > > > > A suggestion for improvement: right now swap-prefetch does a small > > > > bit of swapin every 5 seconds and stays idle inbetween. Could this >

Re: [PATCH] mm: swap prefetch improvements

2007-05-22 Thread Antonino Ingargiola
2007/5/21, Ingo Molnar <[EMAIL PROTECTED]>: * Con Kolivas <[EMAIL PROTECTED]> wrote: > > A suggestion for improvement: right now swap-prefetch does a small > > bit of swapin every 5 seconds and stays idle inbetween. Could this > > perhaps be made more agressive (optionally perhaps), if the syst

Re: [PATCH] mm: swap prefetch improvements

2007-05-21 Thread Ingo Molnar
* Con Kolivas <[EMAIL PROTECTED]> wrote: > > A suggestion for improvement: right now swap-prefetch does a small > > bit of swapin every 5 seconds and stays idle inbetween. Could this > > perhaps be made more agressive (optionally perhaps), if the system > > is not swapping otherwise? If block-

Re: [PATCH] mm: swap prefetch improvements

2007-05-21 Thread Con Kolivas
On Monday 21 May 2007 20:03, Ingo Molnar wrote: > * Con Kolivas <[EMAIL PROTECTED]> wrote: > > It turns out that fixing swap prefetch was not that hard to fix and > > improve upon, and since Andrew hasn't dropped swap prefetch, instead > > here are a swag of fixes and improvements, [...] > > it's a

Re: [PATCH] mm: swap prefetch improvements

2007-05-21 Thread Ingo Molnar
* Con Kolivas <[EMAIL PROTECTED]> wrote: > It turns out that fixing swap prefetch was not that hard to fix and > improve upon, and since Andrew hasn't dropped swap prefetch, instead > here are a swag of fixes and improvements, [...] it's a reliable win on my testbox too: # echo 1 > /proc/sys

Re: [PATCH] mm: swap prefetch improvements

2007-05-12 Thread Paul Jackson
Con wrote: > Ok so change the default value for swap_prefetch to 0 when CPUSETS is > enabled? I don't see why that special case for cpusets is needed. I'm suggesting making no special cases for CPUSETS at all, until and unless we find reason to. In other words, I'm suggesting simply removing t

Re: [PATCH] mm: swap prefetch improvements

2007-05-12 Thread Con Kolivas
On Saturday 12 May 2007 18:14, Paul Jackson wrote: > > Ummm this is what I've been saying for over a year now but noone has been > > listening. > > Well ... if there is a problem using prefetch and cpusets together, > it doesn't look like the two of us are going to find it. > > I should probably lo

Re: [PATCH] mm: swap prefetch improvements

2007-05-12 Thread Paul Jackson
> Ummm this is what I've been saying for over a year now but noone has been > listening. Well ... if there is a problem using prefetch and cpusets together, it doesn't look like the two of us are going to find it. I should probably look at your patch to answer this next question, but being a laz

Re: [PATCH] mm: swap prefetch improvements

2007-05-12 Thread Con Kolivas
On Saturday 12 May 2007 15:51, Paul Jackson wrote: > Con wrote: > > Hmm I'm not really sure what it takes to make it cpuset aware; > > ... > > It is numa aware to some degree. It stores the node id and when it starts > > prefetching it only prefetches to nodes that are suitable for prefetching > >

Re: [PATCH] mm: swap prefetch improvements

2007-05-11 Thread Paul Jackson
Con wrote: > Hmm I'm not really sure what it takes to make it cpuset aware; > ... > It is numa aware to some degree. It stores the node id and when it starts > prefetching it only prefetches to nodes that are suitable for prefetching to > ... > It would be absolutely trivial to add a check for 'n

Re: [PATCH] mm: swap prefetch improvements

2007-05-11 Thread Con Kolivas
On Saturday 12 May 2007 15:03, Paul Jackson wrote: > > Swap prefetch is not cpuset aware so make the config option depend on > > !CPUSETS. > > Ok. > > Could you explain what it means to say "swap prefetch is not cpuset aware", > or could you give a rough idea of what it would take to make it cpuset

Re: [PATCH] mm: swap prefetch improvements

2007-05-11 Thread Paul Jackson
> Swap prefetch is not cpuset aware so make the config option depend on > !CPUSETS. Ok. Could you explain what it means to say "swap prefetch is not cpuset aware", or could you give a rough idea of what it would take to make it cpuset aware? I wouldn't go so far as to say that no one would ever

[PATCH] mm: swap prefetch improvements

2007-05-11 Thread Con Kolivas
It turns out that fixing swap prefetch was not that hard to fix and improve upon, and since Andrew hasn't dropped swap prefetch, instead here are a swag of fixes and improvements, including making it depend on !CPUSETS as Nick requested. These changes lead to dramatic improvements. Eg on a mac