Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-11-18 Thread Shaohua Li
On Wed, Nov 14, 2012 at 01:48:18AM -0800, Hugh Dickins wrote: > On Tue, 6 Nov 2012, Shaohua Li wrote: > > On Wed, Oct 24, 2012 at 09:13:56AM +0800, Shaohua Li wrote: > > > On Tue, Oct 23, 2012 at 09:41:00AM -0400, Rik van Riel wrote: > > > > On 10/23/2012 01:51 AM, Shaohua Li wrote: > > > > > > >

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-11-18 Thread Shaohua Li
On Wed, Nov 14, 2012 at 01:48:18AM -0800, Hugh Dickins wrote: On Tue, 6 Nov 2012, Shaohua Li wrote: On Wed, Oct 24, 2012 at 09:13:56AM +0800, Shaohua Li wrote: On Tue, Oct 23, 2012 at 09:41:00AM -0400, Rik van Riel wrote: On 10/23/2012 01:51 AM, Shaohua Li wrote: I have no

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-11-14 Thread Hugh Dickins
On Tue, 6 Nov 2012, Shaohua Li wrote: > On Wed, Oct 24, 2012 at 09:13:56AM +0800, Shaohua Li wrote: > > On Tue, Oct 23, 2012 at 09:41:00AM -0400, Rik van Riel wrote: > > > On 10/23/2012 01:51 AM, Shaohua Li wrote: > > > > > > >I have no strong point against the global state method. But I'd agree

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-11-14 Thread Hugh Dickins
On Tue, 6 Nov 2012, Shaohua Li wrote: On Wed, Oct 24, 2012 at 09:13:56AM +0800, Shaohua Li wrote: On Tue, Oct 23, 2012 at 09:41:00AM -0400, Rik van Riel wrote: On 10/23/2012 01:51 AM, Shaohua Li wrote: I have no strong point against the global state method. But I'd agree making

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-11-05 Thread Shaohua Li
On Wed, Oct 24, 2012 at 09:13:56AM +0800, Shaohua Li wrote: > On Tue, Oct 23, 2012 at 09:41:00AM -0400, Rik van Riel wrote: > > On 10/23/2012 01:51 AM, Shaohua Li wrote: > > > > >I have no strong point against the global state method. But I'd agree > > >making the > > >heuristic simple is

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-11-05 Thread Shaohua Li
On Wed, Oct 24, 2012 at 09:13:56AM +0800, Shaohua Li wrote: On Tue, Oct 23, 2012 at 09:41:00AM -0400, Rik van Riel wrote: On 10/23/2012 01:51 AM, Shaohua Li wrote: I have no strong point against the global state method. But I'd agree making the heuristic simple is preferred currently.

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-23 Thread Shaohua Li
On Tue, Oct 23, 2012 at 09:41:00AM -0400, Rik van Riel wrote: > On 10/23/2012 01:51 AM, Shaohua Li wrote: > > >I have no strong point against the global state method. But I'd agree making > >the > >heuristic simple is preferred currently. I'm happy about the patch if the > >'+1' > >is removed.

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-23 Thread Rik van Riel
On 10/23/2012 01:51 AM, Shaohua Li wrote: I have no strong point against the global state method. But I'd agree making the heuristic simple is preferred currently. I'm happy about the patch if the '+1' is removed. Without the +1, how will you figure out when to re-enable readahead? -- All

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-23 Thread Rik van Riel
On 10/23/2012 01:51 AM, Shaohua Li wrote: I have no strong point against the global state method. But I'd agree making the heuristic simple is preferred currently. I'm happy about the patch if the '+1' is removed. Without the +1, how will you figure out when to re-enable readahead? -- All

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-23 Thread Shaohua Li
On Tue, Oct 23, 2012 at 09:41:00AM -0400, Rik van Riel wrote: On 10/23/2012 01:51 AM, Shaohua Li wrote: I have no strong point against the global state method. But I'd agree making the heuristic simple is preferred currently. I'm happy about the patch if the '+1' is removed. Without

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-22 Thread Shaohua Li
On Mon, Oct 22, 2012 at 10:16:40PM -0700, Hugh Dickins wrote: > On Mon, 22 Oct 2012, Shaohua Li wrote: > > On Tue, Oct 16, 2012 at 08:50:49AM +0800, Shaohua Li wrote: > > > On Mon, Oct 08, 2012 at 03:09:58PM -0700, Hugh Dickins wrote: > > > > On Thu, 4 Oct 2012, Konstantin Khlebnikov wrote: > > >

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-22 Thread Hugh Dickins
On Mon, 22 Oct 2012, Shaohua Li wrote: > On Tue, Oct 16, 2012 at 08:50:49AM +0800, Shaohua Li wrote: > > On Mon, Oct 08, 2012 at 03:09:58PM -0700, Hugh Dickins wrote: > > > On Thu, 4 Oct 2012, Konstantin Khlebnikov wrote: > > > > > > > Here results of my test. Workload isn't very realistic, but

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-22 Thread Shaohua Li
On Tue, Oct 16, 2012 at 08:50:49AM +0800, Shaohua Li wrote: > On Mon, Oct 08, 2012 at 03:09:58PM -0700, Hugh Dickins wrote: > > On Thu, 4 Oct 2012, Konstantin Khlebnikov wrote: > > > > > Here results of my test. Workload isn't very realistic, but at least it > > > threaded: compiling linux-3.6

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-22 Thread Shaohua Li
On Tue, Oct 16, 2012 at 08:50:49AM +0800, Shaohua Li wrote: On Mon, Oct 08, 2012 at 03:09:58PM -0700, Hugh Dickins wrote: On Thu, 4 Oct 2012, Konstantin Khlebnikov wrote: Here results of my test. Workload isn't very realistic, but at least it threaded: compiling linux-3.6 with

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-22 Thread Hugh Dickins
On Mon, 22 Oct 2012, Shaohua Li wrote: On Tue, Oct 16, 2012 at 08:50:49AM +0800, Shaohua Li wrote: On Mon, Oct 08, 2012 at 03:09:58PM -0700, Hugh Dickins wrote: On Thu, 4 Oct 2012, Konstantin Khlebnikov wrote: Here results of my test. Workload isn't very realistic, but at least it

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-22 Thread Shaohua Li
On Mon, Oct 22, 2012 at 10:16:40PM -0700, Hugh Dickins wrote: On Mon, 22 Oct 2012, Shaohua Li wrote: On Tue, Oct 16, 2012 at 08:50:49AM +0800, Shaohua Li wrote: On Mon, Oct 08, 2012 at 03:09:58PM -0700, Hugh Dickins wrote: On Thu, 4 Oct 2012, Konstantin Khlebnikov wrote: Here

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-15 Thread Shaohua Li
On Mon, Oct 08, 2012 at 03:09:58PM -0700, Hugh Dickins wrote: > On Thu, 4 Oct 2012, Konstantin Khlebnikov wrote: > > > Here results of my test. Workload isn't very realistic, but at least it > > threaded: compiling linux-3.6 with defconfig in 16 threads on tmpfs, > > 512mb ram, dualcore cpu,

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-15 Thread Shaohua Li
On Mon, Oct 08, 2012 at 03:09:58PM -0700, Hugh Dickins wrote: On Thu, 4 Oct 2012, Konstantin Khlebnikov wrote: Here results of my test. Workload isn't very realistic, but at least it threaded: compiling linux-3.6 with defconfig in 16 threads on tmpfs, 512mb ram, dualcore cpu, ordinary

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-09 Thread Konstantin Khlebnikov
Hugh Dickins wrote: On Thu, 4 Oct 2012, Konstantin Khlebnikov wrote: Here results of my test. Workload isn't very realistic, but at least it threaded: compiling linux-3.6 with defconfig in 16 threads on tmpfs, 512mb ram, dualcore cpu, ordinary hard disk. (test script in attachment) average

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-09 Thread Konstantin Khlebnikov
Hugh Dickins wrote: On Thu, 4 Oct 2012, Konstantin Khlebnikov wrote: Here results of my test. Workload isn't very realistic, but at least it threaded: compiling linux-3.6 with defconfig in 16 threads on tmpfs, 512mb ram, dualcore cpu, ordinary hard disk. (test script in attachment) average

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-08 Thread Andrew Morton
On Mon, 8 Oct 2012 15:09:58 -0700 (PDT) Hugh Dickins wrote: > All I want to do right now, is suggest to Andrew that he hold Shaohua's > patch back from 3.7 for the moment: I'll send a response to Sep 7th's > mm-commits mail to suggest that - but no great disaster if he ignores me. Just in the

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-08 Thread Hugh Dickins
On Thu, 4 Oct 2012, Konstantin Khlebnikov wrote: > Here results of my test. Workload isn't very realistic, but at least it > threaded: compiling linux-3.6 with defconfig in 16 threads on tmpfs, > 512mb ram, dualcore cpu, ordinary hard disk. (test script in attachment) > > average results for ten

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-08 Thread Hugh Dickins
On Thu, 4 Oct 2012, Konstantin Khlebnikov wrote: Here results of my test. Workload isn't very realistic, but at least it threaded: compiling linux-3.6 with defconfig in 16 threads on tmpfs, 512mb ram, dualcore cpu, ordinary hard disk. (test script in attachment) average results for ten

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-08 Thread Andrew Morton
On Mon, 8 Oct 2012 15:09:58 -0700 (PDT) Hugh Dickins hu...@google.com wrote: All I want to do right now, is suggest to Andrew that he hold Shaohua's patch back from 3.7 for the moment: I'll send a response to Sep 7th's mm-commits mail to suggest that - but no great disaster if he ignores me.

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-04 Thread Konstantin Khlebnikov
Here results of my test. Workload isn't very realistic, but at least it threaded: compiling linux-3.6 with defconfig in 16 threads on tmpfs, 512mb ram, dualcore cpu, ordinary hard disk. (test script in attachment) average results for ten runs: RA=3RA=0RA=1RA=2

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-04 Thread Konstantin Khlebnikov
Here results of my test. Workload isn't very realistic, but at least it threaded: compiling linux-3.6 with defconfig in 16 threads on tmpfs, 512mb ram, dualcore cpu, ordinary hard disk. (test script in attachment) average results for ten runs: RA=3RA=0RA=1RA=2

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-03 Thread Hugh Dickins
On Tue, 2 Oct 2012, Konstantin Khlebnikov wrote: > Hugh Dickins wrote: > > > > If I boot with mem=900M (and 1G swap: either on hard disk sda, or > > on Vertex II SSD sdb), and mmap anonymous 1000M (either MAP_PRIVATE, > > or MAP_SHARED for a shmem object), and either cycle sequentially round > >

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-03 Thread Hugh Dickins
On Tue, 2 Oct 2012, Konstantin Khlebnikov wrote: Hugh Dickins wrote: If I boot with mem=900M (and 1G swap: either on hard disk sda, or on Vertex II SSD sdb), and mmap anonymous 1000M (either MAP_PRIVATE, or MAP_SHARED for a shmem object), and either cycle sequentially round that making

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-02 Thread Konstantin Khlebnikov
Great job! I'm glad to see that you like my proof of concept patch. I though that +/-10 logic can switch between border states smoothly. But I have no strong experience in such kind of fuzzy-logic stuff, so it's no surprise that my code fails in some cases. (one note below about numbers) Hugh

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-02 Thread Konstantin Khlebnikov
Great job! I'm glad to see that you like my proof of concept patch. I though that +/-10 logic can switch between border states smoothly. But I have no strong experience in such kind of fuzzy-logic stuff, so it's no surprise that my code fails in some cases. (one note below about numbers) Hugh

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-01 Thread Hugh Dickins
Shaohua, Konstantin, Sorry that it takes me so long to to reply on these swapin readahead bounding threads, but I had to try some things out before jumping in, and only found time to experiment last week. On Thu, 6 Sep 2012, Konstantin Khlebnikov wrote: > This patch adds simple tracker for

Re: [PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-10-01 Thread Hugh Dickins
Shaohua, Konstantin, Sorry that it takes me so long to to reply on these swapin readahead bounding threads, but I had to try some things out before jumping in, and only found time to experiment last week. On Thu, 6 Sep 2012, Konstantin Khlebnikov wrote: This patch adds simple tracker for swapin

[PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-09-06 Thread Konstantin Khlebnikov
This patch adds simple tracker for swapin readahread effectiveness, and tunes readahead cluster depending on it. It manage internal state [0..1024] and scales readahead order between 0 and value from sysctl vm.page-cluster (3 by default). Swapout and readahead misses decreases state, swapin and ra

[PATCH RFC] mm/swap: automatic tuning for swapin readahead

2012-09-06 Thread Konstantin Khlebnikov
This patch adds simple tracker for swapin readahread effectiveness, and tunes readahead cluster depending on it. It manage internal state [0..1024] and scales readahead order between 0 and value from sysctl vm.page-cluster (3 by default). Swapout and readahead misses decreases state, swapin and ra