Re: [RFC][PATCH] split file and anonymous page queues #3

2007-03-21 Thread Nikita Danilov
Rik van Riel writes: > Nikita Danilov wrote: > > > Generally speaking, multi-queue replacement mechanisms were tried in the > > past, and they all suffer from the common drawback: once scanning rate > > is different for different queues, so is the notion of "hotness", > > measured by

Re: [RFC][PATCH] split file and anonymous page queues #3

2007-03-21 Thread Rik van Riel
Matt Mackall wrote: On Tue, Mar 20, 2007 at 06:08:10PM -0400, Rik van Riel wrote: - "Active: %8lu kB\n" - "Inactive: %8lu kB\n" ... + "Active(anon): %8lu kB\n" + "Inactive(anon): %8lu kB\n" + "Active(file):

Re: [RFC][PATCH] split file and anonymous page queues #3

2007-03-21 Thread Rik van Riel
Chuck Ebbert wrote: I think you're going to have to use refault rates. AIX 3.5 had to add that. Something like: if refault_rate(anonymous/mmap) > refault_rate(pagecache) drop a pagecache page else drop either How about the opposite? If the page cache refault rate is way higher than

Re: [RFC][PATCH] split file and anonymous page queues #3

2007-03-21 Thread Chuck Ebbert
Rik van Riel wrote: > Nikita Danilov wrote: > >> Probably I am missing something, but I don't see how that can help. For >> example, suppose (for simplicity) that we have swappiness of 100%, and >> that fraction of referenced anon pages gets slightly less than of file >> pages. get_scan_ratio()

Re: [RFC][PATCH] split file and anonymous page queues #3

2007-03-21 Thread Rik van Riel
Nikita Danilov wrote: Generally speaking, multi-queue replacement mechanisms were tried in the past, and they all suffer from the common drawback: once scanning rate is different for different queues, so is the notion of "hotness", measured by scanner. As a result multi-queue scanner fails to

Re: [RFC][PATCH] split file and anonymous page queues #3

2007-03-21 Thread Nikita Danilov
Rik van Riel writes: > Rik van Riel wrote: > > Nikita Danilov wrote: > > > >> Probably I am missing something, but I don't see how that can help. For > >> example, suppose (for simplicity) that we have swappiness of 100%, and > >> that fraction of referenced anon pages gets slightly less

Re: [RFC][PATCH] split file and anonymous page queues #3

2007-03-21 Thread Rik van Riel
Rik van Riel wrote: Nikita Danilov wrote: Probably I am missing something, but I don't see how that can help. For example, suppose (for simplicity) that we have swappiness of 100%, and that fraction of referenced anon pages gets slightly less than of file pages. get_scan_ratio() increases

Re: [RFC][PATCH] split file and anonymous page queues #3

2007-03-21 Thread Rik van Riel
Nikita Danilov wrote: Probably I am missing something, but I don't see how that can help. For example, suppose (for simplicity) that we have swappiness of 100%, and that fraction of referenced anon pages gets slightly less than of file pages. get_scan_ratio() increases anon_percent, and

Re: [RFC][PATCH] split file and anonymous page queues #3

2007-03-21 Thread Nikita Danilov
Rik van Riel writes: > Nikita Danilov wrote: > > Rik van Riel writes: > > > [ OK, I suck. I edited yesterday's email with the new info, but forgot > > >to change the attachment to today's patch. Here is today's patch. ] > > > > > > Split the anonymous and file backed pages out

Re: [RFC][PATCH] split file and anonymous page queues #3

2007-03-21 Thread Rik van Riel
Nikita Danilov wrote: Rik van Riel writes: > [ OK, I suck. I edited yesterday's email with the new info, but forgot >to change the attachment to today's patch. Here is today's patch. ] > > Split the anonymous and file backed pages out onto their own pageout > queues. This we do not

Re: [RFC][PATCH] split file and anonymous page queues #3

2007-03-21 Thread Nikita Danilov
Rik van Riel writes: > [ OK, I suck. I edited yesterday's email with the new info, but forgot >to change the attachment to today's patch. Here is today's patch. ] > > Split the anonymous and file backed pages out onto their own pageout > queues. This we do not unnecessarily churn

Re: [RFC][PATCH] split file and anonymous page queues #3

2007-03-21 Thread Nikita Danilov
Rik van Riel writes: [ OK, I suck. I edited yesterday's email with the new info, but forgot to change the attachment to today's patch. Here is today's patch. ] Split the anonymous and file backed pages out onto their own pageout queues. This we do not unnecessarily churn through

Re: [RFC][PATCH] split file and anonymous page queues #3

2007-03-21 Thread Rik van Riel
Nikita Danilov wrote: Rik van Riel writes: [ OK, I suck. I edited yesterday's email with the new info, but forgot to change the attachment to today's patch. Here is today's patch. ] Split the anonymous and file backed pages out onto their own pageout queues. This we do not

Re: [RFC][PATCH] split file and anonymous page queues #3

2007-03-21 Thread Nikita Danilov
Rik van Riel writes: Nikita Danilov wrote: Rik van Riel writes: [ OK, I suck. I edited yesterday's email with the new info, but forgot to change the attachment to today's patch. Here is today's patch. ] Split the anonymous and file backed pages out onto their own

Re: [RFC][PATCH] split file and anonymous page queues #3

2007-03-21 Thread Rik van Riel
Nikita Danilov wrote: Probably I am missing something, but I don't see how that can help. For example, suppose (for simplicity) that we have swappiness of 100%, and that fraction of referenced anon pages gets slightly less than of file pages. get_scan_ratio() increases anon_percent, and

Re: [RFC][PATCH] split file and anonymous page queues #3

2007-03-21 Thread Rik van Riel
Rik van Riel wrote: Nikita Danilov wrote: Probably I am missing something, but I don't see how that can help. For example, suppose (for simplicity) that we have swappiness of 100%, and that fraction of referenced anon pages gets slightly less than of file pages. get_scan_ratio() increases

Re: [RFC][PATCH] split file and anonymous page queues #3

2007-03-21 Thread Nikita Danilov
Rik van Riel writes: Rik van Riel wrote: Nikita Danilov wrote: Probably I am missing something, but I don't see how that can help. For example, suppose (for simplicity) that we have swappiness of 100%, and that fraction of referenced anon pages gets slightly less than of file

Re: [RFC][PATCH] split file and anonymous page queues #3

2007-03-21 Thread Rik van Riel
Nikita Danilov wrote: Generally speaking, multi-queue replacement mechanisms were tried in the past, and they all suffer from the common drawback: once scanning rate is different for different queues, so is the notion of hotness, measured by scanner. As a result multi-queue scanner fails to

Re: [RFC][PATCH] split file and anonymous page queues #3

2007-03-21 Thread Chuck Ebbert
Rik van Riel wrote: Nikita Danilov wrote: Probably I am missing something, but I don't see how that can help. For example, suppose (for simplicity) that we have swappiness of 100%, and that fraction of referenced anon pages gets slightly less than of file pages. get_scan_ratio() increases

Re: [RFC][PATCH] split file and anonymous page queues #3

2007-03-21 Thread Rik van Riel
Chuck Ebbert wrote: I think you're going to have to use refault rates. AIX 3.5 had to add that. Something like: if refault_rate(anonymous/mmap) refault_rate(pagecache) drop a pagecache page else drop either How about the opposite? If the page cache refault rate is way higher than the

Re: [RFC][PATCH] split file and anonymous page queues #3

2007-03-21 Thread Rik van Riel
Matt Mackall wrote: On Tue, Mar 20, 2007 at 06:08:10PM -0400, Rik van Riel wrote: - Active: %8lu kB\n - Inactive: %8lu kB\n ... + Active(anon): %8lu kB\n + Inactive(anon): %8lu kB\n + Active(file): %8lu kB\n +

Re: [RFC][PATCH] split file and anonymous page queues #3

2007-03-21 Thread Nikita Danilov
Rik van Riel writes: Nikita Danilov wrote: Generally speaking, multi-queue replacement mechanisms were tried in the past, and they all suffer from the common drawback: once scanning rate is different for different queues, so is the notion of hotness, measured by scanner. As a

Re: [RFC][PATCH] split file and anonymous page queues #3

2007-03-20 Thread Matt Mackall
On Tue, Mar 20, 2007 at 06:08:10PM -0400, Rik van Riel wrote: > - "Active: %8lu kB\n" > - "Inactive: %8lu kB\n" ... > + "Active(anon): %8lu kB\n" > + "Inactive(anon): %8lu kB\n" > + "Active(file): %8lu kB\n" > +

[RFC][PATCH] split file and anonymous page queues #3

2007-03-20 Thread Rik van Riel
[ OK, I suck. I edited yesterday's email with the new info, but forgot to change the attachment to today's patch. Here is today's patch. ] Split the anonymous and file backed pages out onto their own pageout queues. This we do not unnecessarily churn through lots of anonymous pages when we

[RFC][PATCH] split file and anonymous page queues #3

2007-03-20 Thread Rik van Riel
Split the anonymous and file backed pages out onto their own pageout queues. This we do not unnecessarily churn through lots of anonymous pages when we do not want to swap them out anyway. This should (with additional tuning) be a great step forward in scalability, allowing Linux to run well on

[RFC][PATCH] split file and anonymous page queues #3

2007-03-20 Thread Rik van Riel
Split the anonymous and file backed pages out onto their own pageout queues. This we do not unnecessarily churn through lots of anonymous pages when we do not want to swap them out anyway. This should (with additional tuning) be a great step forward in scalability, allowing Linux to run well on

[RFC][PATCH] split file and anonymous page queues #3

2007-03-20 Thread Rik van Riel
[ OK, I suck. I edited yesterday's email with the new info, but forgot to change the attachment to today's patch. Here is today's patch. ] Split the anonymous and file backed pages out onto their own pageout queues. This we do not unnecessarily churn through lots of anonymous pages when we

Re: [RFC][PATCH] split file and anonymous page queues #3

2007-03-20 Thread Matt Mackall
On Tue, Mar 20, 2007 at 06:08:10PM -0400, Rik van Riel wrote: - Active: %8lu kB\n - Inactive: %8lu kB\n ... + Active(anon): %8lu kB\n + Inactive(anon): %8lu kB\n + Active(file): %8lu kB\n + Inactive(file):