Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-14 Thread Andrew Morton
Andrea Arcangeli <[EMAIL PROTECTED]> wrote: > > you're right something might be different now > that we don't follow a swapout virtual address space order anymore There's a patch in -mm which attempts to do so, and afair, succeeds. However the performance seems to be crappy. Its main benefit

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-14 Thread Andrea Arcangeli
On Mon, Feb 14, 2005 at 06:36:43PM +, Hugh Dickins wrote: > On Mon, 14 Feb 2005, Andrea Arcangeli wrote: > > > By the way, while we're talking of remove_exclusive_swap_page: > > > a more functional issue I sometimes wonder about, why don't we > > > remove_exclusive_swap_page on write fault?

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-14 Thread Hugh Dickins
On Mon, 14 Feb 2005, Andrea Arcangeli wrote: > > By the way, while we're talking of remove_exclusive_swap_page: > > a more functional issue I sometimes wonder about, why don't we > > remove_exclusive_swap_page on write fault? Keeping the swap slot > > is valuable if read fault, but once the page

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-14 Thread Hugh Dickins
On Mon, 14 Feb 2005, Andrea Arcangeli wrote: By the way, while we're talking of remove_exclusive_swap_page: a more functional issue I sometimes wonder about, why don't we remove_exclusive_swap_page on write fault? Keeping the swap slot is valuable if read fault, but once the page is

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-14 Thread Andrea Arcangeli
On Mon, Feb 14, 2005 at 06:36:43PM +, Hugh Dickins wrote: On Mon, 14 Feb 2005, Andrea Arcangeli wrote: By the way, while we're talking of remove_exclusive_swap_page: a more functional issue I sometimes wonder about, why don't we remove_exclusive_swap_page on write fault? Keeping the

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-14 Thread Andrew Morton
Andrea Arcangeli [EMAIL PROTECTED] wrote: you're right something might be different now that we don't follow a swapout virtual address space order anymore There's a patch in -mm which attempts to do so, and afair, succeeds. However the performance seems to be crappy. Its main benefit at

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-11 Thread Hugh Dickins
On Fri, 11 Feb 2005, Andrea Arcangeli wrote: > > Ok, I'm quite convinced it's correct now. The only thing that can make > mapcount go up without the lock on the page without userspace > intervention (and userspace intervention would make it an undefined > behaviour like in my example with fork),

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-11 Thread Andrea Arcangeli
On Fri, Feb 11, 2005 at 07:23:09AM +, Hugh Dickins wrote: > And it's fine for the behaviour to be somewhat undefined in this > peculiar case: the important thing is just that the page must not > be freed and reused while I/O occurs, hence get_user_page raising > the page_count - which I'm

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-11 Thread Andrea Arcangeli
On Fri, Feb 11, 2005 at 07:23:09AM +, Hugh Dickins wrote: And it's fine for the behaviour to be somewhat undefined in this peculiar case: the important thing is just that the page must not be freed and reused while I/O occurs, hence get_user_page raising the page_count - which I'm _not_

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-11 Thread Hugh Dickins
On Fri, 11 Feb 2005, Andrea Arcangeli wrote: Ok, I'm quite convinced it's correct now. The only thing that can make mapcount go up without the lock on the page without userspace intervention (and userspace intervention would make it an undefined behaviour like in my example with fork), was

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-10 Thread Hugh Dickins
On Thu, 10 Feb 2005, Andrea Arcangeli wrote: > On Thu, Feb 10, 2005 at 08:19:47PM +, Hugh Dickins wrote: > > > > get_user_pages broke COW in advance of the I/O. The reason for > > subsequent COW if the page gets unmapped is precisely because > > can_share_swap_page used page_count to judge

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-10 Thread Andrea Arcangeli
On Thu, Feb 10, 2005 at 08:19:47PM +, Hugh Dickins wrote: > On Thu, 10 Feb 2005, Andrea Arcangeli wrote: > > > > The reason pinned pages cannot be unmapped is that if they're being > > unmapped while a rawio read is in progress, a cow on some shared > > swapcache under I/O could happen. > >

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-10 Thread Hugh Dickins
On Thu, 10 Feb 2005, Andrea Arcangeli wrote: > > The reason pinned pages cannot be unmapped is that if they're being > unmapped while a rawio read is in progress, a cow on some shared > swapcache under I/O could happen. > > If a try_to_unmap + COW over a shared swapcache happens during a rawio >

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-10 Thread Andrea Arcangeli
On Thu, Feb 10, 2005 at 11:16:44AM -0800, Dave Hansen wrote: > We actually do that, in addition to the more active methods of capturing > the memory that we're about to remove. This sounds the best way to go. btw, is this a different codebase from the one that Toshihiro is testing? > You're

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-10 Thread Dave Hansen
On Thu, 2005-02-10 at 20:05 +0100, Andrea Arcangeli wrote: > Overall I see nothing wrong in preventing memory removal while rawio is > in flight. rawio cannot be in flight forever (ptrace and core dump as > well should complete eventually). Why can't you simply drop pages from > the freelist one

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-10 Thread Andrea Arcangeli
On Tue, Feb 08, 2005 at 04:26:26PM +, Hugh Dickins wrote: > Seems it was okay after all, I got confused by an unrelated issue. > Here's what I had in mind, what do you think? Does it indeed help > with your problem? I'm copying Andrea because it was he who devised > that fix to the

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-10 Thread IWAMOTO Toshihiro
At Tue, 8 Feb 2005 16:26:26 + (GMT), Hugh Dickins wrote: > > On Mon, 7 Feb 2005, Hugh Dickins wrote: > > On Thu, 3 Feb 2005, IWAMOTO Toshihiro wrote: > > > The current implementation of memory hotremoval relies on that pages > > > can be unmapped from process spaces. After successful

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-10 Thread IWAMOTO Toshihiro
At Tue, 8 Feb 2005 16:26:26 + (GMT), Hugh Dickins wrote: On Mon, 7 Feb 2005, Hugh Dickins wrote: On Thu, 3 Feb 2005, IWAMOTO Toshihiro wrote: The current implementation of memory hotremoval relies on that pages can be unmapped from process spaces. After successful unmapping,

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-10 Thread Andrea Arcangeli
On Tue, Feb 08, 2005 at 04:26:26PM +, Hugh Dickins wrote: Seems it was okay after all, I got confused by an unrelated issue. Here's what I had in mind, what do you think? Does it indeed help with your problem? I'm copying Andrea because it was he who devised that fix to the

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-10 Thread Dave Hansen
On Thu, 2005-02-10 at 20:05 +0100, Andrea Arcangeli wrote: Overall I see nothing wrong in preventing memory removal while rawio is in flight. rawio cannot be in flight forever (ptrace and core dump as well should complete eventually). Why can't you simply drop pages from the freelist one by

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-10 Thread Andrea Arcangeli
On Thu, Feb 10, 2005 at 11:16:44AM -0800, Dave Hansen wrote: We actually do that, in addition to the more active methods of capturing the memory that we're about to remove. This sounds the best way to go. btw, is this a different codebase from the one that Toshihiro is testing? You're right,

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-10 Thread Hugh Dickins
On Thu, 10 Feb 2005, Andrea Arcangeli wrote: The reason pinned pages cannot be unmapped is that if they're being unmapped while a rawio read is in progress, a cow on some shared swapcache under I/O could happen. If a try_to_unmap + COW over a shared swapcache happens during a rawio read,

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-10 Thread Andrea Arcangeli
On Thu, Feb 10, 2005 at 08:19:47PM +, Hugh Dickins wrote: On Thu, 10 Feb 2005, Andrea Arcangeli wrote: The reason pinned pages cannot be unmapped is that if they're being unmapped while a rawio read is in progress, a cow on some shared swapcache under I/O could happen. If a

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-10 Thread Hugh Dickins
On Thu, 10 Feb 2005, Andrea Arcangeli wrote: On Thu, Feb 10, 2005 at 08:19:47PM +, Hugh Dickins wrote: get_user_pages broke COW in advance of the I/O. The reason for subsequent COW if the page gets unmapped is precisely because can_share_swap_page used page_count to judge

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-09 Thread IWAMOTO Toshihiro
At Mon, 7 Feb 2005 21:24:59 + (GMT), Hugh Dickins wrote: > > On Thu, 3 Feb 2005, IWAMOTO Toshihiro wrote: > > The current implementation of memory hotremoval relies on that pages > > can be unmapped from process spaces. After successful unmapping, > > subsequent accesses to the pages are

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-09 Thread IWAMOTO Toshihiro
At Mon, 7 Feb 2005 21:24:59 + (GMT), Hugh Dickins wrote: On Thu, 3 Feb 2005, IWAMOTO Toshihiro wrote: The current implementation of memory hotremoval relies on that pages can be unmapped from process spaces. After successful unmapping, subsequent accesses to the pages are blocked and

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-08 Thread Hugh Dickins
On Mon, 7 Feb 2005, Hugh Dickins wrote: > On Thu, 3 Feb 2005, IWAMOTO Toshihiro wrote: > > The current implementation of memory hotremoval relies on that pages > > can be unmapped from process spaces. After successful unmapping, > > subsequent accesses to the pages are blocked and don't interfere

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-08 Thread Hugh Dickins
On Mon, 7 Feb 2005, Hugh Dickins wrote: On Thu, 3 Feb 2005, IWAMOTO Toshihiro wrote: The current implementation of memory hotremoval relies on that pages can be unmapped from process spaces. After successful unmapping, subsequent accesses to the pages are blocked and don't interfere the

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-07 Thread Hugh Dickins
On Thu, 3 Feb 2005, IWAMOTO Toshihiro wrote: > The current implementation of memory hotremoval relies on that pages > can be unmapped from process spaces. After successful unmapping, > subsequent accesses to the pages are blocked and don't interfere > the hotremoval operation. > > However, this

Re: [RFC] Changing COW detection to be memory hotplug friendly

2005-02-07 Thread Hugh Dickins
On Thu, 3 Feb 2005, IWAMOTO Toshihiro wrote: The current implementation of memory hotremoval relies on that pages can be unmapped from process spaces. After successful unmapping, subsequent accesses to the pages are blocked and don't interfere the hotremoval operation. However, this code

[RFC] Changing COW detection to be memory hotplug friendly

2005-02-02 Thread IWAMOTO Toshihiro
The current implementation of memory hotremoval relies on that pages can be unmapped from process spaces. After successful unmapping, subsequent accesses to the pages are blocked and don't interfere the hotremoval operation. However, this code if (PageSwapCache(page) &&

[RFC] Changing COW detection to be memory hotplug friendly

2005-02-02 Thread IWAMOTO Toshihiro
The current implementation of memory hotremoval relies on that pages can be unmapped from process spaces. After successful unmapping, subsequent accesses to the pages are blocked and don't interfere the hotremoval operation. However, this code if (PageSwapCache(page)