Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-07-04 Thread Gu Zheng
On 07/04/2013 07:41 PM, Benjamin LaHaise wrote: > On Thu, Jul 04, 2013 at 02:51:18PM +0800, Gu Zheng wrote: >> Hi Ben, >> When I test your patch on kernel 3.10, the kernel panic when aio job >> complete or exit, exactly in aio_free_ring(), the following is a part of >> dmesg. > > What is

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-07-04 Thread Benjamin LaHaise
On Thu, Jul 04, 2013 at 02:51:18PM +0800, Gu Zheng wrote: > Hi Ben, > When I test your patch on kernel 3.10, the kernel panic when aio job > complete or exit, exactly in aio_free_ring(), the following is a part of > dmesg. What is your test case? -ben -- To unsubscribe

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-07-04 Thread Gu Zheng
On 07/03/2013 02:00 AM, Benjamin LaHaise wrote: > On Mon, Jul 01, 2013 at 03:23:39PM +0800, Gu Zheng wrote: >> Hi Ben, >> Are you still working on this patch? >> As you know, using the current anon inode will lead to more than one >> instance of >> aio can not work. Have you found a way to fix

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-07-04 Thread Gu Zheng
On 07/03/2013 02:00 AM, Benjamin LaHaise wrote: On Mon, Jul 01, 2013 at 03:23:39PM +0800, Gu Zheng wrote: Hi Ben, Are you still working on this patch? As you know, using the current anon inode will lead to more than one instance of aio can not work. Have you found a way to fix this issue?

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-07-04 Thread Benjamin LaHaise
On Thu, Jul 04, 2013 at 02:51:18PM +0800, Gu Zheng wrote: Hi Ben, When I test your patch on kernel 3.10, the kernel panic when aio job complete or exit, exactly in aio_free_ring(), the following is a part of dmesg. What is your test case? -ben -- To unsubscribe from

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-07-04 Thread Gu Zheng
On 07/04/2013 07:41 PM, Benjamin LaHaise wrote: On Thu, Jul 04, 2013 at 02:51:18PM +0800, Gu Zheng wrote: Hi Ben, When I test your patch on kernel 3.10, the kernel panic when aio job complete or exit, exactly in aio_free_ring(), the following is a part of dmesg. What is your test

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-07-02 Thread Gu Zheng
On 07/03/2013 02:00 AM, Benjamin LaHaise wrote: > On Mon, Jul 01, 2013 at 03:23:39PM +0800, Gu Zheng wrote: >> Hi Ben, >> Are you still working on this patch? >> As you know, using the current anon inode will lead to more than one >> instance of >> aio can not work. Have you found a way to fix

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-07-02 Thread Benjamin LaHaise
On Mon, Jul 01, 2013 at 03:23:39PM +0800, Gu Zheng wrote: > Hi Ben, > Are you still working on this patch? > As you know, using the current anon inode will lead to more than one instance > of > aio can not work. Have you found a way to fix this issue? Or can we use some > other ones to replace

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-07-02 Thread Benjamin LaHaise
On Mon, Jul 01, 2013 at 03:23:39PM +0800, Gu Zheng wrote: Hi Ben, Are you still working on this patch? As you know, using the current anon inode will lead to more than one instance of aio can not work. Have you found a way to fix this issue? Or can we use some other ones to replace the anon

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-07-02 Thread Gu Zheng
On 07/03/2013 02:00 AM, Benjamin LaHaise wrote: On Mon, Jul 01, 2013 at 03:23:39PM +0800, Gu Zheng wrote: Hi Ben, Are you still working on this patch? As you know, using the current anon inode will lead to more than one instance of aio can not work. Have you found a way to fix this issue?

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-07-01 Thread Gu Zheng
On 06/11/2013 10:45 PM, Benjamin LaHaise wrote: > Hi Tang, > > On Tue, Jun 11, 2013 at 05:42:31PM +0800, Tang Chen wrote: >> Hi Benjamin, >> >> Are you still working on this problem ? >> >> Thanks. :) > > Below is a copy of the most recent version of this patch I have worked > on. This

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-07-01 Thread Gu Zheng
On 06/11/2013 10:45 PM, Benjamin LaHaise wrote: Hi Tang, On Tue, Jun 11, 2013 at 05:42:31PM +0800, Tang Chen wrote: Hi Benjamin, Are you still working on this problem ? Thanks. :) Below is a copy of the most recent version of this patch I have worked on. This version works and

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-06-28 Thread Gu Zheng
On 06/11/2013 10:45 PM, Benjamin LaHaise wrote: > Hi Tang, > > On Tue, Jun 11, 2013 at 05:42:31PM +0800, Tang Chen wrote: >> Hi Benjamin, >> >> Are you still working on this problem ? >> >> Thanks. :) > > Below is a copy of the most recent version of this patch I have worked > on. This

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-06-28 Thread Gu Zheng
On 06/11/2013 10:45 PM, Benjamin LaHaise wrote: Hi Tang, On Tue, Jun 11, 2013 at 05:42:31PM +0800, Tang Chen wrote: Hi Benjamin, Are you still working on this problem ? Thanks. :) Below is a copy of the most recent version of this patch I have worked on. This version works and

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-06-11 Thread Benjamin LaHaise
Hi Tang, On Tue, Jun 11, 2013 at 05:42:31PM +0800, Tang Chen wrote: > Hi Benjamin, > > Are you still working on this problem ? > > Thanks. :) Below is a copy of the most recent version of this patch I have worked on. This version works and stands up to my testing using move_pages() to force

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-06-11 Thread Tang Chen
Hi Benjamin, Are you still working on this problem ? Thanks. :) On 05/21/2013 10:27 AM, Benjamin LaHaise wrote: On Tue, May 21, 2013 at 10:07:52AM +0800, Tang Chen wrote: I'm not saying using two callbacks before and after migration is better. I don't want to use

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-06-11 Thread Tang Chen
Hi Benjamin, Are you still working on this problem ? Thanks. :) On 05/21/2013 10:27 AM, Benjamin LaHaise wrote: On Tue, May 21, 2013 at 10:07:52AM +0800, Tang Chen wrote: I'm not saying using two callbacks before and after migration is better. I don't want to use

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-06-11 Thread Benjamin LaHaise
Hi Tang, On Tue, Jun 11, 2013 at 05:42:31PM +0800, Tang Chen wrote: Hi Benjamin, Are you still working on this problem ? Thanks. :) Below is a copy of the most recent version of this patch I have worked on. This version works and stands up to my testing using move_pages() to force the

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-05-20 Thread Benjamin LaHaise
On Tue, May 21, 2013 at 10:07:52AM +0800, Tang Chen wrote: > I'm not saying using two callbacks before and after migration is better. > I don't want to use address_space_operations is because there is no such > member > for anonymous pages. That depends on the nature of the pinning. For

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-05-20 Thread Tang Chen
Hi Benjamin, Sorry for the late. Please see below. On 05/17/2013 10:37 PM, Benjamin LaHaise wrote: On Fri, May 17, 2013 at 11:28:52AM +0800, Tang Chen wrote: Hi Benjamin, Thank you very much for your idea. :) I have no objection to your idea, but seeing from your patch, this only works for

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-05-20 Thread Tang Chen
Hi Benjamin, Sorry for the late. Please see below. On 05/17/2013 10:37 PM, Benjamin LaHaise wrote: On Fri, May 17, 2013 at 11:28:52AM +0800, Tang Chen wrote: Hi Benjamin, Thank you very much for your idea. :) I have no objection to your idea, but seeing from your patch, this only works for

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-05-20 Thread Benjamin LaHaise
On Tue, May 21, 2013 at 10:07:52AM +0800, Tang Chen wrote: I'm not saying using two callbacks before and after migration is better. I don't want to use address_space_operations is because there is no such member for anonymous pages. That depends on the nature of the pinning. For the

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-05-17 Thread Benjamin LaHaise
On Fri, May 17, 2013 at 11:17:08AM -0700, Zach Brown wrote: > > I ended up working on this a bit today, and managed to cobble together > > something that somewhat works -- please see the patch below. > > Just some quick observations: > > > + ctx->ctx_file = anon_inode_getfile("[aio]",

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-05-17 Thread Zach Brown
> I ended up working on this a bit today, and managed to cobble together > something that somewhat works -- please see the patch below. Just some quick observations: > + ctx->ctx_file = anon_inode_getfile("[aio]", _ctx_fops, ctx, O_RDWR); > + if (IS_ERR(ctx->ctx_file)) { > +

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-05-17 Thread Benjamin LaHaise
On Fri, May 17, 2013 at 11:28:52AM +0800, Tang Chen wrote: > Hi Benjamin, > > Thank you very much for your idea. :) > > I have no objection to your idea, but seeing from your patch, this only > works for aio subsystem because you changed the way to allocate the aio > ring pages, with a file

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-05-17 Thread Benjamin LaHaise
On Fri, May 17, 2013 at 11:28:52AM +0800, Tang Chen wrote: Hi Benjamin, Thank you very much for your idea. :) I have no objection to your idea, but seeing from your patch, this only works for aio subsystem because you changed the way to allocate the aio ring pages, with a file mapping.

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-05-17 Thread Zach Brown
I ended up working on this a bit today, and managed to cobble together something that somewhat works -- please see the patch below. Just some quick observations: + ctx-ctx_file = anon_inode_getfile([aio], aio_ctx_fops, ctx, O_RDWR); + if (IS_ERR(ctx-ctx_file)) { +

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-05-17 Thread Benjamin LaHaise
On Fri, May 17, 2013 at 11:17:08AM -0700, Zach Brown wrote: I ended up working on this a bit today, and managed to cobble together something that somewhat works -- please see the patch below. Just some quick observations: + ctx-ctx_file = anon_inode_getfile([aio], aio_ctx_fops, ctx,

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-05-16 Thread Tang Chen
Hi Benjamin, Thank you very much for your idea. :) I have no objection to your idea, but seeing from your patch, this only works for aio subsystem because you changed the way to allocate the aio ring pages, with a file mapping. So far as I know, not only aio, but also other subsystems, such

[WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-05-16 Thread Benjamin LaHaise
On Thu, May 16, 2013 at 01:54:18PM +0800, Tang Chen wrote: ... > OK, I'll try to figure out a proper place to put the callbacks. > But I think we need to add something new to struct page. I'm just > not sure if it is OK. Maybe we can discuss more about it when I send > a RFC patch. ... I ended up

[WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-05-16 Thread Benjamin LaHaise
On Thu, May 16, 2013 at 01:54:18PM +0800, Tang Chen wrote: ... OK, I'll try to figure out a proper place to put the callbacks. But I think we need to add something new to struct page. I'm just not sure if it is OK. Maybe we can discuss more about it when I send a RFC patch. ... I ended up

Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

2013-05-16 Thread Tang Chen
Hi Benjamin, Thank you very much for your idea. :) I have no objection to your idea, but seeing from your patch, this only works for aio subsystem because you changed the way to allocate the aio ring pages, with a file mapping. So far as I know, not only aio, but also other subsystems, such

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-05-15 Thread Tang Chen
Hi Mel, On 05/15/2013 09:24 PM, Mel Gorman wrote: If it is to be an address space operations sturcture then you'll need a pseudo mapping structure for anonymous pages that are pinned by aio -- similar in principal to how swapper_space is used for managing PageSwapCache or how anon_vma

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-05-15 Thread Mel Gorman
On Tue, May 14, 2013 at 11:55:31AM +0800, Tang Chen wrote: > Hi Mel, > > On 05/13/2013 05:19 PM, Mel Gorman wrote: > >>For memory hot-remove case, the aio pages are pined in memory and making > >>the pages cannot be offlined, furthermore, the pages cannot be removed. > >> > >>IIUC, you mean

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-05-15 Thread Tang Chen
Hi Benjamin, Mel, On 05/15/2013 10:09 AM, Tang Chen wrote: Hi Benjamin, Mel, Please see below. On 05/14/2013 09:58 PM, Benjamin LaHaise wrote: On Tue, May 14, 2013 at 09:24:58AM +0800, Tang Chen wrote: Hi Mel, Benjamin, Jeff, On 05/13/2013 11:01 PM, Benjamin LaHaise wrote: On Mon, May 13,

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-05-15 Thread Tang Chen
Hi Mel, On 05/15/2013 09:24 PM, Mel Gorman wrote: If it is to be an address space operations sturcture then you'll need a pseudo mapping structure for anonymous pages that are pinned by aio -- similar in principal to how swapper_space is used for managing PageSwapCache or how anon_vma

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-05-15 Thread Tang Chen
Hi Benjamin, Mel, On 05/15/2013 10:09 AM, Tang Chen wrote: Hi Benjamin, Mel, Please see below. On 05/14/2013 09:58 PM, Benjamin LaHaise wrote: On Tue, May 14, 2013 at 09:24:58AM +0800, Tang Chen wrote: Hi Mel, Benjamin, Jeff, On 05/13/2013 11:01 PM, Benjamin LaHaise wrote: On Mon, May 13,

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-05-15 Thread Mel Gorman
On Tue, May 14, 2013 at 11:55:31AM +0800, Tang Chen wrote: Hi Mel, On 05/13/2013 05:19 PM, Mel Gorman wrote: For memory hot-remove case, the aio pages are pined in memory and making the pages cannot be offlined, furthermore, the pages cannot be removed. IIUC, you mean implement

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-05-14 Thread Tang Chen
Hi Benjamin, Mel, Please see below. On 05/14/2013 09:58 PM, Benjamin LaHaise wrote: On Tue, May 14, 2013 at 09:24:58AM +0800, Tang Chen wrote: Hi Mel, Benjamin, Jeff, On 05/13/2013 11:01 PM, Benjamin LaHaise wrote: On Mon, May 13, 2013 at 10:54:03AM -0400, Jeff Moyer wrote: How do you

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-05-14 Thread Benjamin LaHaise
On Tue, May 14, 2013 at 09:24:58AM +0800, Tang Chen wrote: > Hi Mel, Benjamin, Jeff, > > On 05/13/2013 11:01 PM, Benjamin LaHaise wrote: > >On Mon, May 13, 2013 at 10:54:03AM -0400, Jeff Moyer wrote: > >>How do you propose to move the ring pages? > > > >It's the same problem as doing a TLB

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-05-14 Thread Benjamin LaHaise
On Tue, May 14, 2013 at 09:24:58AM +0800, Tang Chen wrote: Hi Mel, Benjamin, Jeff, On 05/13/2013 11:01 PM, Benjamin LaHaise wrote: On Mon, May 13, 2013 at 10:54:03AM -0400, Jeff Moyer wrote: How do you propose to move the ring pages? It's the same problem as doing a TLB shootdown: flush

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-05-14 Thread Tang Chen
Hi Benjamin, Mel, Please see below. On 05/14/2013 09:58 PM, Benjamin LaHaise wrote: On Tue, May 14, 2013 at 09:24:58AM +0800, Tang Chen wrote: Hi Mel, Benjamin, Jeff, On 05/13/2013 11:01 PM, Benjamin LaHaise wrote: On Mon, May 13, 2013 at 10:54:03AM -0400, Jeff Moyer wrote: How do you

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-05-13 Thread Tang Chen
Hi Mel, On 05/13/2013 05:19 PM, Mel Gorman wrote: For memory hot-remove case, the aio pages are pined in memory and making the pages cannot be offlined, furthermore, the pages cannot be removed. IIUC, you mean implement migrate_unpin() and migrate_pin() callbacks in aio subsystem, and call

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-05-13 Thread Tang Chen
Hi Mel, Benjamin, Jeff, On 05/13/2013 11:01 PM, Benjamin LaHaise wrote: On Mon, May 13, 2013 at 10:54:03AM -0400, Jeff Moyer wrote: How do you propose to move the ring pages? It's the same problem as doing a TLB shootdown: flush the old pages from userspace's mapping, copy any existing data

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-05-13 Thread Benjamin LaHaise
On Mon, May 13, 2013 at 10:19:02AM +0100, Mel Gorman wrote: > On Mon, May 13, 2013 at 05:11:43PM +0800, Tang Chen wrote: ... > > If so, I'm wondering where should we put this callback pointers ? > > In struct page ? > > > > No, I would expect the callbacks to be part the address space operations

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-05-13 Thread Benjamin LaHaise
On Mon, May 13, 2013 at 10:54:03AM -0400, Jeff Moyer wrote: > How do you propose to move the ring pages? It's the same problem as doing a TLB shootdown: flush the old pages from userspace's mapping, copy any existing data to the new pages, then repopulate the page tables. It will likely

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-05-13 Thread Jeff Moyer
Benjamin LaHaise writes: > On Mon, May 13, 2013 at 10:19:02AM +0100, Mel Gorman wrote: >> On Mon, May 13, 2013 at 05:11:43PM +0800, Tang Chen wrote: > ... >> > If so, I'm wondering where should we put this callback pointers ? >> > In struct page ? >> > >> >> No, I would expect the callbacks to

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-05-13 Thread Mel Gorman
On Mon, May 13, 2013 at 05:11:43PM +0800, Tang Chen wrote: > Hi Mel, > > On 02/06/2013 05:56 PM, Mel Gorman wrote: > > > >There is the possibility that callbacks could be introduced for > >migrate_unpin() and migrate_pin() that takes a list of PFN pairs > >(old,new). The unpin callback should

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-05-13 Thread Tang Chen
Hi Mel, On 02/06/2013 05:56 PM, Mel Gorman wrote: There is the possibility that callbacks could be introduced for migrate_unpin() and migrate_pin() that takes a list of PFN pairs (old,new). The unpin callback should release the old PFNs and barrier against any operations until the

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-05-13 Thread Tang Chen
Hi Mel, On 02/06/2013 05:56 PM, Mel Gorman wrote: There is the possibility that callbacks could be introduced for migrate_unpin() and migrate_pin() that takes a list of PFN pairs (old,new). The unpin callback should release the old PFNs and barrier against any operations until the

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-05-13 Thread Mel Gorman
On Mon, May 13, 2013 at 05:11:43PM +0800, Tang Chen wrote: Hi Mel, On 02/06/2013 05:56 PM, Mel Gorman wrote: There is the possibility that callbacks could be introduced for migrate_unpin() and migrate_pin() that takes a list of PFN pairs (old,new). The unpin callback should release the

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-05-13 Thread Jeff Moyer
Benjamin LaHaise b...@kvack.org writes: On Mon, May 13, 2013 at 10:19:02AM +0100, Mel Gorman wrote: On Mon, May 13, 2013 at 05:11:43PM +0800, Tang Chen wrote: ... If so, I'm wondering where should we put this callback pointers ? In struct page ? No, I would expect the callbacks to be

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-05-13 Thread Benjamin LaHaise
On Mon, May 13, 2013 at 10:54:03AM -0400, Jeff Moyer wrote: How do you propose to move the ring pages? It's the same problem as doing a TLB shootdown: flush the old pages from userspace's mapping, copy any existing data to the new pages, then repopulate the page tables. It will likely require

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-05-13 Thread Benjamin LaHaise
On Mon, May 13, 2013 at 10:19:02AM +0100, Mel Gorman wrote: On Mon, May 13, 2013 at 05:11:43PM +0800, Tang Chen wrote: ... If so, I'm wondering where should we put this callback pointers ? In struct page ? No, I would expect the callbacks to be part the address space operations which

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-05-13 Thread Tang Chen
Hi Mel, Benjamin, Jeff, On 05/13/2013 11:01 PM, Benjamin LaHaise wrote: On Mon, May 13, 2013 at 10:54:03AM -0400, Jeff Moyer wrote: How do you propose to move the ring pages? It's the same problem as doing a TLB shootdown: flush the old pages from userspace's mapping, copy any existing data

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-05-13 Thread Tang Chen
Hi Mel, On 05/13/2013 05:19 PM, Mel Gorman wrote: For memory hot-remove case, the aio pages are pined in memory and making the pages cannot be offlined, furthermore, the pages cannot be removed. IIUC, you mean implement migrate_unpin() and migrate_pin() callbacks in aio subsystem, and call

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-02-20 Thread Lin Feng
Hi Wanpeng, On 02/20/2013 07:37 PM, Wanpeng Li wrote: >> + * This function first calls get_user_pages() to get the candidate pages, >> and >> >+ * then check to ensure all pages are from non movable zone. Otherwise >> >migrate > How about "Otherwise migrate candidate pages which have already

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-02-20 Thread Lin Feng
Hi Wanpeng, On 02/20/2013 07:37 PM, Wanpeng Li wrote: + * This function first calls get_user_pages() to get the candidate pages, and + * then check to ensure all pages are from non movable zone. Otherwise migrate How about Otherwise migrate candidate pages which have already been

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-02-07 Thread Minchan Kim
Hi Mel, On Wed, Feb 06, 2013 at 09:56:17AM +, Mel Gorman wrote: > On Wed, Feb 06, 2013 at 09:42:34AM +0900, Minchan Kim wrote: > > On Tue, Feb 05, 2013 at 12:01:37PM +, Mel Gorman wrote: > > > On Tue, Feb 05, 2013 at 05:21:52PM +0800, Lin Feng wrote: > > > > get_user_pages() always tries

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-02-07 Thread Minchan Kim
Hi Mel, On Wed, Feb 06, 2013 at 09:56:17AM +, Mel Gorman wrote: On Wed, Feb 06, 2013 at 09:42:34AM +0900, Minchan Kim wrote: On Tue, Feb 05, 2013 at 12:01:37PM +, Mel Gorman wrote: On Tue, Feb 05, 2013 at 05:21:52PM +0800, Lin Feng wrote: get_user_pages() always tries to

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-02-06 Thread Mel Gorman
On Wed, Feb 06, 2013 at 09:42:34AM +0900, Minchan Kim wrote: > On Tue, Feb 05, 2013 at 12:01:37PM +, Mel Gorman wrote: > > On Tue, Feb 05, 2013 at 05:21:52PM +0800, Lin Feng wrote: > > > get_user_pages() always tries to allocate pages from movable zone, which > > > is not > > > reliable to

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-02-06 Thread Mel Gorman
On Wed, Feb 06, 2013 at 09:42:34AM +0900, Minchan Kim wrote: On Tue, Feb 05, 2013 at 12:01:37PM +, Mel Gorman wrote: On Tue, Feb 05, 2013 at 05:21:52PM +0800, Lin Feng wrote: get_user_pages() always tries to allocate pages from movable zone, which is not reliable to memory

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-02-05 Thread Benjamin LaHaise
On Wed, Feb 06, 2013 at 09:42:34AM +0900, Minchan Kim wrote: > THP degradation by increasing MIGRATE_UNMOVABLE? > Lin said most of GUP pages release the page in short so is it really problem? > Even in embedded, we don't use THP yet but CMA and GUP call would be not too > often > but failing of

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-02-05 Thread Minchan Kim
On Tue, Feb 05, 2013 at 12:01:37PM +, Mel Gorman wrote: > On Tue, Feb 05, 2013 at 05:21:52PM +0800, Lin Feng wrote: > > get_user_pages() always tries to allocate pages from movable zone, which is > > not > > reliable to memory hotremove framework in some case. > > > > This patch introduces

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-02-05 Thread Mel Gorman
On Tue, Feb 05, 2013 at 05:21:52PM +0800, Lin Feng wrote: > get_user_pages() always tries to allocate pages from movable zone, which is > not > reliable to memory hotremove framework in some case. > > This patch introduces a new library function called > get_user_pages_non_movable() > to pin

[PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-02-05 Thread Lin Feng
get_user_pages() always tries to allocate pages from movable zone, which is not reliable to memory hotremove framework in some case. This patch introduces a new library function called get_user_pages_non_movable() to pin pages only from zone non-movable in memory. It's a wrapper of

[PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-02-05 Thread Lin Feng
get_user_pages() always tries to allocate pages from movable zone, which is not reliable to memory hotremove framework in some case. This patch introduces a new library function called get_user_pages_non_movable() to pin pages only from zone non-movable in memory. It's a wrapper of

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-02-05 Thread Mel Gorman
On Tue, Feb 05, 2013 at 05:21:52PM +0800, Lin Feng wrote: get_user_pages() always tries to allocate pages from movable zone, which is not reliable to memory hotremove framework in some case. This patch introduces a new library function called get_user_pages_non_movable() to pin pages

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-02-05 Thread Minchan Kim
On Tue, Feb 05, 2013 at 12:01:37PM +, Mel Gorman wrote: On Tue, Feb 05, 2013 at 05:21:52PM +0800, Lin Feng wrote: get_user_pages() always tries to allocate pages from movable zone, which is not reliable to memory hotremove framework in some case. This patch introduces a new

Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()

2013-02-05 Thread Benjamin LaHaise
On Wed, Feb 06, 2013 at 09:42:34AM +0900, Minchan Kim wrote: THP degradation by increasing MIGRATE_UNMOVABLE? Lin said most of GUP pages release the page in short so is it really problem? Even in embedded, we don't use THP yet but CMA and GUP call would be not too often but failing of CMA