Re: [PATCH v9 3/5] virtio_balloon: introduce migration primitives to balloon pages

2012-08-27 Thread Rafael Aquini
On Sun, Aug 26, 2012 at 10:42:44AM +0300, Michael S. Tsirkin wrote: Reading two atomics and doing math? Result can even be negative. I did not look at use closely but it looks suspicious. Doc on atomic_read says: The read is atomic in that the return value is guaranteed to be one of the

Re: [PATCH v9 0/5] make balloon pages movable by compaction

2012-08-27 Thread Rafael Aquini
On Sun, Aug 26, 2012 at 06:44:23PM +0300, Michael S. Tsirkin wrote: I am simply asking how was this patchset tested. It would be nice to have this info in commit log. Since this is an optimization patch it is strange to see one with no numbers at all. For example, you probably run some

Re: [PATCH v9 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-27 Thread Rafael Aquini
On Sun, Aug 26, 2012 at 10:55:58AM +0300, Michael S. Tsirkin wrote: On Sat, Aug 25, 2012 at 02:24:56AM -0300, Rafael Aquini wrote: Memory fragmentation introduced by ballooning might reduce significantly the number of 2MB contiguous memory blocks that can be used within a guest, thus

[PATCH v9 3/5] virtio_balloon: introduce migration primitives to balloon pages

2012-08-24 Thread Rafael Aquini
queue operations; - pages_lock (spinlock): special protection to balloon's pages bookmarking elements (list and atomic counters) against the potential memory compaction concurrency; Signed-off-by: Rafael Aquini --- drivers/virtio

[PATCH v9 5/5] mm: add vm event counters for balloon pages compaction

2012-08-24 Thread Rafael Aquini
This patch introduces a new set of vm event counters to keep track of ballooned pages compaction activity. Signed-off-by: Rafael Aquini --- drivers/virtio/virtio_balloon.c | 1 + include/linux/vm_event_item.h | 8 +++- mm/balloon_compaction.c | 2 ++ mm/migrate.c

[PATCH v9 4/5] mm: introduce putback_movable_pages()

2012-08-24 Thread Rafael Aquini
this patch introduces putback_movable_pages() to properly cope with cases where the isolated pageset contains ballooned pages and LRU pages, thus fixing the mentioned inelegant hack around putback_lru_pages(). Signed-off-by: Rafael Aquini --- include/linux/migrate.h | 2 ++ mm/compaction.c |

[PATCH v9 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-24 Thread Rafael Aquini
. This patch introduces a common interface to help a balloon driver on making its page set movable to compaction, and thus allowing the system to better leverage the compation efforts on memory defragmentation. Signed-off-by: Rafael Aquini --- include/linux/balloon_compaction.h | 137

[PATCH v9 0/5] make balloon pages movable by compaction

2012-08-24 Thread Rafael Aquini
ose subsystems aware of ballooned pages and allow memory balloon pages become movable within a guest, thus avoiding the aforementioned fragmentation issue Rafael Aquini (5): mm: introduce a common interface for balloon pages mobility mm: introduce compaction and migration for ballooned pages

[PATCH v9 2/5] mm: introduce compaction and migration for ballooned pages

2012-08-24 Thread Rafael Aquini
. This patch introduces the helper functions as well as the necessary changes to teach compaction and migration bits how to cope with pages which are part of a guest memory balloon, in order to make them movable by memory compaction procedures. Signed-off-by: Rafael Aquini --- mm/compaction.c | 47

[PATCH v9 2/5] mm: introduce compaction and migration for ballooned pages

2012-08-24 Thread Rafael Aquini
. This patch introduces the helper functions as well as the necessary changes to teach compaction and migration bits how to cope with pages which are part of a guest memory balloon, in order to make them movable by memory compaction procedures. Signed-off-by: Rafael Aquini aqu...@redhat.com --- mm

[PATCH v9 0/5] make balloon pages movable by compaction

2012-08-24 Thread Rafael Aquini
aware of ballooned pages and allow memory balloon pages become movable within a guest, thus avoiding the aforementioned fragmentation issue Rafael Aquini (5): mm: introduce a common interface for balloon pages mobility mm: introduce compaction and migration for ballooned pages virtio_balloon

[PATCH v9 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-24 Thread Rafael Aquini
. This patch introduces a common interface to help a balloon driver on making its page set movable to compaction, and thus allowing the system to better leverage the compation efforts on memory defragmentation. Signed-off-by: Rafael Aquini aqu...@redhat.com --- include/linux/balloon_compaction.h | 137

[PATCH v9 4/5] mm: introduce putback_movable_pages()

2012-08-24 Thread Rafael Aquini
putback_movable_pages() to properly cope with cases where the isolated pageset contains ballooned pages and LRU pages, thus fixing the mentioned inelegant hack around putback_lru_pages(). Signed-off-by: Rafael Aquini aqu...@redhat.com --- include/linux/migrate.h | 2 ++ mm/compaction.c

[PATCH v9 5/5] mm: add vm event counters for balloon pages compaction

2012-08-24 Thread Rafael Aquini
This patch introduces a new set of vm event counters to keep track of ballooned pages compaction activity. Signed-off-by: Rafael Aquini aqu...@redhat.com --- drivers/virtio/virtio_balloon.c | 1 + include/linux/vm_event_item.h | 8 +++- mm/balloon_compaction.c | 2 ++ mm

[PATCH v9 3/5] virtio_balloon: introduce migration primitives to balloon pages

2012-08-24 Thread Rafael Aquini
queue operations; - pages_lock (spinlock): special protection to balloon's pages bookmarking elements (list and atomic counters) against the potential memory compaction concurrency; Signed-off-by: Rafael Aquini aqu...@redhat.com --- drivers

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 09:38:48PM -0300, Rafael Aquini wrote: > On Thu, Aug 23, 2012 at 09:33:53PM -0300, Rafael Aquini wrote: > > On Fri, Aug 24, 2012 at 02:36:16AM +0300, Michael S. Tsirkin wrote: > > > I would wake it each time after adding a page, then it > > > can

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 09:33:53PM -0300, Rafael Aquini wrote: > On Fri, Aug 24, 2012 at 02:36:16AM +0300, Michael S. Tsirkin wrote: > > I would wake it each time after adding a page, then it > > can stop waiting when it leaks enough. > > But again, it's cleaner to just keep t

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Fri, Aug 24, 2012 at 02:36:16AM +0300, Michael S. Tsirkin wrote: > I would wake it each time after adding a page, then it > can stop waiting when it leaks enough. > But again, it's cleaner to just keep tracking all > pages, let mm hang on to them by keeping a reference. > Here is a rough idea

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Fri, Aug 24, 2012 at 02:36:16AM +0300, Michael S. Tsirkin wrote: > On Thu, Aug 23, 2012 at 02:28:45PM -0300, Rafael Aquini wrote: > > On Thu, Aug 23, 2012 at 07:25:05PM +0300, Michael S. Tsirkin wrote: > > > On Thu, Aug 23, 2012 at 04:53:28PM +0300, Michael S. Tsirkin wrote:

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 07:25:05PM +0300, Michael S. Tsirkin wrote: > On Thu, Aug 23, 2012 at 04:53:28PM +0300, Michael S. Tsirkin wrote: > > Basically it was very simple: we assumed page->lru was never > > touched for an allocated page, so it's safe to use it for > > internal book-keeping by the

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 12:03:15PM -0400, Rik van Riel wrote: > > > >Not "longer" - apparently forever unless user resend the leak command. > >It's wrong - it should > >1. not tell host if nothing was done > >2. after migration finished leak and tell host > > Agreed. If the balloon is told to

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 04:53:29PM +0300, Michael S. Tsirkin wrote: > On Thu, Aug 23, 2012 at 10:06:07AM -0300, Rafael Aquini wrote: > > On Thu, Aug 23, 2012 at 03:34:32PM +0300, Michael S. Tsirkin wrote: > > > > So, nothing has changed here. > > > > >

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 03:34:32PM +0300, Michael S. Tsirkin wrote: > > So, nothing has changed here. > > Yes, your patch does change things: > leak_balloon now might return without freeing any pages. > In that case we will not be making any progress, and just > spin, pinning CPU. That's a

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 01:01:07PM +0300, Michael S. Tsirkin wrote: > > So, when remove_common() calls leak_balloon() looping on > > vb->num_pages, that won't become a tight loop. > > The scheme was apparently working before this series, and it will remain > > working > > after it. > > It seems

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 01:01:07PM +0300, Michael S. Tsirkin wrote: So, when remove_common() calls leak_balloon() looping on vb-num_pages, that won't become a tight loop. The scheme was apparently working before this series, and it will remain working after it. It seems that before

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 03:34:32PM +0300, Michael S. Tsirkin wrote: So, nothing has changed here. Yes, your patch does change things: leak_balloon now might return without freeing any pages. In that case we will not be making any progress, and just spin, pinning CPU. That's a transitory

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 04:53:29PM +0300, Michael S. Tsirkin wrote: On Thu, Aug 23, 2012 at 10:06:07AM -0300, Rafael Aquini wrote: On Thu, Aug 23, 2012 at 03:34:32PM +0300, Michael S. Tsirkin wrote: So, nothing has changed here. Yes, your patch does change things: leak_balloon now

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 12:03:15PM -0400, Rik van Riel wrote: Not longer - apparently forever unless user resend the leak command. It's wrong - it should 1. not tell host if nothing was done 2. after migration finished leak and tell host Agreed. If the balloon is told to leak N pages,

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 07:25:05PM +0300, Michael S. Tsirkin wrote: On Thu, Aug 23, 2012 at 04:53:28PM +0300, Michael S. Tsirkin wrote: Basically it was very simple: we assumed page-lru was never touched for an allocated page, so it's safe to use it for internal book-keeping by the driver.

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Fri, Aug 24, 2012 at 02:36:16AM +0300, Michael S. Tsirkin wrote: On Thu, Aug 23, 2012 at 02:28:45PM -0300, Rafael Aquini wrote: On Thu, Aug 23, 2012 at 07:25:05PM +0300, Michael S. Tsirkin wrote: On Thu, Aug 23, 2012 at 04:53:28PM +0300, Michael S. Tsirkin wrote: Basically it was very

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Fri, Aug 24, 2012 at 02:36:16AM +0300, Michael S. Tsirkin wrote: I would wake it each time after adding a page, then it can stop waiting when it leaks enough. But again, it's cleaner to just keep tracking all pages, let mm hang on to them by keeping a reference. Here is a rough idea on

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 09:33:53PM -0300, Rafael Aquini wrote: On Fri, Aug 24, 2012 at 02:36:16AM +0300, Michael S. Tsirkin wrote: I would wake it each time after adding a page, then it can stop waiting when it leaks enough. But again, it's cleaner to just keep tracking all pages, let mm

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 09:38:48PM -0300, Rafael Aquini wrote: On Thu, Aug 23, 2012 at 09:33:53PM -0300, Rafael Aquini wrote: On Fri, Aug 24, 2012 at 02:36:16AM +0300, Michael S. Tsirkin wrote: I would wake it each time after adding a page, then it can stop waiting when it leaks enough

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-22 Thread Rafael Aquini
On Wed, Aug 22, 2012 at 12:33:17PM +0300, Michael S. Tsirkin wrote: > Hmm, so this will busy wait which is unelegant. > We need some event IMO. No, it does not busy wait. leak_balloon() is mutual exclusive with migration steps, so for the case we have one racing against the other, we really want

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-22 Thread Rafael Aquini
On Wed, Aug 22, 2012 at 12:33:17PM +0300, Michael S. Tsirkin wrote: Hmm, so this will busy wait which is unelegant. We need some event IMO. No, it does not busy wait. leak_balloon() is mutual exclusive with migration steps, so for the case we have one racing against the other, we really want

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Wed, Aug 22, 2012 at 03:07:41AM +0300, Michael S. Tsirkin wrote: > On Tue, Aug 21, 2012 at 05:45:56PM -0300, Rafael Aquini wrote: > > On Tue, Aug 21, 2012 at 10:30:31PM +0300, Michael S. Tsirkin wrote: > > > On Tue, Aug 21, 2012 at 04:23:58PM -0300, Rafael Aquini wrote: >

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 10:30:31PM +0300, Michael S. Tsirkin wrote: > On Tue, Aug 21, 2012 at 04:23:58PM -0300, Rafael Aquini wrote: > > On Tue, Aug 21, 2012 at 10:13:30PM +0300, Michael S. Tsirkin wrote: > > > > > > > > I believe rcu_dereference_protected(

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 10:16:12PM +0300, Michael S. Tsirkin wrote: > On Tue, Aug 21, 2012 at 02:55:03PM -0300, Rafael Aquini wrote: > > On Tue, Aug 21, 2012 at 04:52:23PM +0300, Michael S. Tsirkin wrote: > > > > + * address_space_operations utilized method

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 10:13:30PM +0300, Michael S. Tsirkin wrote: > > > > I believe rcu_dereference_protected() is what I want/need here, since this > > code > > is always called for pages which we hold locked (PG_locked bit). > > It would only help if we locked the page while updating the

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 09:24:32AM -0700, Paul E. McKenney wrote: > On Tue, Aug 21, 2012 at 05:20:11PM +0200, Peter Zijlstra wrote: > > On Tue, 2012-08-21 at 09:47 -0300, Rafael Aquini wrote: > > > + mapping = rcu_access_pointer(page->mapping); >

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 06:41:42PM +0300, Michael S. Tsirkin wrote: > On Tue, Aug 21, 2012 at 05:16:06PM +0200, Peter Zijlstra wrote: > > On Tue, 2012-08-21 at 16:52 +0300, Michael S. Tsirkin wrote: > > > > + rcu_read_lock(); > > > > + mapping =

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 04:52:23PM +0300, Michael S. Tsirkin wrote: > > + * address_space_operations utilized methods for ballooned pages: > > + * .migratepage- used to perform balloon's page migration (as is) > > + * .launder_page - used to isolate a page from balloon's page list > > +

[PATCH v8 3/5] virtio_balloon: introduce migration primitives to balloon pages

2012-08-21 Thread Rafael Aquini
against concurrent list handling operations; - virtio_baloon->pages list handling sync by RCU operations; Signed-off-by: Rafael Aquini --- drivers/virtio/virtio_balloon.c | 210 +--- 1 file changed, 199 insertions(+), 11 deleti

[PATCH v8 2/5] mm: introduce compaction and migration for ballooned pages

2012-08-21 Thread Rafael Aquini
. This patch introduces the helper functions as well as the necessary changes to teach compaction and migration bits how to cope with pages which are part of a guest memory balloon, in order to make them movable by memory compaction procedures. Signed-off-by: Rafael Aquini --- mm/compaction.c | 47

[PATCH v8 4/5] mm: introduce putback_movable_pages()

2012-08-21 Thread Rafael Aquini
this patch introduces putback_movable_pages() to properly cope with cases where the isolated pageset contains ballooned pages and LRU pages, thus fixing the mentioned inelegant hack around putback_lru_pages(). Signed-off-by: Rafael Aquini --- include/linux/migrate.h | 2 ++ mm/compaction.c |

[PATCH v8 5/5] mm: add vm event counters for balloon pages compaction

2012-08-21 Thread Rafael Aquini
This patch introduces a new set of vm event counters to keep track of ballooned pages compaction activity. Signed-off-by: Rafael Aquini --- drivers/virtio/virtio_balloon.c | 2 ++ include/linux/vm_event_item.h | 8 +++- mm/balloon_compaction.c | 6 -- mm/migrate.c

[PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
. This patch introduces a common interface to help a balloon driver on making its page set movable to compaction, and thus allowing the system to better leverage the compation efforts on memory defragmentation. Signed-off-by: Rafael Aquini --- include/linux/balloon_compaction.h | 113

[PATCH v8 0/5] make balloon pages movable by compaction

2012-08-21 Thread Rafael Aquini
ose subsystems aware of ballooned pages and allow memory balloon pages become movable within a guest, thus avoiding the aforementioned fragmentation issue Rafael Aquini (5): mm: introduce a common interface for balloon pages mobility mm: introduce compaction and migration for ballooned pages

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Wed, Aug 22, 2012 at 03:07:41AM +0300, Michael S. Tsirkin wrote: On Tue, Aug 21, 2012 at 05:45:56PM -0300, Rafael Aquini wrote: On Tue, Aug 21, 2012 at 10:30:31PM +0300, Michael S. Tsirkin wrote: On Tue, Aug 21, 2012 at 04:23:58PM -0300, Rafael Aquini wrote: On Tue, Aug 21, 2012 at 10

[PATCH v8 0/5] make balloon pages movable by compaction

2012-08-21 Thread Rafael Aquini
aware of ballooned pages and allow memory balloon pages become movable within a guest, thus avoiding the aforementioned fragmentation issue Rafael Aquini (5): mm: introduce a common interface for balloon pages mobility mm: introduce compaction and migration for ballooned pages virtio_balloon

[PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
. This patch introduces a common interface to help a balloon driver on making its page set movable to compaction, and thus allowing the system to better leverage the compation efforts on memory defragmentation. Signed-off-by: Rafael Aquini aqu...@redhat.com --- include/linux/balloon_compaction.h | 113

[PATCH v8 5/5] mm: add vm event counters for balloon pages compaction

2012-08-21 Thread Rafael Aquini
This patch introduces a new set of vm event counters to keep track of ballooned pages compaction activity. Signed-off-by: Rafael Aquini aqu...@redhat.com --- drivers/virtio/virtio_balloon.c | 2 ++ include/linux/vm_event_item.h | 8 +++- mm/balloon_compaction.c | 6 -- mm

[PATCH v8 4/5] mm: introduce putback_movable_pages()

2012-08-21 Thread Rafael Aquini
putback_movable_pages() to properly cope with cases where the isolated pageset contains ballooned pages and LRU pages, thus fixing the mentioned inelegant hack around putback_lru_pages(). Signed-off-by: Rafael Aquini aqu...@redhat.com --- include/linux/migrate.h | 2 ++ mm/compaction.c

[PATCH v8 2/5] mm: introduce compaction and migration for ballooned pages

2012-08-21 Thread Rafael Aquini
. This patch introduces the helper functions as well as the necessary changes to teach compaction and migration bits how to cope with pages which are part of a guest memory balloon, in order to make them movable by memory compaction procedures. Signed-off-by: Rafael Aquini aqu...@redhat.com --- mm

[PATCH v8 3/5] virtio_balloon: introduce migration primitives to balloon pages

2012-08-21 Thread Rafael Aquini
against concurrent list handling operations; - virtio_baloon-pages list handling sync by RCU operations; Signed-off-by: Rafael Aquini aqu...@redhat.com --- drivers/virtio/virtio_balloon.c | 210 +--- 1 file changed, 199 insertions(+), 11

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 04:52:23PM +0300, Michael S. Tsirkin wrote: + * address_space_operations utilized methods for ballooned pages: + * .migratepage- used to perform balloon's page migration (as is) + * .launder_page - used to isolate a page from balloon's page list + *

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 06:41:42PM +0300, Michael S. Tsirkin wrote: On Tue, Aug 21, 2012 at 05:16:06PM +0200, Peter Zijlstra wrote: On Tue, 2012-08-21 at 16:52 +0300, Michael S. Tsirkin wrote: + rcu_read_lock(); + mapping = rcu_dereference(page-mapping); +

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 09:24:32AM -0700, Paul E. McKenney wrote: On Tue, Aug 21, 2012 at 05:20:11PM +0200, Peter Zijlstra wrote: On Tue, 2012-08-21 at 09:47 -0300, Rafael Aquini wrote: + mapping = rcu_access_pointer(page-mapping); + if (mapping) + mapping

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 10:13:30PM +0300, Michael S. Tsirkin wrote: I believe rcu_dereference_protected() is what I want/need here, since this code is always called for pages which we hold locked (PG_locked bit). It would only help if we locked the page while updating the mapping, as

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 10:16:12PM +0300, Michael S. Tsirkin wrote: On Tue, Aug 21, 2012 at 02:55:03PM -0300, Rafael Aquini wrote: On Tue, Aug 21, 2012 at 04:52:23PM +0300, Michael S. Tsirkin wrote: + * address_space_operations utilized methods for ballooned pages: + * .migratepage

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 10:30:31PM +0300, Michael S. Tsirkin wrote: On Tue, Aug 21, 2012 at 04:23:58PM -0300, Rafael Aquini wrote: On Tue, Aug 21, 2012 at 10:13:30PM +0300, Michael S. Tsirkin wrote: I believe rcu_dereference_protected() is what I want/need here, since this code

Re: [PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-15 Thread Rafael Aquini
On Tue, Aug 14, 2012 at 10:31:09PM +0300, Michael S. Tsirkin wrote: > > > now CPU1 executes the next instruction: > > > > > > } > > > > > > which would normally return to function's caller, > > > but it has been overwritten by CPU2 so we get corruption. > > > > > > No? > > > > At the point

Re: [PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-15 Thread Rafael Aquini
On Tue, Aug 14, 2012 at 10:31:09PM +0300, Michael S. Tsirkin wrote: now CPU1 executes the next instruction: } which would normally return to function's caller, but it has been overwritten by CPU2 so we get corruption. No? At the point CPU2 is unloading the module,

Re: [PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-14 Thread Rafael Aquini
On Tue, Aug 14, 2012 at 11:49:06PM +0300, Michael S. Tsirkin wrote: > On Tue, Aug 14, 2012 at 05:29:50PM -0300, Rafael Aquini wrote: > > On Tue, Aug 14, 2012 at 11:24:01PM +0300, Michael S. Tsirkin wrote: > > > On Tue, Aug 14, 2012 at 05:08:31PM -0300, Rafael Aquini wrote: >

Re: [PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-14 Thread Rafael Aquini
On Tue, Aug 14, 2012 at 11:24:01PM +0300, Michael S. Tsirkin wrote: > On Tue, Aug 14, 2012 at 05:08:31PM -0300, Rafael Aquini wrote: > > On Tue, Aug 14, 2012 at 10:59:16PM +0300, Michael S. Tsirkin wrote: > > > > > > What if there is m

Re: [PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-14 Thread Rafael Aquini
On Tue, Aug 14, 2012 at 10:51:39PM +0300, Michael S. Tsirkin wrote: > What I think you should do is use rcu for access. > And here sync rcu before freeing. > Maybe an overkill but at least a documented synchronization > primitive, and it is very light weight. > I liked your suggestion on

Re: [PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-14 Thread Rafael Aquini
On Tue, Aug 14, 2012 at 10:59:16PM +0300, Michael S. Tsirkin wrote: > > > > What if there is more than one balloon device? > > > > > > Is it possible to load this driver twice, or are you foreseeing a future > > > case > > > where this driver will be able to manage several distinct memory

Re: [PATCH v7 1/4] mm: introduce compaction and migration for virtio ballooned pages

2012-08-14 Thread Rafael Aquini
On Tue, Aug 14, 2012 at 10:48:37PM +0300, Michael S. Tsirkin wrote: > > > > E.g. kvm can emulate hyperv so it could in theory have hyperv balloon. > > This is mm stuff it is best not to tie it to specific drivers. > > But of course I agree this is not top priority, no need > to block submission

Re: [PATCH v7 1/4] mm: introduce compaction and migration for virtio ballooned pages

2012-08-14 Thread Rafael Aquini
On Tue, Aug 14, 2012 at 10:35:25PM +0300, Michael S. Tsirkin wrote: > > > > +/* __isolate_lru_page() counterpart for a ballooned page */ > > > > +bool isolate_balloon_page(struct page *page) > > > > +{ > > > > + if (WARN_ON(!movable_balloon_page(page))) > > > > > > Looks like this actually

Re: [PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-14 Thread Rafael Aquini
On Tue, Aug 14, 2012 at 11:33:20AM +0300, Michael S. Tsirkin wrote: > On Tue, Aug 14, 2012 at 09:29:49AM +0930, Rusty Russell wrote: > > On Mon, 13 Aug 2012 11:41:23 +0300, "Michael S. Tsirkin" > > wrote: > > > On Fri, Aug 10, 2012 at 02:

Re: [PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-14 Thread Rafael Aquini
On Mon, Aug 13, 2012 at 11:41:23AM +0300, Michael S. Tsirkin wrote: > > @@ -141,7 +151,10 @@ static void fill_balloon(struct virtio_balloon *vb, > > size_t num) > > set_page_pfns(vb->pfns + vb->num_pfns, page); > > vb->num_pages += VIRTIO_BALLOON_PAGES_PER_PAGE; > >

Re: [PATCH v7 1/4] mm: introduce compaction and migration for virtio ballooned pages

2012-08-14 Thread Rafael Aquini
On Mon, Aug 13, 2012 at 11:26:19AM +0300, Michael S. Tsirkin wrote: > > +static inline bool movable_balloon_page(struct page *page) > > +{ > > + return (page->mapping && page->mapping == balloon_mapping); > > I am guessing this needs smp_read_barrier_depends, and maybe > ACCESS_ONCE ... > I'm

Re: [PATCH v7 1/4] mm: introduce compaction and migration for virtio ballooned pages

2012-08-14 Thread Rafael Aquini
On Mon, Aug 13, 2012 at 11:26:19AM +0300, Michael S. Tsirkin wrote: +static inline bool movable_balloon_page(struct page *page) +{ + return (page-mapping page-mapping == balloon_mapping); I am guessing this needs smp_read_barrier_depends, and maybe ACCESS_ONCE ... I'm curious about

Re: [PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-14 Thread Rafael Aquini
On Mon, Aug 13, 2012 at 11:41:23AM +0300, Michael S. Tsirkin wrote: @@ -141,7 +151,10 @@ static void fill_balloon(struct virtio_balloon *vb, size_t num) set_page_pfns(vb-pfns + vb-num_pfns, page); vb-num_pages += VIRTIO_BALLOON_PAGES_PER_PAGE;

Re: [PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-14 Thread Rafael Aquini
On Tue, Aug 14, 2012 at 11:33:20AM +0300, Michael S. Tsirkin wrote: On Tue, Aug 14, 2012 at 09:29:49AM +0930, Rusty Russell wrote: On Mon, 13 Aug 2012 11:41:23 +0300, Michael S. Tsirkin m...@redhat.com wrote: On Fri, Aug 10, 2012 at 02:55:15PM -0300, Rafael Aquini wrote

Re: [PATCH v7 1/4] mm: introduce compaction and migration for virtio ballooned pages

2012-08-14 Thread Rafael Aquini
On Tue, Aug 14, 2012 at 10:35:25PM +0300, Michael S. Tsirkin wrote: +/* __isolate_lru_page() counterpart for a ballooned page */ +bool isolate_balloon_page(struct page *page) +{ + if (WARN_ON(!movable_balloon_page(page))) Looks like this actually can happen if the page

Re: [PATCH v7 1/4] mm: introduce compaction and migration for virtio ballooned pages

2012-08-14 Thread Rafael Aquini
On Tue, Aug 14, 2012 at 10:48:37PM +0300, Michael S. Tsirkin wrote: E.g. kvm can emulate hyperv so it could in theory have hyperv balloon. This is mm stuff it is best not to tie it to specific drivers. But of course I agree this is not top priority, no need to block submission on this,

Re: [PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-14 Thread Rafael Aquini
On Tue, Aug 14, 2012 at 10:59:16PM +0300, Michael S. Tsirkin wrote: What if there is more than one balloon device? Is it possible to load this driver twice, or are you foreseeing a future case where this driver will be able to manage several distinct memory balloons for

Re: [PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-14 Thread Rafael Aquini
On Tue, Aug 14, 2012 at 10:51:39PM +0300, Michael S. Tsirkin wrote: What I think you should do is use rcu for access. And here sync rcu before freeing. Maybe an overkill but at least a documented synchronization primitive, and it is very light weight. I liked your suggestion on barriers, as

Re: [PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-14 Thread Rafael Aquini
On Tue, Aug 14, 2012 at 11:24:01PM +0300, Michael S. Tsirkin wrote: On Tue, Aug 14, 2012 at 05:08:31PM -0300, Rafael Aquini wrote: On Tue, Aug 14, 2012 at 10:59:16PM +0300, Michael S. Tsirkin wrote: What if there is more than one balloon device? Is it possible to load

Re: [PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-14 Thread Rafael Aquini
On Tue, Aug 14, 2012 at 11:49:06PM +0300, Michael S. Tsirkin wrote: On Tue, Aug 14, 2012 at 05:29:50PM -0300, Rafael Aquini wrote: On Tue, Aug 14, 2012 at 11:24:01PM +0300, Michael S. Tsirkin wrote: On Tue, Aug 14, 2012 at 05:08:31PM -0300, Rafael Aquini wrote: On Tue, Aug 14, 2012 at 10

[PATCH v7 1/4] mm: introduce compaction and migration for virtio ballooned pages

2012-08-10 Thread Rafael Aquini
. This patch introduces the helper functions as well as the necessary changes to teach compaction and migration bits how to cope with pages which are part of a guest memory balloon, in order to make them movable by memory compaction procedures. Signed-off-by: Rafael Aquini --- include/linux/mm.h | 17

[PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-10 Thread Rafael Aquini
against concurrent list handling operations; Signed-off-by: Rafael Aquini --- drivers/virtio/virtio_balloon.c | 138 +--- include/linux/virtio_balloon.h | 4 ++ 2 files changed, 134 insertions(+), 8 deletions(-) diff --git a/drivers

[PATCH v7 3/4] mm: introduce putback_movable_pages()

2012-08-10 Thread Rafael Aquini
this patch introduces putback_movable_pages() to properly cope with cases where the isolated pageset contains ballooned pages and LRU pages, thus fixing the mentioned inelegant hack around putback_lru_pages(). Signed-off-by: Rafael Aquini --- include/linux/migrate.h | 2 ++ mm/compaction.c |

[PATCH v7 4/4] mm: add vm event counters for balloon pages compaction

2012-08-10 Thread Rafael Aquini
This patch introduces a new set of vm event counters to keep track of ballooned pages compaction activity. Signed-off-by: Rafael Aquini --- drivers/virtio/virtio_balloon.c | 1 + include/linux/vm_event_item.h | 8 +++- mm/compaction.c | 2 ++ mm/migrate.c

[PATCH v7 0/4] make balloon pages movable by compaction

2012-08-10 Thread Rafael Aquini
ose subsystems aware of ballooned pages and allow memory balloon pages become movable within a guest, thus avoiding the aforementioned fragmentation issue Rafael Aquini (4): mm: introduce compaction and migration for virtio ballooned pages virtio_balloon: introduce migration primitives to ball

[PATCH v7 0/4] make balloon pages movable by compaction

2012-08-10 Thread Rafael Aquini
aware of ballooned pages and allow memory balloon pages become movable within a guest, thus avoiding the aforementioned fragmentation issue Rafael Aquini (4): mm: introduce compaction and migration for virtio ballooned pages virtio_balloon: introduce migration primitives to balloon pages mm

[PATCH v7 4/4] mm: add vm event counters for balloon pages compaction

2012-08-10 Thread Rafael Aquini
This patch introduces a new set of vm event counters to keep track of ballooned pages compaction activity. Signed-off-by: Rafael Aquini aqu...@redhat.com --- drivers/virtio/virtio_balloon.c | 1 + include/linux/vm_event_item.h | 8 +++- mm/compaction.c | 2 ++ mm

[PATCH v7 3/4] mm: introduce putback_movable_pages()

2012-08-10 Thread Rafael Aquini
putback_movable_pages() to properly cope with cases where the isolated pageset contains ballooned pages and LRU pages, thus fixing the mentioned inelegant hack around putback_lru_pages(). Signed-off-by: Rafael Aquini aqu...@redhat.com --- include/linux/migrate.h | 2 ++ mm/compaction.c

[PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-10 Thread Rafael Aquini
against concurrent list handling operations; Signed-off-by: Rafael Aquini aqu...@redhat.com --- drivers/virtio/virtio_balloon.c | 138 +--- include/linux/virtio_balloon.h | 4 ++ 2 files changed, 134 insertions(+), 8 deletions(-) diff

[PATCH v7 1/4] mm: introduce compaction and migration for virtio ballooned pages

2012-08-10 Thread Rafael Aquini
. This patch introduces the helper functions as well as the necessary changes to teach compaction and migration bits how to cope with pages which are part of a guest memory balloon, in order to make them movable by memory compaction procedures. Signed-off-by: Rafael Aquini aqu...@redhat.com --- include/linux

Re: [PATCH v6 1/3] mm: introduce compaction and migration for virtio ballooned pages

2012-08-09 Thread Rafael Aquini
On Thu, Aug 09, 2012 at 11:48:36AM -0300, Rafael Aquini wrote: > Sure! > what do you think of: > > +/* putback_lru_page() counterpart for a ballooned page */ > +void putback_balloon_page(struct page *page) > +{ > + lock_page(page); > + if (!WARN_ON(!m

Re: [PATCH v6 1/3] mm: introduce compaction and migration for virtio ballooned pages

2012-08-09 Thread Rafael Aquini
On Thu, Aug 09, 2012 at 10:00:19AM +0100, Mel Gorman wrote: > On Wed, Aug 08, 2012 at 07:53:19PM -0300, Rafael Aquini wrote: > > Memory fragmentation introduced by ballooning might reduce significantly > > the number of 2MB contiguous memory blocks that can be used within a

Re: [PATCH v6 1/3] mm: introduce compaction and migration for virtio ballooned pages

2012-08-09 Thread Rafael Aquini
On Thu, Aug 09, 2012 at 10:00:19AM +0100, Mel Gorman wrote: On Wed, Aug 08, 2012 at 07:53:19PM -0300, Rafael Aquini wrote: Memory fragmentation introduced by ballooning might reduce significantly the number of 2MB contiguous memory blocks that can be used within a guest, thus imposing

Re: [PATCH v6 1/3] mm: introduce compaction and migration for virtio ballooned pages

2012-08-09 Thread Rafael Aquini
On Thu, Aug 09, 2012 at 11:48:36AM -0300, Rafael Aquini wrote: Sure! what do you think of: +/* putback_lru_page() counterpart for a ballooned page */ +void putback_balloon_page(struct page *page) +{ + lock_page(page); + if (!WARN_ON(!movable_balloon_page(page

[PATCH v6 2/3] virtio_balloon: introduce migration primitives to balloon pages

2012-08-08 Thread Rafael Aquini
against concurrent list handling operations; Signed-off-by: Rafael Aquini --- drivers/virtio/virtio_balloon.c | 138 +--- include/linux/virtio_balloon.h | 4 ++ 2 files changed, 134 insertions(+), 8 deletions(-) diff --git a/drivers

[PATCH v6 0/3] make balloon pages movable by compaction

2012-08-08 Thread Rafael Aquini
ose subsystems aware of ballooned pages and allow memory balloon pages become movable within a guest, thus avoiding the aforementioned fragmentation issue Rafael Aquini (3): mm: introduce compaction and migration for virtio ballooned pages virtio_balloon: introduce migration primitives to ball

[PATCH v6 1/3] mm: introduce compaction and migration for virtio ballooned pages

2012-08-08 Thread Rafael Aquini
. This patch introduces the helper functions as well as the necessary changes to teach compaction and migration bits how to cope with pages which are part of a guest memory balloon, in order to make them movable by memory compaction procedures. Signed-off-by: Rafael Aquini --- include/linux/mm.h | 17

[PATCH v6 3/3] mm: add vm event counters for balloon pages compaction

2012-08-08 Thread Rafael Aquini
This patch is only for testing report purposes and shall be dropped in case of the rest of this patchset getting accepted for merging. Signed-off-by: Rafael Aquini --- drivers/virtio/virtio_balloon.c | 1 + include/linux/vm_event_item.h | 2 ++ mm/compaction.c | 1 + mm

[PATCH v6 3/3] mm: add vm event counters for balloon pages compaction

2012-08-08 Thread Rafael Aquini
This patch is only for testing report purposes and shall be dropped in case of the rest of this patchset getting accepted for merging. Signed-off-by: Rafael Aquini aqu...@redhat.com --- drivers/virtio/virtio_balloon.c | 1 + include/linux/vm_event_item.h | 2 ++ mm/compaction.c

<    1   2   3   4   5   6   7   >