Re: [PATCH 15/20] balloon: Convert to migrate_folio

2022-06-08 Thread Rafael Aquini
On Wed, Jun 08, 2022 at 11:59:31AM +0200, David Hildenbrand wrote: > On 07.06.22 21:21, Matthew Wilcox wrote: > > On Tue, Jun 07, 2022 at 03:24:15PM +0100, Matthew Wilcox wrote: > >> On Tue, Jun 07, 2022 at 09:36:21AM +0200, David Hildenbrand wrote: > >>> On 06.06.22 22:40, Matthew Wilcox (Oracle)

Re: [PATCH] mm/balloon_compaction: make balloon page compaction callbacks static

2022-01-25 Thread Rafael Aquini
> unsigned long flags; > @@ -230,7 +230,7 @@ void balloon_page_putback(struct page *page) > > > /* move_to_new_page() counterpart for a ballooned page */ > -int balloon_page_migrate(struct address_space *mapping, > +static int balloon_page_migrate(struct address_space *m

Re: [PATCH 2/2] virtio_balloon: fix race between migration and ballooning

2016-01-09 Thread Rafael Aquini
On Sat, Jan 09, 2016 at 11:43:31PM +0200, Michael S. Tsirkin wrote: > On Fri, Jan 08, 2016 at 02:56:14PM -0500, Rafael Aquini wrote: > > On Fri, Jan 01, 2016 at 11:36:13AM +0200, Michael S. Tsirkin wrote: > > > On Mon, Dec 28, 2015 at 08:35:13AM +0900,

Re: [PATCH 2/2] virtio_balloon: fix race between migration and ballooning

2016-01-08 Thread Rafael Aquini
On Fri, Jan 01, 2016 at 11:36:13AM +0200, Michael S. Tsirkin wrote: > On Mon, Dec 28, 2015 at 08:35:13AM +0900, Minchan Kim wrote: > > In balloon_page_dequeue, pages_lock should cover the loop > > (ie, list_for_each_entry_safe). Otherwise, the cursor page could > > be isolated by compaction and

Re: [PATCH 2/2] virtio_balloon: fix race between migration and ballooning

2015-12-27 Thread Rafael Aquini
> __count_vm_event(BALLOON_DEFLATE); > - spin_unlock_irqrestore(_dev_info->pages_lock, flags); > unlock_page(page); > dequeued_page = true; > break; > } > }

Re: [PATCH 1/2] virtio_balloon: fix race by fill and leak

2015-12-27 Thread Rafael Aquini
gt; - mutex_unlock(>balloon_lock); > release_pages_balloon(vb); > + mutex_unlock(>balloon_lock); > return num_freed_pages; > } > > -- > 1.9.1 > Acked-by: Rafael Aquini <aqu...@redhat.com> ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH 0/4] enable migration of driver pages

2015-07-13 Thread Rafael Aquini
/balloon_compaction.c| 72 ++ mm/compaction.c| 8 ++-- mm/migrate.c | 24 +++--- 12 files changed, 160 insertions(+), 74 deletions(-) -- 2.1.4 Acked-by: Rafael Aquini aqu...@redhat.com

Re: [RFCv3 0/5] enable migration of driver pages

2015-07-08 Thread Rafael Aquini
/balloon_compaction.c| 71 ++- mm/compaction.c| 8 ++-- mm/migrate.c | 24 +++ 12 files changed, 154 insertions(+), 75 deletions(-) -- 2.1.4 Acked-by: Rafael Aquini aqu...@redhat.com

Re: [RFC 0/4] enable migration of non-LRU pages

2015-06-05 Thread Rafael Aquini
On Tue, Jun 02, 2015 at 04:27:40PM +0900, Gioh Kim wrote: Hello, This series try to enable migration of non-LRU pages, such as driver's page. My ARM-based platform occured severe fragmentation problem after long-term (several days) test. Sometimes even order-3 page allocation failed. It

Re: [RFC 0/4] enable migration of non-LRU pages

2015-06-05 Thread Rafael Aquini
On Fri, Jun 05, 2015 at 11:35:49PM +0900, Gioh Kim wrote: On Tue, Jun 02, 2015 at 04:27:40PM +0900, Gioh Kim wrote: Hello, This series try to enable migration of non-LRU pages, such as driver's page. My ARM-based platform occured severe fragmentation problem after long-term (several days)

Re: [PATCH v11 4/7] mm: introduce compaction and migration for ballooned pages

2012-11-27 Thread Rafael Aquini
On Tue, Nov 20, 2012 at 03:33:24PM -0800, Andrew Morton wrote: On Fri, 9 Nov 2012 12:16:02 + Mel Gorman m...@csn.ul.ie wrote: On Wed, Nov 07, 2012 at 01:05:51AM -0200, Rafael Aquini wrote: Memory fragmentation introduced by ballooning might reduce significantly the number of 2MB

Re: [PATCH v12 4/7] mm: introduce compaction and migration for ballooned pages

2012-11-22 Thread Rafael Aquini
On Thu, Nov 22, 2012 at 09:19:15AM -0500, Sasha Levin wrote: And managed to reproduce it only once through last night, here is the dump I got before the oops: [ 2760.356820] page:ead00e00 count:1 mapcount:-2147287036 mapping:04f4 index:0xd00e0003 [ 2760.362354]

Re: [PATCH v12 4/7] mm: introduce compaction and migration for ballooned pages

2012-11-20 Thread Rafael Aquini
On Sun, Nov 18, 2012 at 09:59:47AM -0500, Sasha Levin wrote: On Sat, Nov 17, 2012 at 4:54 PM, Rafael Aquini aqu...@redhat.com wrote: On Sat, Nov 17, 2012 at 01:01:30PM -0500, Sasha Levin wrote: I'm getting the following while fuzzing using trinity inside a KVM tools guest, on latest

Re: [PATCH v12 4/7] mm: introduce compaction and migration for ballooned pages

2012-11-17 Thread Rafael Aquini
On Sat, Nov 17, 2012 at 01:01:30PM -0500, Sasha Levin wrote: I'm getting the following while fuzzing using trinity inside a KVM tools guest, on latest -next: [ 1642.783728] BUG: unable to handle kernel NULL pointer dereference at 0194 [ 1642.785083] IP: [8122b354]

Re: [PATCH -next] virtio: balloon: fix missing unlock on error in fill_balloon()

2012-11-13 Thread Rafael Aquini
On Mon, Nov 12, 2012 at 03:34:53PM -0800, Andrew Morton wrote: On Mon, 12 Nov 2012 21:50:40 +0800 Wei Yongjun weiyj...@gmail.com wrote: From: Wei Yongjun yongjun_...@trendmicro.com.cn Add the missing unlock before return from function fill_balloon() in the error handling case.

Re: [patch] virtio_balloon: unlock on error in fill_balloon()

2012-11-13 Thread Rafael Aquini
On Tue, Nov 13, 2012 at 10:43:03AM +0300, Dan Carpenter wrote: We recently added locking in fill_balloon() but there was one error path which was missed. Signed-off-by: Dan Carpenter dan.carpen...@oracle.com --- Only needed in linux-next. diff --git a/drivers/virtio/virtio_balloon.c

Re: [PATCH -next] virtio: balloon: fix missing unlock on error in fill_balloon()

2012-11-12 Thread Rafael Aquini
On Mon, Nov 12, 2012 at 09:50:40PM +0800, Wei Yongjun wrote: From: Wei Yongjun yongjun_...@trendmicro.com.cn Add the missing unlock before return from function fill_balloon() in the error handling case. Introduced by 9864a8(virtio_balloon: introduce migration primitives to balloon pages)

[PATCH v12 0/7] make balloon pages movable by compaction

2012-11-11 Thread Rafael Aquini
747 499 Compaction success 244 145 Compaction failures 503 354 Compaction pages moved 370888 474837 Compaction move failure 77378 65259 ===END stress-highalloc Rafael Aquini (7): mm: adjust

[PATCH v12 1/7] mm: adjust address_space_operations.migratepage() return code

2012-11-11 Thread Rafael Aquini
This patch introduces MIGRATEPAGE_SUCCESS as the default return code for address_space_operations.migratepage() method and documents the expected return code for the same method in failure cases. Signed-off-by: Rafael Aquini aqu...@redhat.com --- fs/hugetlbfs/inode.c| 4 ++-- include/linux

[PATCH v12 2/7] mm: redefine address_space.assoc_mapping

2012-11-11 Thread Rafael Aquini
data structures through -private_data. Also, all users of old -assoc_mapping element are converted to reflect its new name and type change (-private_data). Signed-off-by: Rafael Aquini aqu...@redhat.com --- fs/buffer.c| 12 ++-- fs/gfs2/glock.c| 2 +- fs/inode.c | 2

[PATCH v12 3/7] mm: introduce a common interface for balloon pages mobility

2012-11-11 Thread Rafael Aquini
. This patch introduces a common interface to help a balloon driver on making its page set movable to compaction, and thus allowing the system to better leverage the compation efforts on memory defragmentation. Signed-off-by: Rafael Aquini aqu...@redhat.com Acked-by: Mel Gorman m...@csn.ul.ie --- include

[PATCH v12 4/7] mm: introduce compaction and migration for ballooned pages

2012-11-11 Thread Rafael Aquini
. This patch introduces the helper functions as well as the necessary changes to teach compaction and migration bits how to cope with pages which are part of a guest memory balloon, in order to make them movable by memory compaction procedures. Signed-off-by: Rafael Aquini aqu...@redhat.com Acked-by: Mel

[PATCH v12 5/7] virtio_balloon: introduce migration primitives to balloon pages

2012-11-11 Thread Rafael Aquini
providing protection against concurrent access introduced by parallel memory migration threads. - balloon_lock (mutex) : synchronizes the access demand to elements of struct virtio_balloon and its queue operations; Signed-off-by: Rafael Aquini aqu...@redhat.com Acked-by: Michael

[PATCH v12 7/7] mm: add vm event counters for balloon pages compaction

2012-11-11 Thread Rafael Aquini
This patch introduces a new set of vm event counters to keep track of ballooned pages compaction activity. Signed-off-by: Rafael Aquini aqu...@redhat.com --- include/linux/balloon_compaction.h | 7 +++ include/linux/vm_event_item.h | 7 ++- mm/balloon_compaction.c| 2

Re: [PATCH v11 7/7] mm: add vm event counters for balloon pages compaction

2012-11-11 Thread Rafael Aquini
On Sat, Nov 10, 2012 at 05:55:38PM +0200, Michael S. Tsirkin wrote: mutex_unlock(vb-balloon_lock); + balloon_event_count(COMPACTBALLOONMIGRATED); return MIGRATEPAGE_BALLOON_SUCCESS; } Looks like any ballon would need to do this. Can this chunk go into caller instead?

Re: [PATCH v11 7/7] mm: add vm event counters for balloon pages compaction

2012-11-09 Thread Rafael Aquini
On Fri, Nov 09, 2012 at 12:20:33PM +, Mel Gorman wrote: On Wed, Nov 07, 2012 at 01:05:54AM -0200, Rafael Aquini wrote: This patch introduces a new set of vm event counters to keep track of ballooned pages compaction activity. Signed-off-by: Rafael Aquini aqu...@redhat.com Other

Re: [PATCH v11 3/7] mm: introduce a common interface for balloon pages mobility

2012-11-09 Thread Rafael Aquini
On Fri, Nov 09, 2012 at 04:23:27PM +, Mel Gorman wrote: On Fri, Nov 09, 2012 at 12:53:22PM -0200, Rafael Aquini wrote: SNIP If you get the barrier issue sorted out then feel free to add Acked-by: Mel Gorman m...@csn.ul.ie I believe we can drop the barriers stuff

Re: [PATCH v11 3/7] mm: introduce a common interface for balloon pages mobility

2012-11-07 Thread Rafael Aquini
On Wed, Nov 07, 2012 at 01:02:07PM -0800, Andrew Morton wrote: On Wed, 7 Nov 2012 01:05:50 -0200 Rafael Aquini aqu...@redhat.com wrote: Memory fragmentation introduced by ballooning might reduce significantly the number of 2MB contiguous memory blocks that can be used within a guest

Re: [PATCH v11 1/7] mm: adjust address_space_operations.migratepage() return code

2012-11-07 Thread Rafael Aquini
On Wed, Nov 07, 2012 at 11:56:10AM -0800, Andrew Morton wrote: On Wed, 7 Nov 2012 01:05:48 -0200 Rafael Aquini aqu...@redhat.com wrote: This patch introduces MIGRATEPAGE_SUCCESS as the default return code for address_space_operations.migratepage() method and documents the expected

Re: [PATCH v11 5/7] virtio_balloon: introduce migration primitives to balloon pages

2012-11-07 Thread Rafael Aquini
On Wed, Nov 07, 2012 at 11:58:10AM -0800, Andrew Morton wrote: On Wed, 7 Nov 2012 01:05:52 -0200 Rafael Aquini aqu...@redhat.com wrote: Memory fragmentation introduced by ballooning might reduce significantly the number of 2MB contiguous memory blocks that can be used within a guest

Re: [PATCH v11 5/7] virtio_balloon: introduce migration primitives to balloon pages

2012-11-07 Thread Rafael Aquini
On Wed, Nov 07, 2012 at 04:11:46PM -0800, Andrew Morton wrote: On Thu, 08 Nov 2012 09:32:18 +1030 Rusty Russell ru...@rustcorp.com.au wrote: Rafael Aquini aqu...@redhat.com writes: + * virtballoon_migratepage - perform the balloon page migration on behalf

Re: [PATCH v11 5/7] virtio_balloon: introduce migration primitives to balloon pages

2012-11-07 Thread Rafael Aquini
On Thu, Nov 08, 2012 at 09:32:18AM +1030, Rusty Russell wrote: The first one can be delayed, the second one can be delayed if the host didn't ask for VIRTIO_BALLOON_F_MUST_TELL_HOST (qemu doesn't). We could implement a proper request queue for these, and return -EAGAIN if the queue fills.

[PATCH v11 1/7] mm: adjust address_space_operations.migratepage() return code

2012-11-06 Thread Rafael Aquini
This patch introduces MIGRATEPAGE_SUCCESS as the default return code for address_space_operations.migratepage() method and documents the expected return code for the same method in failure cases. Signed-off-by: Rafael Aquini aqu...@redhat.com --- fs/hugetlbfs/inode.c| 4 ++-- include/linux

[PATCH v11 0/7] make balloon pages movable by compaction

2012-11-06 Thread Rafael Aquini
747 499 Compaction success 244 145 Compaction failures 503 354 Compaction pages moved 370888 474837 Compaction move failure 77378 65259 ===END stress-highalloc Rafael Aquini (7): mm: adjust

[PATCH v11 5/7] virtio_balloon: introduce migration primitives to balloon pages

2012-11-06 Thread Rafael Aquini
providing protection against concurrent access introduced by parallel memory migration threads. - balloon_lock (mutex) : synchronizes the access demand to elements of struct virtio_balloon and its queue operations; Signed-off-by: Rafael Aquini aqu...@redhat.com --- drivers

[PATCH v11 6/7] mm: introduce putback_movable_pages()

2012-11-06 Thread Rafael Aquini
putback_movable_pages() to properly cope with cases where the isolated pageset contains ballooned pages and LRU pages, thus fixing the mentioned inelegant hack around putback_lru_pages(). Signed-off-by: Rafael Aquini aqu...@redhat.com --- include/linux/migrate.h | 2 ++ mm/compaction.c

[PATCH v11 7/7] mm: add vm event counters for balloon pages compaction

2012-11-06 Thread Rafael Aquini
This patch introduces a new set of vm event counters to keep track of ballooned pages compaction activity. Signed-off-by: Rafael Aquini aqu...@redhat.com --- drivers/virtio/virtio_balloon.c | 1 + include/linux/vm_event_item.h | 8 +++- mm/balloon_compaction.c | 2 ++ mm

Re: [PATCH v10 1/5] mm: introduce a common interface for balloon pages mobility

2012-09-25 Thread Rafael Aquini
On Tue, Sep 25, 2012 at 03:05:49AM +0200, Michael S. Tsirkin wrote: If these are all under page lock these barriers just confuse things, because they are almost never enough by themselves. So in that case it would be better to drop them and document usage as you are going to. Would the

Re: [PATCH v10 3/5] virtio_balloon: introduce migration primitives to balloon pages

2012-09-25 Thread Rafael Aquini
On Tue, Sep 25, 2012 at 02:40:24AM +0200, Michael S. Tsirkin wrote: @@ -139,9 +158,15 @@ static void fill_balloon(struct virtio_balloon *vb, size_t num) break; } set_page_pfns(vb-pfns + vb-num_pfns, page); - vb-num_pages +=

Re: [PATCH v10 3/5] virtio_balloon: introduce migration primitives to balloon pages

2012-09-18 Thread Rafael Aquini
On Mon, Sep 17, 2012 at 03:15:52PM -0700, Andrew Morton wrote: + /* Number of balloon pages isolated from 'pages' list for compaction */ + unsigned int num_isolated_pages; Is it utterly inconceivable that this counter could exceed 4G, ever? /* Number of balloon pages we've told

Re: [PATCH v10 1/5] mm: introduce a common interface for balloon pages mobility

2012-09-18 Thread Rafael Aquini
On Mon, Sep 17, 2012 at 03:15:43PM -0700, Andrew Morton wrote: +/* return code to identify when a ballooned page has been migrated */ +#define BALLOON_MIGRATION_RETURN 0xba1100 I didn't really spend enough time to work out why this was done this way, but I know a hack when I see one!

[PATCH v10 0/5] make balloon pages movable by compaction

2012-09-17 Thread Rafael Aquini
1378 968 Compaction success 406 595 Compaction failures 972 373 Compaction pages moved 3104073 1790932 Compaction move failure 92713 41252 ===END stress-highalloc Rafael Aquini (5): mm: introduce a common interface

[PATCH v10 1/5] mm: introduce a common interface for balloon pages mobility

2012-09-17 Thread Rafael Aquini
. This patch introduces a common interface to help a balloon driver on making its page set movable to compaction, and thus allowing the system to better leverage the compation efforts on memory defragmentation. Signed-off-by: Rafael Aquini aqu...@redhat.com --- include/linux/balloon_compaction.h | 147

[PATCH v10 2/5] mm: introduce compaction and migration for ballooned pages

2012-09-17 Thread Rafael Aquini
. This patch introduces the helper functions as well as the necessary changes to teach compaction and migration bits how to cope with pages which are part of a guest memory balloon, in order to make them movable by memory compaction procedures. Signed-off-by: Rafael Aquini aqu...@redhat.com --- mm

[PATCH v10 4/5] mm: introduce putback_movable_pages()

2012-09-17 Thread Rafael Aquini
putback_movable_pages() to properly cope with cases where the isolated pageset contains ballooned pages and LRU pages, thus fixing the mentioned inelegant hack around putback_lru_pages(). Signed-off-by: Rafael Aquini aqu...@redhat.com --- include/linux/migrate.h | 2 ++ mm/compaction.c

[PATCH v10 3/5] virtio_balloon: introduce migration primitives to balloon pages

2012-09-17 Thread Rafael Aquini
bookmarking elements (list and atomic counters) against the potential memory compaction concurrency; Signed-off-by: Rafael Aquini aqu...@redhat.com --- drivers/virtio/virtio_balloon.c | 305 +--- 1 file changed

[PATCH v10 5/5] mm: add vm event counters for balloon pages compaction

2012-09-17 Thread Rafael Aquini
This patch introduces a new set of vm event counters to keep track of ballooned pages compaction activity. Signed-off-by: Rafael Aquini aqu...@redhat.com --- drivers/virtio/virtio_balloon.c | 1 + include/linux/vm_event_item.h | 8 +++- mm/balloon_compaction.c | 2 ++ mm

Re: [PATCH v9 3/5] virtio_balloon: introduce migration primitives to balloon pages

2012-08-28 Thread Rafael Aquini
On Tue, Aug 28, 2012 at 06:54:10PM +0300, Michael S. Tsirkin wrote: On Mon, Aug 27, 2012 at 04:47:13PM -0300, Rafael Aquini wrote: On Sun, Aug 26, 2012 at 10:42:44AM +0300, Michael S. Tsirkin wrote: Reading two atomics and doing math? Result can even be negative. I did not look at use

Re: [PATCH v9 3/5] virtio_balloon: introduce migration primitives to balloon pages

2012-08-27 Thread Rafael Aquini
On Sun, Aug 26, 2012 at 10:42:44AM +0300, Michael S. Tsirkin wrote: Reading two atomics and doing math? Result can even be negative. I did not look at use closely but it looks suspicious. Doc on atomic_read says: The read is atomic in that the return value is guaranteed to be one of the

Re: [PATCH v9 0/5] make balloon pages movable by compaction

2012-08-27 Thread Rafael Aquini
On Sun, Aug 26, 2012 at 06:44:23PM +0300, Michael S. Tsirkin wrote: I am simply asking how was this patchset tested. It would be nice to have this info in commit log. Since this is an optimization patch it is strange to see one with no numbers at all. For example, you probably run some

[PATCH v9 2/5] mm: introduce compaction and migration for ballooned pages

2012-08-24 Thread Rafael Aquini
. This patch introduces the helper functions as well as the necessary changes to teach compaction and migration bits how to cope with pages which are part of a guest memory balloon, in order to make them movable by memory compaction procedures. Signed-off-by: Rafael Aquini aqu...@redhat.com --- mm

[PATCH v9 0/5] make balloon pages movable by compaction

2012-08-24 Thread Rafael Aquini
aware of ballooned pages and allow memory balloon pages become movable within a guest, thus avoiding the aforementioned fragmentation issue Rafael Aquini (5): mm: introduce a common interface for balloon pages mobility mm: introduce compaction and migration for ballooned pages virtio_balloon

[PATCH v9 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-24 Thread Rafael Aquini
. This patch introduces a common interface to help a balloon driver on making its page set movable to compaction, and thus allowing the system to better leverage the compation efforts on memory defragmentation. Signed-off-by: Rafael Aquini aqu...@redhat.com --- include/linux/balloon_compaction.h | 137

[PATCH v9 4/5] mm: introduce putback_movable_pages()

2012-08-24 Thread Rafael Aquini
putback_movable_pages() to properly cope with cases where the isolated pageset contains ballooned pages and LRU pages, thus fixing the mentioned inelegant hack around putback_lru_pages(). Signed-off-by: Rafael Aquini aqu...@redhat.com --- include/linux/migrate.h | 2 ++ mm/compaction.c

[PATCH v9 3/5] virtio_balloon: introduce migration primitives to balloon pages

2012-08-24 Thread Rafael Aquini
queue operations; - pages_lock (spinlock): special protection to balloon's pages bookmarking elements (list and atomic counters) against the potential memory compaction concurrency; Signed-off-by: Rafael Aquini aqu...@redhat.com --- drivers

[PATCH v9 5/5] mm: add vm event counters for balloon pages compaction

2012-08-24 Thread Rafael Aquini
This patch introduces a new set of vm event counters to keep track of ballooned pages compaction activity. Signed-off-by: Rafael Aquini aqu...@redhat.com --- drivers/virtio/virtio_balloon.c | 1 + include/linux/vm_event_item.h | 8 +++- mm/balloon_compaction.c | 2 ++ mm

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 01:01:07PM +0300, Michael S. Tsirkin wrote: So, when remove_common() calls leak_balloon() looping on vb-num_pages, that won't become a tight loop. The scheme was apparently working before this series, and it will remain working after it. It seems that before

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 03:34:32PM +0300, Michael S. Tsirkin wrote: So, nothing has changed here. Yes, your patch does change things: leak_balloon now might return without freeing any pages. In that case we will not be making any progress, and just spin, pinning CPU. That's a transitory

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 04:53:29PM +0300, Michael S. Tsirkin wrote: On Thu, Aug 23, 2012 at 10:06:07AM -0300, Rafael Aquini wrote: On Thu, Aug 23, 2012 at 03:34:32PM +0300, Michael S. Tsirkin wrote: So, nothing has changed here. Yes, your patch does change things: leak_balloon now

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 12:03:15PM -0400, Rik van Riel wrote: Not longer - apparently forever unless user resend the leak command. It's wrong - it should 1. not tell host if nothing was done 2. after migration finished leak and tell host Agreed. If the balloon is told to leak N pages,

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 07:25:05PM +0300, Michael S. Tsirkin wrote: On Thu, Aug 23, 2012 at 04:53:28PM +0300, Michael S. Tsirkin wrote: Basically it was very simple: we assumed page-lru was never touched for an allocated page, so it's safe to use it for internal book-keeping by the driver.

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Fri, Aug 24, 2012 at 02:36:16AM +0300, Michael S. Tsirkin wrote: On Thu, Aug 23, 2012 at 02:28:45PM -0300, Rafael Aquini wrote: On Thu, Aug 23, 2012 at 07:25:05PM +0300, Michael S. Tsirkin wrote: On Thu, Aug 23, 2012 at 04:53:28PM +0300, Michael S. Tsirkin wrote: Basically it was very

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Fri, Aug 24, 2012 at 02:36:16AM +0300, Michael S. Tsirkin wrote: I would wake it each time after adding a page, then it can stop waiting when it leaks enough. But again, it's cleaner to just keep tracking all pages, let mm hang on to them by keeping a reference. Here is a rough idea on

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 09:33:53PM -0300, Rafael Aquini wrote: On Fri, Aug 24, 2012 at 02:36:16AM +0300, Michael S. Tsirkin wrote: I would wake it each time after adding a page, then it can stop waiting when it leaks enough. But again, it's cleaner to just keep tracking all pages, let mm

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rafael Aquini
On Thu, Aug 23, 2012 at 09:38:48PM -0300, Rafael Aquini wrote: On Thu, Aug 23, 2012 at 09:33:53PM -0300, Rafael Aquini wrote: On Fri, Aug 24, 2012 at 02:36:16AM +0300, Michael S. Tsirkin wrote: I would wake it each time after adding a page, then it can stop waiting when it leaks enough

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-22 Thread Rafael Aquini
On Wed, Aug 22, 2012 at 12:33:17PM +0300, Michael S. Tsirkin wrote: Hmm, so this will busy wait which is unelegant. We need some event IMO. No, it does not busy wait. leak_balloon() is mutual exclusive with migration steps, so for the case we have one racing against the other, we really want

[PATCH v8 0/5] make balloon pages movable by compaction

2012-08-21 Thread Rafael Aquini
aware of ballooned pages and allow memory balloon pages become movable within a guest, thus avoiding the aforementioned fragmentation issue Rafael Aquini (5): mm: introduce a common interface for balloon pages mobility mm: introduce compaction and migration for ballooned pages virtio_balloon

[PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
. This patch introduces a common interface to help a balloon driver on making its page set movable to compaction, and thus allowing the system to better leverage the compation efforts on memory defragmentation. Signed-off-by: Rafael Aquini aqu...@redhat.com --- include/linux/balloon_compaction.h | 113

[PATCH v8 3/5] virtio_balloon: introduce migration primitives to balloon pages

2012-08-21 Thread Rafael Aquini
against concurrent list handling operations; - virtio_baloon-pages list handling sync by RCU operations; Signed-off-by: Rafael Aquini aqu...@redhat.com --- drivers/virtio/virtio_balloon.c | 210 +--- 1 file changed, 199 insertions(+), 11

[PATCH v8 2/5] mm: introduce compaction and migration for ballooned pages

2012-08-21 Thread Rafael Aquini
. This patch introduces the helper functions as well as the necessary changes to teach compaction and migration bits how to cope with pages which are part of a guest memory balloon, in order to make them movable by memory compaction procedures. Signed-off-by: Rafael Aquini aqu...@redhat.com --- mm

[PATCH v8 4/5] mm: introduce putback_movable_pages()

2012-08-21 Thread Rafael Aquini
putback_movable_pages() to properly cope with cases where the isolated pageset contains ballooned pages and LRU pages, thus fixing the mentioned inelegant hack around putback_lru_pages(). Signed-off-by: Rafael Aquini aqu...@redhat.com --- include/linux/migrate.h | 2 ++ mm/compaction.c

[PATCH v8 5/5] mm: add vm event counters for balloon pages compaction

2012-08-21 Thread Rafael Aquini
This patch introduces a new set of vm event counters to keep track of ballooned pages compaction activity. Signed-off-by: Rafael Aquini aqu...@redhat.com --- drivers/virtio/virtio_balloon.c | 2 ++ include/linux/vm_event_item.h | 8 +++- mm/balloon_compaction.c | 6 -- mm

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 06:41:42PM +0300, Michael S. Tsirkin wrote: On Tue, Aug 21, 2012 at 05:16:06PM +0200, Peter Zijlstra wrote: On Tue, 2012-08-21 at 16:52 +0300, Michael S. Tsirkin wrote: + rcu_read_lock(); + mapping = rcu_dereference(page-mapping); +

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 04:52:23PM +0300, Michael S. Tsirkin wrote: + * address_space_operations utilized methods for ballooned pages: + * .migratepage- used to perform balloon's page migration (as is) + * .launder_page - used to isolate a page from balloon's page list + *

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 09:24:32AM -0700, Paul E. McKenney wrote: On Tue, Aug 21, 2012 at 05:20:11PM +0200, Peter Zijlstra wrote: On Tue, 2012-08-21 at 09:47 -0300, Rafael Aquini wrote: + mapping = rcu_access_pointer(page-mapping); + if (mapping) + mapping

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 10:13:30PM +0300, Michael S. Tsirkin wrote: I believe rcu_dereference_protected() is what I want/need here, since this code is always called for pages which we hold locked (PG_locked bit). It would only help if we locked the page while updating the mapping, as

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 10:16:12PM +0300, Michael S. Tsirkin wrote: On Tue, Aug 21, 2012 at 02:55:03PM -0300, Rafael Aquini wrote: On Tue, Aug 21, 2012 at 04:52:23PM +0300, Michael S. Tsirkin wrote: + * address_space_operations utilized methods for ballooned pages: + * .migratepage

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Tue, Aug 21, 2012 at 10:30:31PM +0300, Michael S. Tsirkin wrote: On Tue, Aug 21, 2012 at 04:23:58PM -0300, Rafael Aquini wrote: On Tue, Aug 21, 2012 at 10:13:30PM +0300, Michael S. Tsirkin wrote: I believe rcu_dereference_protected() is what I want/need here, since this code

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-21 Thread Rafael Aquini
On Wed, Aug 22, 2012 at 03:07:41AM +0300, Michael S. Tsirkin wrote: On Tue, Aug 21, 2012 at 05:45:56PM -0300, Rafael Aquini wrote: On Tue, Aug 21, 2012 at 10:30:31PM +0300, Michael S. Tsirkin wrote: On Tue, Aug 21, 2012 at 04:23:58PM -0300, Rafael Aquini wrote: On Tue, Aug 21, 2012 at 10

Re: [PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-15 Thread Rafael Aquini
On Tue, Aug 14, 2012 at 10:31:09PM +0300, Michael S. Tsirkin wrote: now CPU1 executes the next instruction: } which would normally return to function's caller, but it has been overwritten by CPU2 so we get corruption. No? At the point CPU2 is unloading the module,

Re: [PATCH v7 1/4] mm: introduce compaction and migration for virtio ballooned pages

2012-08-14 Thread Rafael Aquini
On Mon, Aug 13, 2012 at 11:26:19AM +0300, Michael S. Tsirkin wrote: +static inline bool movable_balloon_page(struct page *page) +{ + return (page-mapping page-mapping == balloon_mapping); I am guessing this needs smp_read_barrier_depends, and maybe ACCESS_ONCE ... I'm curious about

Re: [PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-14 Thread Rafael Aquini
On Mon, Aug 13, 2012 at 11:41:23AM +0300, Michael S. Tsirkin wrote: @@ -141,7 +151,10 @@ static void fill_balloon(struct virtio_balloon *vb, size_t num) set_page_pfns(vb-pfns + vb-num_pfns, page); vb-num_pages += VIRTIO_BALLOON_PAGES_PER_PAGE;

Re: [PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-14 Thread Rafael Aquini
On Tue, Aug 14, 2012 at 11:33:20AM +0300, Michael S. Tsirkin wrote: On Tue, Aug 14, 2012 at 09:29:49AM +0930, Rusty Russell wrote: On Mon, 13 Aug 2012 11:41:23 +0300, Michael S. Tsirkin m...@redhat.com wrote: On Fri, Aug 10, 2012 at 02:55:15PM -0300, Rafael Aquini wrote

Re: [PATCH v7 1/4] mm: introduce compaction and migration for virtio ballooned pages

2012-08-14 Thread Rafael Aquini
On Tue, Aug 14, 2012 at 10:35:25PM +0300, Michael S. Tsirkin wrote: +/* __isolate_lru_page() counterpart for a ballooned page */ +bool isolate_balloon_page(struct page *page) +{ + if (WARN_ON(!movable_balloon_page(page))) Looks like this actually can happen if the page

Re: [PATCH v7 1/4] mm: introduce compaction and migration for virtio ballooned pages

2012-08-14 Thread Rafael Aquini
On Tue, Aug 14, 2012 at 10:48:37PM +0300, Michael S. Tsirkin wrote: E.g. kvm can emulate hyperv so it could in theory have hyperv balloon. This is mm stuff it is best not to tie it to specific drivers. But of course I agree this is not top priority, no need to block submission on this,

Re: [PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-14 Thread Rafael Aquini
On Tue, Aug 14, 2012 at 10:59:16PM +0300, Michael S. Tsirkin wrote: What if there is more than one balloon device? Is it possible to load this driver twice, or are you foreseeing a future case where this driver will be able to manage several distinct memory balloons for

Re: [PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-14 Thread Rafael Aquini
On Tue, Aug 14, 2012 at 11:24:01PM +0300, Michael S. Tsirkin wrote: On Tue, Aug 14, 2012 at 05:08:31PM -0300, Rafael Aquini wrote: On Tue, Aug 14, 2012 at 10:59:16PM +0300, Michael S. Tsirkin wrote: What if there is more than one balloon device? Is it possible to load

Re: [PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-14 Thread Rafael Aquini
On Tue, Aug 14, 2012 at 11:49:06PM +0300, Michael S. Tsirkin wrote: On Tue, Aug 14, 2012 at 05:29:50PM -0300, Rafael Aquini wrote: On Tue, Aug 14, 2012 at 11:24:01PM +0300, Michael S. Tsirkin wrote: On Tue, Aug 14, 2012 at 05:08:31PM -0300, Rafael Aquini wrote: On Tue, Aug 14, 2012 at 10

[PATCH v7 1/4] mm: introduce compaction and migration for virtio ballooned pages

2012-08-10 Thread Rafael Aquini
. This patch introduces the helper functions as well as the necessary changes to teach compaction and migration bits how to cope with pages which are part of a guest memory balloon, in order to make them movable by memory compaction procedures. Signed-off-by: Rafael Aquini aqu...@redhat.com --- include/linux

[PATCH v7 0/4] make balloon pages movable by compaction

2012-08-10 Thread Rafael Aquini
aware of ballooned pages and allow memory balloon pages become movable within a guest, thus avoiding the aforementioned fragmentation issue Rafael Aquini (4): mm: introduce compaction and migration for virtio ballooned pages virtio_balloon: introduce migration primitives to balloon pages mm

[PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-10 Thread Rafael Aquini
against concurrent list handling operations; Signed-off-by: Rafael Aquini aqu...@redhat.com --- drivers/virtio/virtio_balloon.c | 138 +--- include/linux/virtio_balloon.h | 4 ++ 2 files changed, 134 insertions(+), 8 deletions(-) diff

[PATCH v7 3/4] mm: introduce putback_movable_pages()

2012-08-10 Thread Rafael Aquini
putback_movable_pages() to properly cope with cases where the isolated pageset contains ballooned pages and LRU pages, thus fixing the mentioned inelegant hack around putback_lru_pages(). Signed-off-by: Rafael Aquini aqu...@redhat.com --- include/linux/migrate.h | 2 ++ mm/compaction.c

Re: [PATCH v6 1/3] mm: introduce compaction and migration for virtio ballooned pages

2012-08-09 Thread Rafael Aquini
On Thu, Aug 09, 2012 at 10:00:19AM +0100, Mel Gorman wrote: On Wed, Aug 08, 2012 at 07:53:19PM -0300, Rafael Aquini wrote: Memory fragmentation introduced by ballooning might reduce significantly the number of 2MB contiguous memory blocks that can be used within a guest, thus imposing

[PATCH v6 1/3] mm: introduce compaction and migration for virtio ballooned pages

2012-08-08 Thread Rafael Aquini
. This patch introduces the helper functions as well as the necessary changes to teach compaction and migration bits how to cope with pages which are part of a guest memory balloon, in order to make them movable by memory compaction procedures. Signed-off-by: Rafael Aquini aqu...@redhat.com --- include/linux

[PATCH v6 3/3] mm: add vm event counters for balloon pages compaction

2012-08-08 Thread Rafael Aquini
This patch is only for testing report purposes and shall be dropped in case of the rest of this patchset getting accepted for merging. Signed-off-by: Rafael Aquini aqu...@redhat.com --- drivers/virtio/virtio_balloon.c | 1 + include/linux/vm_event_item.h | 2 ++ mm/compaction.c

[PATCH v6 0/3] make balloon pages movable by compaction

2012-08-08 Thread Rafael Aquini
aware of ballooned pages and allow memory balloon pages become movable within a guest, thus avoiding the aforementioned fragmentation issue Rafael Aquini (3): mm: introduce compaction and migration for virtio ballooned pages virtio_balloon: introduce migration primitives to balloon pages mm: add

[PATCH v6 2/3] virtio_balloon: introduce migration primitives to balloon pages

2012-08-08 Thread Rafael Aquini
against concurrent list handling operations; Signed-off-by: Rafael Aquini aqu...@redhat.com --- drivers/virtio/virtio_balloon.c | 138 +--- include/linux/virtio_balloon.h | 4 ++ 2 files changed, 134 insertions(+), 8 deletions(-) diff

[PATCH v5 2/3] virtio_balloon: introduce migration primitives to balloon pages

2012-08-06 Thread Rafael Aquini
against concurrent list handling operations; Signed-off-by: Rafael Aquini aqu...@redhat.com --- drivers/virtio/virtio_balloon.c | 138 +--- include/linux/virtio_balloon.h | 4 ++ 2 files changed, 134 insertions(+), 8 deletions(-) diff

[PATCH v5 1/3] mm: introduce compaction and migration for virtio ballooned pages

2012-08-06 Thread Rafael Aquini
. This patch introduces the helper functions as well as the necessary changes to teach compaction and migration bits how to cope with pages which are part of a guest memory balloon, in order to make them movable by memory compaction procedures. Signed-off-by: Rafael Aquini aqu...@redhat.com --- include/linux

[PATCH v5 0/3] make balloon pages movable by compaction

2012-08-06 Thread Rafael Aquini
aware of ballooned pages and allow memory balloon pages become movable within a guest, thus avoiding the aforementioned fragmentation issue Rafael Aquini (3): mm: introduce compaction and migration for virtio ballooned pages virtio_balloon: introduce migration primitives to balloon pages mm: add

  1   2   >