[PATCH v18 09/10] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

the reporting in advance by sending the stop cmd id to the guest via the configuration register. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Michal Hocko <mho...@kernel.org

[PATCH v18 09/10] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

the reporting in advance by sending the stop cmd id to the guest via the configuration register. Signed-off-by: Wei Wang Signed-off-by: Liang Li Cc: Michael S. Tsirkin Cc: Michal Hocko --- drivers/virtio/virtio_balloon.c | 202 +--- include/uapi/linux

[PATCH v18 05/10] xbitmap: add more operations

to help searching. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Matthew Wilcox <mawil...@microsoft.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Michal Hocko <mho...@kernel.org> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Tetsuo Handa <penguin-ker.

[PATCH v18 05/10] xbitmap: add more operations

to help searching. Signed-off-by: Wei Wang Cc: Matthew Wilcox Cc: Andrew Morton Cc: Michal Hocko Cc: Michael S. Tsirkin Cc: Tetsuo Handa Suggested-by: Matthew Wilcox --- include/linux/xbitmap.h | 8 +- lib/xbitmap.c| 180

[PATCH v18 08/10] mm: support reporting free page blocks

ges but are written after the report function returns will be captured by the hypervisor, and they will be added to the next round of memory transfer. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michal Hocko <mho...@kernel.org> C

[PATCH v18 08/10] mm: support reporting free page blocks

ges but are written after the report function returns will be captured by the hypervisor, and they will be added to the next round of memory transfer. Signed-off-by: Wei Wang Signed-off-by: Liang Li Cc: Michal Hocko Cc: Michael S. Tsirkin Acked-by: Michal Hocko --- include/linux/mm.h | 6

[PATCH v18 10/10] virtio-balloon: don't report free pages when page poisoning is enabled

skips the reporting of free pages in the above case. Reported-by: Michael S. Tsirkin <m...@redhat.com> Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Michal Hocko <mho...@suse.com> --- drivers/virtio/virtio_balloon.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) di

[PATCH v18 10/10] virtio-balloon: don't report free pages when page poisoning is enabled

skips the reporting of free pages in the above case. Reported-by: Michael S. Tsirkin Signed-off-by: Wei Wang Cc: Michal Hocko --- drivers/virtio/virtio_balloon.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio

[PATCH v18 04/10] xbitmap: potential improvement

x_tree_preload() after kmalloc() and returns a boolean to indicate the success or failure. Also, add the __must_check annotation to xb_preload for prudence purpose. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Matthew Wilcox <mawil...@microsoft.com> Cc: Andrew Morton <a...@

[PATCH v18 04/10] xbitmap: potential improvement

x_tree_preload() after kmalloc() and returns a boolean to indicate the success or failure. Also, add the __must_check annotation to xb_preload for prudence purpose. Signed-off-by: Wei Wang Cc: Matthew Wilcox Cc: Andrew Morton Cc: Michal Hocko Cc: Michael S. Tsirkin Cc: Tetsuo Handa ---

[PATCH v18 02/10] radix tree test suite: remove ARRAY_SIZE to avoid redefinition

ARRAY_SIZE() has been defined in include/linux/kernel.h, and "make" complains a warning of redefinition of ARRAY_SIZE() in testing/radix/linux/kernel.h. So, remove ARRAY_SIZE() from there. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Matthew Wilcox <mawil...@microsoft.com

[PATCH v18 02/10] radix tree test suite: remove ARRAY_SIZE to avoid redefinition

ARRAY_SIZE() has been defined in include/linux/kernel.h, and "make" complains a warning of redefinition of ARRAY_SIZE() in testing/radix/linux/kernel.h. So, remove ARRAY_SIZE() from there. Signed-off-by: Wei Wang Cc: Matthew Wilcox Cc: Andrew Morton --- tools/testing/radix-tree/linu

Re: [PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

On 11/14/2017 01:32 AM, Michael S. Tsirkin wrote: You should Cc Nitesh who is working on a related feature. OK, I'll do. We have two more issues which haven't been discussed yet, please have a check below. On Mon, Nov 13, 2017 at 06:34:48PM +0800, Wei Wang wrote: Ping for comments

Re: [PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

On 11/14/2017 01:32 AM, Michael S. Tsirkin wrote: You should Cc Nitesh who is working on a related feature. OK, I'll do. We have two more issues which haven't been discussed yet, please have a check below. On Mon, Nov 13, 2017 at 06:34:48PM +0800, Wei Wang wrote: Ping for comments

Re: [virtio-dev] Re: [PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

On 11/17/2017 07:35 PM, Wei Wang wrote: On 11/16/2017 09:27 PM, Wei Wang wrote: On 11/16/2017 04:32 AM, Michael S. Tsirkin wrote: On Fri, Nov 03, 2017 at 04:13:06PM +0800, Wei Wang wrote: Negotiation of the VIRTIO_BALLOON_F_FREE_PAGE_VQ feature indicates the support of reporting hints

Re: [virtio-dev] Re: [PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

On 11/17/2017 07:35 PM, Wei Wang wrote: On 11/16/2017 09:27 PM, Wei Wang wrote: On 11/16/2017 04:32 AM, Michael S. Tsirkin wrote: On Fri, Nov 03, 2017 at 04:13:06PM +0800, Wei Wang wrote: Negotiation of the VIRTIO_BALLOON_F_FREE_PAGE_VQ feature indicates the support of reporting hints

Re: [virtio-dev] Re: [PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

On 11/16/2017 09:27 PM, Wei Wang wrote: On 11/16/2017 04:32 AM, Michael S. Tsirkin wrote: On Fri, Nov 03, 2017 at 04:13:06PM +0800, Wei Wang wrote: Negotiation of the VIRTIO_BALLOON_F_FREE_PAGE_VQ feature indicates the support of reporting hints of guest free pages to the host via virtio

Re: [virtio-dev] Re: [PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

On 11/16/2017 09:27 PM, Wei Wang wrote: On 11/16/2017 04:32 AM, Michael S. Tsirkin wrote: On Fri, Nov 03, 2017 at 04:13:06PM +0800, Wei Wang wrote: Negotiation of the VIRTIO_BALLOON_F_FREE_PAGE_VQ feature indicates the support of reporting hints of guest free pages to the host via virtio

Re: [virtio-dev] Re: [PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

On 11/16/2017 04:32 AM, Michael S. Tsirkin wrote: On Fri, Nov 03, 2017 at 04:13:06PM +0800, Wei Wang wrote: Negotiation of the VIRTIO_BALLOON_F_FREE_PAGE_VQ feature indicates the support of reporting hints of guest free pages to the host via virtio-balloon. The host requests the guest to report

Re: [virtio-dev] Re: [PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

On 11/16/2017 04:32 AM, Michael S. Tsirkin wrote: On Fri, Nov 03, 2017 at 04:13:06PM +0800, Wei Wang wrote: Negotiation of the VIRTIO_BALLOON_F_FREE_PAGE_VQ feature indicates the support of reporting hints of guest free pages to the host via virtio-balloon. The host requests the guest to report

Re: [PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

On 11/15/2017 09:26 PM, Michael S. Tsirkin wrote: On Wed, Nov 15, 2017 at 11:47:58AM +0800, Wei Wang wrote: On 11/15/2017 05:21 AM, Michael S. Tsirkin wrote: On Tue, Nov 14, 2017 at 08:02:03PM +0800, Wei Wang wrote: On 11/14/2017 01:32 AM, Michael S. Tsirkin wrote: - guest2host_cmd: written

Re: [PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

On 11/15/2017 09:26 PM, Michael S. Tsirkin wrote: On Wed, Nov 15, 2017 at 11:47:58AM +0800, Wei Wang wrote: On 11/15/2017 05:21 AM, Michael S. Tsirkin wrote: On Tue, Nov 14, 2017 at 08:02:03PM +0800, Wei Wang wrote: On 11/14/2017 01:32 AM, Michael S. Tsirkin wrote: - guest2host_cmd: written

Re: [PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

On 11/15/2017 05:21 AM, Michael S. Tsirkin wrote: On Tue, Nov 14, 2017 at 08:02:03PM +0800, Wei Wang wrote: On 11/14/2017 01:32 AM, Michael S. Tsirkin wrote: - guest2host_cmd: written by the guest to ACK to the host about the commands that have been received. The host will clear

Re: [PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

On 11/15/2017 05:21 AM, Michael S. Tsirkin wrote: On Tue, Nov 14, 2017 at 08:02:03PM +0800, Wei Wang wrote: On 11/14/2017 01:32 AM, Michael S. Tsirkin wrote: - guest2host_cmd: written by the guest to ACK to the host about the commands that have been received. The host will clear

Re: [PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

On 11/14/2017 01:32 AM, Michael S. Tsirkin wrote: - guest2host_cmd: written by the guest to ACK to the host about the commands that have been received. The host will clear the corresponding bits on the host2guest_cmd register. The guest also uses this register to send commands to the host (e.g.

Re: [PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

On 11/14/2017 01:32 AM, Michael S. Tsirkin wrote: - guest2host_cmd: written by the guest to ACK to the host about the commands that have been received. The host will clear the corresponding bits on the host2guest_cmd register. The guest also uses this register to send commands to the host (e.g.

Re: [PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

Ping for comments, thanks. On 11/03/2017 04:13 PM, Wei Wang wrote: Negotiation of the VIRTIO_BALLOON_F_FREE_PAGE_VQ feature indicates the support of reporting hints of guest free pages to the host via virtio-balloon. The host requests the guest to report the free pages by sending commands via

Re: [PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

Ping for comments, thanks. On 11/03/2017 04:13 PM, Wei Wang wrote: Negotiation of the VIRTIO_BALLOON_F_FREE_PAGE_VQ feature indicates the support of reporting hints of guest free pages to the host via virtio-balloon. The host requests the guest to report the free pages by sending commands via

Re: [PULL] virtio: last minute bugfix

On 11/08/2017 03:23 AM, Michael S. Tsirkin wrote: On Tue, Nov 07, 2017 at 08:13:10PM +0200, Michael S. Tsirkin wrote: On Tue, Nov 07, 2017 at 09:29:59AM -0800, Linus Torvalds wrote: On Tue, Nov 7, 2017 at 9:23 AM, Linus Torvalds wrote: I guess I'll take it, but

Re: [PULL] virtio: last minute bugfix

On 11/08/2017 03:23 AM, Michael S. Tsirkin wrote: On Tue, Nov 07, 2017 at 08:13:10PM +0200, Michael S. Tsirkin wrote: On Tue, Nov 07, 2017 at 09:29:59AM -0800, Linus Torvalds wrote: On Tue, Nov 7, 2017 at 9:23 AM, Linus Torvalds wrote: I guess I'll take it, but please don't do things like

Re: [PATCH v17 4/6] virtio-balloon: VIRTIO_BALLOON_F_SG

On 11/04/2017 07:28 PM, Tetsuo Handa wrote: Wei Wang wrote: On 11/03/2017 07:25 PM, Tetsuo Handa wrote: If this is inside vb->balloon_lock mutex (isn't this?), xb_set_page() must not use __GFP_DIRECT_RECLAIM allocation, for leak_balloon_sg_oom() will be blocked on vb->balloon_lock mut

Re: [PATCH v17 4/6] virtio-balloon: VIRTIO_BALLOON_F_SG

On 11/04/2017 07:28 PM, Tetsuo Handa wrote: Wei Wang wrote: On 11/03/2017 07:25 PM, Tetsuo Handa wrote: If this is inside vb->balloon_lock mutex (isn't this?), xb_set_page() must not use __GFP_DIRECT_RECLAIM allocation, for leak_balloon_sg_oom() will be blocked on vb->balloon_lock mut

Re: [PATCH v17 1/6] lib/xbitmap: Introduce xbitmap

On 11/03/2017 06:55 PM, Tetsuo Handa wrote: I'm commenting without understanding the logic. Wei Wang wrote: + +bool xb_preload(gfp_t gfp); + Want __must_check annotation, for __radix_tree_preload() is marked with __must_check annotation. By error failing to check result of xb_preload

Re: [PATCH v17 1/6] lib/xbitmap: Introduce xbitmap

On 11/03/2017 06:55 PM, Tetsuo Handa wrote: I'm commenting without understanding the logic. Wei Wang wrote: + +bool xb_preload(gfp_t gfp); + Want __must_check annotation, for __radix_tree_preload() is marked with __must_check annotation. By error failing to check result of xb_preload

Re: [PATCH v17 4/6] virtio-balloon: VIRTIO_BALLOON_F_SG

On 11/03/2017 07:25 PM, Tetsuo Handa wrote: Wei Wang wrote: @@ -164,6 +284,8 @@ static unsigned fill_balloon(struct virtio_balloon *vb, size_t num) break; } + if (use_sg && xb_set_page(vb, page, _min, _max) < 0) Isn't this leaking "

Re: [PATCH v17 4/6] virtio-balloon: VIRTIO_BALLOON_F_SG

On 11/03/2017 07:25 PM, Tetsuo Handa wrote: Wei Wang wrote: @@ -164,6 +284,8 @@ static unsigned fill_balloon(struct virtio_balloon *vb, size_t num) break; } + if (use_sg && xb_set_page(vb, page, _min, _max) < 0) Isn't this leaking "

Re: [PATCH v1 0/3] Virtio-balloon Improvement

On 10/22/2017 11:19 AM, Michael S. Tsirkin wrote: On Fri, Oct 20, 2017 at 07:54:23PM +0800, Wei Wang wrote: This patch series intends to summarize the recent contributions made by Michael S. Tsirkin, Tetsuo Handa, Michal Hocko etc. via reporting and discussing the related deadlock issues

Re: [PATCH v1 0/3] Virtio-balloon Improvement

On 10/22/2017 11:19 AM, Michael S. Tsirkin wrote: On Fri, Oct 20, 2017 at 07:54:23PM +0800, Wei Wang wrote: This patch series intends to summarize the recent contributions made by Michael S. Tsirkin, Tetsuo Handa, Michal Hocko etc. via reporting and discussing the related deadlock issues

[PATCH v17 2/6] radix tree test suite: add tests for xbitmap

From: Matthew Wilcox <mawil...@microsoft.com> Add the following tests for xbitmap: 1) single bit test: single bit set/clear/find; 2) bit range test: set/clear a range of bits and find a 0 or 1 bit in the range. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Matthew W

[PATCH v17 2/6] radix tree test suite: add tests for xbitmap

From: Matthew Wilcox Add the following tests for xbitmap: 1) single bit test: single bit set/clear/find; 2) bit range test: set/clear a range of bits and find a 0 or 1 bit in the range. Signed-off-by: Wei Wang Cc: Matthew Wilcox Cc: Andrew Morton Cc: Michael S. Tsirkin --- tools/include

[PATCH v17 3/6] mm/balloon_compaction.c: split balloon page allocation and enqueue

Michal Hocko <mho...@suse.com> Cc: Wei Wang <wei.w.w...@intel.com> Reviewed-by: Wei Wang <wei.w.w...@intel.com> --- drivers/virtio/virtio_balloon.c| 23 ++- include/linux/balloon_compaction.h | 34 +- mm/balloon_compacti

[PATCH v17 3/6] mm/balloon_compaction.c: split balloon page allocation and enqueue

ng_notifier_call_chain() leak_balloon() tries to take that balloon_lock and deadlocks Reported-by: Tetsuo Handa Signed-off-by: Michael S. Tsirkin Cc: Michal Hocko Cc: Wei Wang Reviewed-by: Wei Wang

[PATCH v17 5/6] mm: support reporting free page blocks

ges but are written after the report function returns will be captured by the hypervisor, and they will be added to the next round of memory transfer. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michal Hocko <mho...@kernel.org> C

[PATCH v17 5/6] mm: support reporting free page blocks

ges but are written after the report function returns will be captured by the hypervisor, and they will be added to the next round of memory transfer. Signed-off-by: Wei Wang Signed-off-by: Liang Li Cc: Michal Hocko Cc: Michael S. Tsirkin Acked-by: Michal Hocko --- include/linux/mm.h | 6

[PATCH v17 4/6] virtio-balloon: VIRTIO_BALLOON_F_SG

() in chunks rather than page by page. With this new feature, the above ballooning process takes ~492ms resulting in an improvement of ~88%. TODO: optimize stage 1) by allocating/freeing a chunk of pages instead of a single page each time. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Sign

[PATCH v17 4/6] virtio-balloon: VIRTIO_BALLOON_F_SG

() in chunks rather than page by page. With this new feature, the above ballooning process takes ~492ms resulting in an improvement of ~88%. TODO: optimize stage 1) by allocating/freeing a chunk of pages instead of a single page each time. Signed-off-by: Wei Wang Signed-off-by: Liang Li Suggested

[PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

finish free page reporting). - free_page_cmd_id: the sequence id of the free page report command given by the host. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Michal Hocko <mho...@ker

[PATCH v17 1/6] lib/xbitmap: Introduce xbitmap

ed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Matthew Wilcox <mawil...@microsoft.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Michal Hocko <mho...@kernel.org> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Tetsuo Handa <penguin-ker...@i-love.sakura.ne.jp> v

[PATCH v17 1/6] lib/xbitmap: Introduce xbitmap

(), xb_find_next_set_bit(), xb_find_next_zero_bit(). More possible optimizations to add in the future: 1) xb_set_bit_range: set a range of bits. 2) when searching a bit, if the bit is not found in the slot, move on to the next slot directly. 3) add Tags to help searching. Signed-off-by: Wei Wang Cc: Matthew

[PATCH v17 6/6] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ

finish free page reporting). - free_page_cmd_id: the sequence id of the free page report command given by the host. Signed-off-by: Wei Wang Signed-off-by: Liang Li Cc: Michael S. Tsirkin Cc: Michal Hocko --- drivers/virtio/virtio_balloon.c | 234 include

[PATCH v17 0/6] Virtio-balloon Enhancement

dix tree test suite: add tests for xbitmap Michael S. Tsirkin (1): mm/balloon_compaction.c: split balloon page allocation and enqueue Wei Wang (3): virtio-balloon: VIRTIO_BALLOON_F_SG mm: support reporting free page blocks virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ drivers/virtio/

[PATCH v17 0/6] Virtio-balloon Enhancement

dix tree test suite: add tests for xbitmap Michael S. Tsirkin (1): mm/balloon_compaction.c: split balloon page allocation and enqueue Wei Wang (3): virtio-balloon: VIRTIO_BALLOON_F_SG mm: support reporting free page blocks virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_VQ drivers/virtio/

Re: [PATCH] virtio_balloon: fix deadlock on OOM

Tetsuo Handa <penguin-ker...@i-love.sakura.ne.jp> Cc: Michal Hocko <mho...@suse.com> Cc: Wei Wang <wei.w.w...@intel.com> --- The "virtio-balloon enhancement" series has a dependency on this patch. Could you send out a new version soon? Or I can include it in the series if you want. Best, Wei

Re: [PATCH] virtio_balloon: fix deadlock on OOM

Tetsuo Handa Cc: Michal Hocko Cc: Wei Wang --- The "virtio-balloon enhancement" series has a dependency on this patch. Could you send out a new version soon? Or I can include it in the series if you want. Best, Wei

Re: [PATCH v1 3/3] virtio-balloon: stop inflating when OOM occurs

On 10/23/2017 01:13 AM, Michael S. Tsirkin wrote: On Fri, Oct 20, 2017 at 07:54:26PM +0800, Wei Wang wrote: This patch forces the cease of the inflating work when OOM occurs. The fundamental idea of memory ballooning is to take out some guest pages when the guest has low memory utilization, so

Re: [PATCH v1 3/3] virtio-balloon: stop inflating when OOM occurs

On 10/23/2017 01:13 AM, Michael S. Tsirkin wrote: On Fri, Oct 20, 2017 at 07:54:26PM +0800, Wei Wang wrote: This patch forces the cease of the inflating work when OOM occurs. The fundamental idea of memory ballooning is to take out some guest pages when the guest has low memory utilization, so

Re: [PATCH v1 1/3] virtio-balloon: replace the coarse-grained balloon_lock

On 10/22/2017 07:50 PM, Tetsuo Handa wrote: Wei Wang wrote: @@ -162,20 +160,20 @@ static unsigned fill_balloon(struct virtio_balloon *vb, size_t num) msleep(200); break; } - set_page_pfns(vb, vb->pfns + vb->nu

Re: [PATCH v1 1/3] virtio-balloon: replace the coarse-grained balloon_lock

On 10/22/2017 07:50 PM, Tetsuo Handa wrote: Wei Wang wrote: @@ -162,20 +160,20 @@ static unsigned fill_balloon(struct virtio_balloon *vb, size_t num) msleep(200); break; } - set_page_pfns(vb, vb->pfns + vb->nu

Re: [PATCH v1 2/3] virtio-balloon: deflate up to oom_pages on OOM

On 10/22/2017 12:11 PM, Tetsuo Handa wrote: Michael S. Tsirkin wrote: - num_freed_pages = leak_balloon(vb, oom_pages); + + /* Don't deflate more than the number of inflated pages */ + while (npages && atomic64_read(>num_pages)) + npages -= leak_balloon(vb,

Re: [PATCH v1 2/3] virtio-balloon: deflate up to oom_pages on OOM

On 10/22/2017 12:11 PM, Tetsuo Handa wrote: Michael S. Tsirkin wrote: - num_freed_pages = leak_balloon(vb, oom_pages); + + /* Don't deflate more than the number of inflated pages */ + while (npages && atomic64_read(>num_pages)) + npages -= leak_balloon(vb,

Re: [PATCH v1 1/3] virtio-balloon: replace the coarse-grained balloon_lock

On 10/22/2017 01:20 PM, Tetsuo Handa wrote: Wei Wang wrote: The balloon_lock was used to synchronize the access demand to elements of struct virtio_balloon and its queue operations (please see commit e22504296d). This prevents the concurrent run of the leak_balloon and fill_balloon functions

Re: [PATCH v1 1/3] virtio-balloon: replace the coarse-grained balloon_lock

On 10/22/2017 01:20 PM, Tetsuo Handa wrote: Wei Wang wrote: The balloon_lock was used to synchronize the access demand to elements of struct virtio_balloon and its queue operations (please see commit e22504296d). This prevents the concurrent run of the leak_balloon and fill_balloon functions

Re: [PATCH v1 0/3] Virtio-balloon Improvement

On 10/22/2017 11:19 AM, Michael S. Tsirkin wrote: On Fri, Oct 20, 2017 at 07:54:23PM +0800, Wei Wang wrote: This patch series intends to summarize the recent contributions made by Michael S. Tsirkin, Tetsuo Handa, Michal Hocko etc. via reporting and discussing the related deadlock issues

Re: [PATCH v1 0/3] Virtio-balloon Improvement

On 10/22/2017 11:19 AM, Michael S. Tsirkin wrote: On Fri, Oct 20, 2017 at 07:54:23PM +0800, Wei Wang wrote: This patch series intends to summarize the recent contributions made by Michael S. Tsirkin, Tetsuo Handa, Michal Hocko etc. via reporting and discussing the related deadlock issues

[PATCH v1 2/3] virtio-balloon: deflate up to oom_pages on OOM

The current implementation only deflates 256 pages even when a user specifies more than that via the oom_pages module param. This patch enables the deflating of up to oom_pages pages if there are enough inflated pages. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Michael S. Tsir

[PATCH v1 2/3] virtio-balloon: deflate up to oom_pages on OOM

The current implementation only deflates 256 pages even when a user specifies more than that via the oom_pages module param. This patch enables the deflating of up to oom_pages pages if there are enough inflated pages. Signed-off-by: Wei Wang Cc: Michael S. Tsirkin Cc: Michal Hocko Cc: Tetsuo

[PATCH v1 0/3] Virtio-balloon Improvement

ble processes and fall into kernel panic finally. Wei Wang (3): virtio-balloon: replace the coarse-grained balloon_lock virtio-balloon: deflate up to oom_pages on OOM virtio-balloon: stop inflating when OOM occurs drivers/virtio/virtio_balloon.c | 149

[PATCH v1 1/3] virtio-balloon: replace the coarse-grained balloon_lock

of the shared data (i.e. vb->pnfs, vb->num_pfns). This enables leak_balloon and fill_balloon to run concurrently and solves the deadlock issue. Reported-by: Tetsuo Handa <penguin-ker...@i-love.sakura.ne.jp> Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Michael S. Tsirkin &l

[PATCH v1 3/3] virtio-balloon: stop inflating when OOM occurs

requests is sent to guest when the guest is still under memory pressure, still no pages will be inflated. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Tetsuo Handa <penguin-ker...@i-love.sakura.ne.jp> Cc: Michal Hocko <mho...@ker

[PATCH v1 1/3] virtio-balloon: replace the coarse-grained balloon_lock

of the shared data (i.e. vb->pnfs, vb->num_pfns). This enables leak_balloon and fill_balloon to run concurrently and solves the deadlock issue. Reported-by: Tetsuo Handa Signed-off-by: Wei Wang Cc: Michael S. Tsirkin Cc: Tetsuo Handa Cc: Michal Hocko --- drivers/virtio/virtio_balloon.c

[PATCH v1 3/3] virtio-balloon: stop inflating when OOM occurs

requests is sent to guest when the guest is still under memory pressure, still no pages will be inflated. Signed-off-by: Wei Wang Cc: Michael S. Tsirkin Cc: Tetsuo Handa Cc: Michal Hocko --- drivers/virtio/virtio_balloon.c | 33 + 1 file changed, 29 insertions

[PATCH v1 0/3] Virtio-balloon Improvement

ble processes and fall into kernel panic finally. Wei Wang (3): virtio-balloon: replace the coarse-grained balloon_lock virtio-balloon: deflate up to oom_pages on OOM virtio-balloon: stop inflating when OOM occurs drivers/virtio/virtio_balloon.c | 149

Re: [PATCH v16 5/5] virtio-balloon: VIRTIO_BALLOON_F_CTRL_VQ

On 10/13/2017 09:38 PM, Michael S. Tsirkin wrote: On Thu, Oct 12, 2017 at 11:54:56AM +0800, Wei Wang wrote: But I think flushing is very fragile. You will easily run into races if one of the actors gets out of sync and keeps adding data. I think adding an ID in the free vq stream is a more

Re: [PATCH v16 5/5] virtio-balloon: VIRTIO_BALLOON_F_CTRL_VQ

On 10/13/2017 09:38 PM, Michael S. Tsirkin wrote: On Thu, Oct 12, 2017 at 11:54:56AM +0800, Wei Wang wrote: But I think flushing is very fragile. You will easily run into races if one of the actors gets out of sync and keeps adding data. I think adding an ID in the free vq stream is a more

Re: [PATCH] virtio_balloon: fix deadlock on OOM

even in presence of lock contention. But Wei Wang is proposing VIRTIO_BALLOON_F_SG which will try to allocate memory, isn't he? Hopefully that can be fixed by allocating outside the lock. I think that would still have an issue even without the lock, because we can't do any memory allocation

Re: [PATCH] virtio_balloon: fix deadlock on OOM

even in presence of lock contention. But Wei Wang is proposing VIRTIO_BALLOON_F_SG which will try to allocate memory, isn't he? Hopefully that can be fixed by allocating outside the lock. I think that would still have an issue even without the lock, because we can't do any memory allocation

Re: [lkp-robot] [ipv6] 2b760fcf5c: WARNING:suspicious_RCU_usage

On Thu, Oct 12, 2017 at 7:03 PM, kernel test robot wrote: > > FYI, we noticed the following commit (built with gcc-6): > > commit: 2b760fcf5cfb34e8610df56d83745b2b74ae1379 ("ipv6: hook up exception > table to store dst cache") >

Re: [lkp-robot] [ipv6] 2b760fcf5c: WARNING:suspicious_RCU_usage

On Thu, Oct 12, 2017 at 7:03 PM, kernel test robot wrote: > > FYI, we noticed the following commit (built with gcc-6): > > commit: 2b760fcf5cfb34e8610df56d83745b2b74ae1379 ("ipv6: hook up exception > table to store dst cache") > https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git

Re: [PATCH v16 5/5] virtio-balloon: VIRTIO_BALLOON_F_CTRL_VQ

On 10/11/2017 09:49 PM, Michael S. Tsirkin wrote: On Wed, Oct 11, 2017 at 02:03:20PM +0800, Wei Wang wrote: On 10/10/2017 11:15 PM, Michael S. Tsirkin wrote: On Mon, Oct 02, 2017 at 04:38:01PM +, Wang, Wei W wrote: On Sunday, October 1, 2017 11:19 AM, Michael S. Tsirkin wrote: On Sat

Re: [PATCH v16 5/5] virtio-balloon: VIRTIO_BALLOON_F_CTRL_VQ

On 10/11/2017 09:49 PM, Michael S. Tsirkin wrote: On Wed, Oct 11, 2017 at 02:03:20PM +0800, Wei Wang wrote: On 10/10/2017 11:15 PM, Michael S. Tsirkin wrote: On Mon, Oct 02, 2017 at 04:38:01PM +, Wang, Wei W wrote: On Sunday, October 1, 2017 11:19 AM, Michael S. Tsirkin wrote: On Sat

Re: [PATCH v16 5/5] virtio-balloon: VIRTIO_BALLOON_F_CTRL_VQ

On 10/10/2017 11:15 PM, Michael S. Tsirkin wrote: On Mon, Oct 02, 2017 at 04:38:01PM +, Wang, Wei W wrote: On Sunday, October 1, 2017 11:19 AM, Michael S. Tsirkin wrote: On Sat, Sep 30, 2017 at 12:05:54PM +0800, Wei Wang wrote: +static void ctrlq_send_cmd(struct virtio_balloon *vb

Re: [PATCH v16 5/5] virtio-balloon: VIRTIO_BALLOON_F_CTRL_VQ

On 10/10/2017 11:15 PM, Michael S. Tsirkin wrote: On Mon, Oct 02, 2017 at 04:38:01PM +, Wang, Wei W wrote: On Sunday, October 1, 2017 11:19 AM, Michael S. Tsirkin wrote: On Sat, Sep 30, 2017 at 12:05:54PM +0800, Wei Wang wrote: +static void ctrlq_send_cmd(struct virtio_balloon *vb

Re: [PATCH v16 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

On 10/11/2017 10:26 AM, Tetsuo Handa wrote: > Wei Wang wrote: >> On 10/10/2017 09:09 PM, Tetsuo Handa wrote: >>> Wei Wang wrote: >>>>> And even if we could remove balloon_lock, you still cannot use >>>>> __GFP_DIRECT_RECLAIM at xb_set_page(). I thin

Re: [PATCH v16 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

On 10/11/2017 10:26 AM, Tetsuo Handa wrote: > Wei Wang wrote: >> On 10/10/2017 09:09 PM, Tetsuo Handa wrote: >>> Wei Wang wrote: >>>>> And even if we could remove balloon_lock, you still cannot use >>>>> __GFP_DIRECT_RECLAIM at xb_set_page(). I thin

Re: [PATCH v16 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

On 10/10/2017 09:09 PM, Tetsuo Handa wrote: Wei Wang wrote: And even if we could remove balloon_lock, you still cannot use __GFP_DIRECT_RECLAIM at xb_set_page(). I think you will need to use "whether it is safe to wait" flag from "[PATCH] virtio: avoid poss

Re: [PATCH v16 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

On 10/10/2017 09:09 PM, Tetsuo Handa wrote: Wei Wang wrote: And even if we could remove balloon_lock, you still cannot use __GFP_DIRECT_RECLAIM at xb_set_page(). I think you will need to use "whether it is safe to wait" flag from "[PATCH] virtio: avoid poss

Re: [PATCH][V2] ipv6: fix incorrect bitwise operator used on rt6i_flags

F_CACHE entry bit. > > Detected by CoverityScan, CID#1457734, #1457747 ("Wrong operator used") > > Fixes: 35732d01fe31 ("ipv6: introduce a hash table to store dst cache") > Signed-off-by: Colin Ian King <colin.k...@canonical.com> > --- Acked-by: Wei

Re: [PATCH][V2] ipv6: fix incorrect bitwise operator used on rt6i_flags

1457734, #1457747 ("Wrong operator used") > > Fixes: 35732d01fe31 ("ipv6: introduce a hash table to store dst cache") > Signed-off-by: Colin Ian King > --- Acked-by: Wei Wang > net/ipv6/route.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(

Re: [PATCH][net-next] ipv6: fix incorrect bitwise operator used on rt6i_flags

Martin KaFai Lau <ka...@fb.com> > Thanks a lot for catching this. Yes. It should have been '&' instead of '|'. Acked-by: Wei Wang <wei...@google.com> >> >> Detected by CoverityScan, CID#1457747 ("Wrong operator used") >> >> Fixes: 35732d01fe31 (&q

Re: [PATCH][net-next] ipv6: fix incorrect bitwise operator used on rt6i_flags

er suspect to me. I >> believe this is fixed by using & instead to just check the >> RTF_CACHE entry bit. > Good catch. LGTM. If rt does not have RTF_CACHE set, it should not be in the > exception table. > > Acked-by: Martin KaFai Lau > Thanks a lot for catching this

Re: [PATCH v16 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

On 10/10/2017 07:08 PM, Tetsuo Handa wrote: Wei Wang wrote: On 10/09/2017 11:20 PM, Michael S. Tsirkin wrote: On Sat, Sep 30, 2017 at 12:05:52PM +0800, Wei Wang wrote: +static inline void xb_set_page(struct virtio_balloon *vb, + struct page *page

Re: [PATCH v16 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

On 10/10/2017 07:08 PM, Tetsuo Handa wrote: Wei Wang wrote: On 10/09/2017 11:20 PM, Michael S. Tsirkin wrote: On Sat, Sep 30, 2017 at 12:05:52PM +0800, Wei Wang wrote: +static inline void xb_set_page(struct virtio_balloon *vb, + struct page *page

Re: [PATCH v16 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

On 10/09/2017 11:20 PM, Michael S. Tsirkin wrote: On Sat, Sep 30, 2017 at 12:05:52PM +0800, Wei Wang wrote: +static inline void xb_set_page(struct virtio_balloon *vb, + struct page *page, + unsigned long *pfn_min

Re: [PATCH v16 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

On 10/09/2017 11:20 PM, Michael S. Tsirkin wrote: On Sat, Sep 30, 2017 at 12:05:52PM +0800, Wei Wang wrote: +static inline void xb_set_page(struct virtio_balloon *vb, + struct page *page, + unsigned long *pfn_min

Re: [PATCH v16 0/5] Virtio-balloon Enhancement

On 10/01/2017 09:25 PM, Damian Tometzki wrote: Hello, where i can found the patch in git.kernel.org ? We don't have patches there. If you want to try this feature, you can get the qemu side draft code here: https://github.com/wei-w-wang/qemu-lm Best, Wei

Re: [PATCH v16 0/5] Virtio-balloon Enhancement

On 10/01/2017 09:25 PM, Damian Tometzki wrote: Hello, where i can found the patch in git.kernel.org ? We don't have patches there. If you want to try this feature, you can get the qemu side draft code here: https://github.com/wei-w-wang/qemu-lm Best, Wei

[PATCH v16 2/5] radix tree test suite: add tests for xbitmap

From: Matthew Wilcox <mawil...@microsoft.com> Add the following tests for xbitmap: 1) single bit test: single bit set/clear/find; 2) bit range test: set/clear a range of bits and find a 0 or 1 bit in the range. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Cc: Matthew W

[PATCH v16 2/5] radix tree test suite: add tests for xbitmap

From: Matthew Wilcox Add the following tests for xbitmap: 1) single bit test: single bit set/clear/find; 2) bit range test: set/clear a range of bits and find a 0 or 1 bit in the range. Signed-off-by: Wei Wang Cc: Matthew Wilcox Cc: Andrew Morton Cc: Michael S. Tsirkin --- tools/include

[PATCH v16 4/5] mm: support reporting free page blocks

ges but are written after the report function returns will be captured by the hypervisor, and they will be added to the next round of memory transfer. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michal Hocko <mho...@kernel.org> C

[PATCH v16 3/5] virtio-balloon: VIRTIO_BALLOON_F_SG

() in chunks rather than page by page. With this new feature, the above ballooning process takes ~492ms resulting in an improvement of ~88%. TODO: optimize stage 1) by allocating/freeing a chunk of pages instead of a single page each time. Signed-off-by: Wei Wang <wei.w.w...@intel.com> Sign

<    2   3   4   5   6   7   8   9   10   11   >