Re: [PATCH RFC] virtio_balloon: conservative balloon page shrinking

2020-02-11 Thread Tetsuo Handa
On 2020/02/10 16:27, Wang, Wei W wrote: >> Well, my comment is rather: "Do not try to reserve guest's memory. In other >> words, do not try to maintain balloons on the guest side. Since host would >> be able to cache file data on the host's cache, guests would be able to >> quickly fetch file data

RE: [PATCH RFC] virtio_balloon: conservative balloon page shrinking

2020-02-09 Thread Tetsuo Handa
Wang, Wei W wrote: > On Saturday, February 8, 2020 8:33 PM, Tetsuo Handa wrote: > > > > Is this NUMA aware? Can "node-A's NR_FILE_PAGES is already 0 and > > node-B's NR_FILE_PAGES is not 0, but allocation request which triggered this > > shrinker

Re: [PATCH RFC] virtio_balloon: conservative balloon page shrinking

2020-02-08 Thread Tetsuo Handa
On 2020/02/06 17:01, Wei Wang wrote: > There are cases that users want to shrink balloon pages after the > pagecache depleted. The conservative_shrinker lets the shrinker > shrink balloon pages when all the pagecache has been reclaimed. > > @@ -796,6 +800,10 @@ static unsigned long shrink_balloon_

Re: [PATCH v3 2/2] virtio_balloon: replace oom notifier with shrinker

2018-08-06 Thread Tetsuo Handa
On 2018/08/06 21:44, Wang, Wei W wrote: > On Monday, August 6, 2018 6:29 PM, Tetsuo Handa wrote: >> On 2018/08/06 18:56, Wei Wang wrote: >>> On 08/03/2018 08:11 PM, Tetsuo Handa wrote: >>>> On 2018/08/03 17:32, Wei Wang wrote: >>>>> +stati

Re: [PATCH v3 2/2] virtio_balloon: replace oom notifier with shrinker

2018-08-06 Thread Tetsuo Handa
On 2018/08/06 18:56, Wei Wang wrote: > On 08/03/2018 08:11 PM, Tetsuo Handa wrote: >> On 2018/08/03 17:32, Wei Wang wrote: >>> +static int virtio_balloon_register_shrinker(struct virtio_balloon *vb) >>> +{ >>> +    vb->shrinker.scan_objects = vi

Re: [PATCH v3 2/2] virtio_balloon: replace oom notifier with shrinker

2018-08-03 Thread Tetsuo Handa
On 2018/08/03 17:32, Wei Wang wrote: > +static int virtio_balloon_register_shrinker(struct virtio_balloon *vb) > +{ > + vb->shrinker.scan_objects = virtio_balloon_shrinker_scan; > + vb->shrinker.count_objects = virtio_balloon_shrinker_count; > + vb->shrinker.batch = 0; > + vb->shrin

Re: [PATCH v2 2/2] virtio_balloon: replace oom notifier with shrinker

2018-08-02 Thread Tetsuo Handa
On 2018/08/02 19:32, Wei Wang wrote: > On 08/01/2018 07:34 PM, Michal Hocko wrote: >> Do you have any numbers for how does this work in practice? > > It works in this way: for example, we can set the parameter, > balloon_pages_to_shrink, > to shrink 1GB memory once shrink scan is called. Now, we

Re: [virtio-dev] Re: [PATCH v25 2/2] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT

2018-01-26 Thread Tetsuo Handa
On 2018/01/26 12:31, Wei Wang wrote: > On 01/26/2018 10:42 AM, Michael S. Tsirkin wrote: >> On Fri, Jan 26, 2018 at 09:40:44AM +0800, Wei Wang wrote: >>> On 01/25/2018 09:49 PM, Michael S. Tsirkin wrote: On Thu, Jan 25, 2018 at 05:14:06PM +0800, Wei Wang wrote: > >>> The controversy is t

Re: [PATCH v24 2/2] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT

2018-01-25 Thread Tetsuo Handa
On 2018/01/25 12:32, Wei Wang wrote: > On 01/25/2018 01:15 AM, Michael S. Tsirkin wrote: >> On Wed, Jan 24, 2018 at 06:42:42PM +0800, Wei Wang wrote: >> + >> +static void report_free_page_func(struct work_struct *work) >> +{ >> +struct virtio_balloon *vb; >> +unsigned long flags; >> + >> +

Re: [PATCH v17 4/6] virtio-balloon: VIRTIO_BALLOON_F_SG

2017-11-04 Thread Tetsuo Handa
Wei Wang wrote: > On 11/03/2017 07:25 PM, Tetsuo Handa wrote: > >> @@ -184,8 +307,12 @@ static unsigned fill_balloon(struct virtio_balloon > >> *vb, size_t num) > >> > >>num_allocated_pages = vb->num_pfns; > >>

Re: [PATCH v17 4/6] virtio-balloon: VIRTIO_BALLOON_F_SG

2017-11-03 Thread Tetsuo Handa
Wei Wang wrote: > @@ -164,6 +284,8 @@ static unsigned fill_balloon(struct virtio_balloon *vb, > size_t num) > break; > } > > + if (use_sg && xb_set_page(vb, page, &pfn_min, &pfn_max) < 0) Isn't this leaking "page" ? > + break;

Re: [PATCH v17 3/6] mm/balloon_compaction.c: split balloon page allocation and enqueue

2017-11-03 Thread Tetsuo Handa
Wei Wang wrote: > Here's a detailed analysis of the deadlock by Tetsuo Handa: > > In leak_balloon(), mutex_lock(&vb->balloon_lock) is called in order to > serialize against fill_balloon(). But in fill_balloon(), > alloc_page(GFP_HIGHUSER[_MOVABLE] | __GFP_NOMEMALLOC |

Re: [PATCH v17 1/6] lib/xbitmap: Introduce xbitmap

2017-11-03 Thread Tetsuo Handa
I'm commenting without understanding the logic. Wei Wang wrote: > + > +bool xb_preload(gfp_t gfp); > + Want __must_check annotation, for __radix_tree_preload() is marked with __must_check annotation. By error failing to check result of xb_preload() will lead to preemption kept disabled unexpected

Re: [PATCH v1 1/3] virtio-balloon: replace the coarse-grained balloon_lock

2017-10-22 Thread Tetsuo Handa
_PER_PAGE) { > >>page = balloon_page_dequeue(vb_dev_info); > > If balloon_page_dequeue() can be concurrently called by both host's request > > and guest's OOM event, is (!dequeued_page) test in balloon_page_dequeue() > > safe? >

Re: [PATCH v1 1/3] virtio-balloon: replace the coarse-grained balloon_lock

2017-10-21 Thread Tetsuo Handa
Wei Wang wrote: > The balloon_lock was used to synchronize the access demand to elements > of struct virtio_balloon and its queue operations (please see commit > e22504296d). This prevents the concurrent run of the leak_balloon and > fill_balloon functions, thereby resulting in a deadlock issue on

Re: [PATCH v1 2/3] virtio-balloon: deflate up to oom_pages on OOM

2017-10-21 Thread Tetsuo Handa
Michael S. Tsirkin wrote: > On Fri, Oct 20, 2017 at 07:54:25PM +0800, Wei Wang wrote: > > The current implementation only deflates 256 pages even when a user > > specifies more than that via the oom_pages module param. This patch > > enables the deflating of up to oom_pages pages if there are enoug

Re: [PATCH] virtio: avoid possible OOM lockup at virtballoon_oom_notify()

2017-10-19 Thread Tetsuo Handa
Michael S. Tsirkin wrote: > On Wed, Oct 18, 2017 at 07:59:23PM +0900, Tetsuo Handa wrote: > > Do you see anything wrong with the patch I used for emulating > > VIRTIO_BALLOON_F_DEFLATE_ON_OOM path (shown below) ? > > > > >

Re: [PATCH] virtio: avoid possible OOM lockup at virtballoon_oom_notify()

2017-10-18 Thread Tetsuo Handa
Tetsuo Handa wrote: > 20171016-deflate.log.xz continued printing "puff" messages without any OOM > killer messages, for fill_balloon() always inflates faster than leak_balloon() > deflates. > > Since the OOM killer cannot be invoked unless leak_balloon() complete

Re: [PATCH] virtio: avoid possible OOM lockup at virtballoon_oom_notify()

2017-10-16 Thread Tetsuo Handa
Tetsuo Handa wrote: > Michael S. Tsirkin wrote: > > > > > > > > The proper fix isn't that hard - just avoid allocations under lock. > > > > > > > > Patch posted, pls take a look. > > > > > > Your patch allocates pages in

Re: [PATCH] virtio: avoid possible OOM lockup at virtballoon_oom_notify()

2017-10-14 Thread Tetsuo Handa
Michael S. Tsirkin wrote: > > > > > > The proper fix isn't that hard - just avoid allocations under lock. > > > > > > Patch posted, pls take a look. > > > > Your patch allocates pages in order to inflate the balloon, but > > your patch will allow leak_balloon() to deflate the balloon. > > How de

Re: [PATCH] virtio: avoid possible OOM lockup at virtballoon_oom_notify()

2017-10-13 Thread Tetsuo Handa
Michael S. Tsirkin wrote: > On Tue, Oct 10, 2017 at 07:47:37PM +0900, Tetsuo Handa wrote: > > In leak_balloon(), mutex_lock(&vb->balloon_lock) is called in order to > > serialize against fill_balloon(). But in fill_balloon(), > > alloc_page(GFP_HIGHUSER[_MOVABLE] | __GF

Re: [PATCH] virtio_balloon: fix deadlock on OOM

2017-10-13 Thread Tetsuo Handa
Michael S. Tsirkin wrote: > This is a replacement for > [PATCH] virtio: avoid possible OOM lockup at virtballoon_oom_notify() > but unlike that patch it actually deflates on oom even in presence of > lock contention. But Wei Wang is proposing VIRTIO_BALLOON_F_SG which will try to allocate me

Re: mm, virtio: possible OOM lockup at virtballoon_oom_notify()

2017-09-28 Thread Tetsuo Handa
Michael S. Tsirkin wrote: > On Mon, Sep 11, 2017 at 07:27:19PM +0900, Tetsuo Handa wrote: > > Hello. > > > > I noticed that virtio_balloon is using register_oom_notifier() and > > leak_balloon() from virtballoon_oom_notify() might depend on > > __GFP_DIRECT_RECLA

mm, virtio: possible OOM lockup at virtballoon_oom_notify()

2017-09-11 Thread Tetsuo Handa
Hello. I noticed that virtio_balloon is using register_oom_notifier() and leak_balloon() from virtballoon_oom_notify() might depend on __GFP_DIRECT_RECLAIM memory allocation. In leak_balloon(), mutex_lock(&vb->balloon_lock) is called in order to serialize against fill_balloon(). But in fill_ballo

[PATCH] virtio: Avoid possible kernel panic if DEBUG is enabled.

2015-02-06 Thread Tetsuo Handa
>From 11fd997d724f520ca628615e7ffbfd7901c40b62 Mon Sep 17 00:00:00 2001 From: Tetsuo Handa Date: Fri, 6 Feb 2015 13:28:38 +0900 Subject: [PATCH] virtio: Avoid possible kernel panic if DEBUG is enabled. The virtqueue_add() calls START_USE() upon entry. The virtqueue_kick() is called if