Re: About the performance of hyper-v

2021-06-01 Thread Liang Li
== > > Analyze events for all VMs, all VCPUs: > > VM-EXITSamples Samples% Time%Min TimeMax > > Time Avg time > >MSR_WRITE 92404589.96%81.10% 0.42us > > 68.42us 1.26us ( +- 0.07% ) > >DR_ACC

Re: About the performance of hyper-v

2021-05-23 Thread Liang Li
> >> > Analyze events for all VMs, all VCPUs: > >> > VM-EXITSamples Samples% Time%Min TimeMax > >> > Time Avg time > >> > EXTERNAL_INTERRUPT 47183159.89%68.58% 0.64us > >> > 65.42us 2.34us ( +- 0.11% ) > >> >MSR_WRITE 2

Re: About the performance of hyper-v

2021-05-21 Thread Liang Li
> > Hi Vitaly, > > > > I found a case that the virtualization overhead was almost doubled > > when turning on Hper-v related features compared to that without any > > no hyper-v feature. It happens when running a 3D game in windows > > guest in qemu kvm environment. > > > > By investigation, I fou

About the performance of hyper-v

2021-05-19 Thread Liang Li
[resend for missing cc] Hi Vitaly, I found a case that the virtualization overhead was almost doubled when turning on Hper-v related features compared to that without any no hyper-v feature. It happens when running a 3D game in windows guest in qemu kvm environment. By investigation, I found th

Re: [RFC PATCH 1/3] mm: support hugetlb free page reporting

2020-12-23 Thread Liang Li
> >>> +static int > >>> +hugepage_reporting_cycle(struct page_reporting_dev_info *prdev, > >>> + struct hstate *h, unsigned int nid, > >>> + struct scatterlist *sgl, unsigned int *offset) > >>> +{ > >>> + struct list_head *list = &h->hugepage_freelists[

Re: [RFC PATCH 1/3] mm: support hugetlb free page reporting

2020-12-23 Thread Liang Li
> > > > + spin_lock_irq(&hugetlb_lock); > > > > + > > > > + if (huge_page_order(h) > MAX_ORDER) > > > > + budget = HUGEPAGE_REPORTING_CAPACITY; > > > > + else > > > > + budget = HUGEPAGE_REPORTING_CAPACITY * 32; > > > > > > Wouldn't huge_page_order alwa

Re: [RFC PATCH 1/3] mm: support hugetlb free page reporting

2020-12-22 Thread Liang Li
> On 12/21/20 11:46 PM, Liang Li wrote: > > Free page reporting only supports buddy pages, it can't report the > > free pages reserved for hugetlbfs case. On the other hand, hugetlbfs > > is a good choice for a system with a huge amount of RAM, because it > &g

Re: [RFC PATCH 1/3] mm: support hugetlb free page reporting

2020-12-22 Thread Liang Li
> On 12/22/20 11:59 AM, Alexander Duyck wrote: > > On Mon, Dec 21, 2020 at 11:47 PM Liang Li > > wrote: > >> + > >> + if (huge_page_order(h) > MAX_ORDER) > >> + budget = HUGEPAGE_REPORTING_CAPACITY; > >> + else >

Re: [RFC PATCH 1/3] mm: support hugetlb free page reporting

2020-12-22 Thread Liang Li
> > +hugepage_reporting_cycle(struct page_reporting_dev_info *prdev, > > +struct hstate *h, unsigned int nid, > > +struct scatterlist *sgl, unsigned int *offset) > > +{ > > + struct list_head *list = &h->hugepage_freelists[nid]; > > + unsi

Re: [RFC PATCH 3/3] mm: support free hugepage pre zero out

2020-12-22 Thread Liang Li
> > Free page reporting in virtio-balloon doesn't give you any guarantees > > regarding zeroing of pages. Take a look at the QEMU implementation - > > e.g., with vfio all reports are simply ignored. > > > > Also, I am not sure if mangling such details ("zeroing of pages") into > > the page reportin

Re: [RFC PATCH 2/3] virtio-balloon: add support for providing free huge page reports to host

2020-12-22 Thread Liang Li
On Tue, Dec 22, 2020 at 4:28 PM David Hildenbrand wrote: > > On 22.12.20 08:48, Liang Li wrote: > > Free page reporting only supports buddy pages, it can't report the > > free pages reserved for hugetlbfs case. On the other hand, hugetlbfs > > The virtio-balloon

[RFC PATCH 0/3 updated] add support for free hugepage reporting

2020-12-22 Thread Liang Li
lloon driver, changes for the virtio spec are needed. Before that, I need the feedback of the comunity about this new feature. This RFC is baed on my previous series: '[RFC v2 PATCH 0/4] speed up page allocation for __GFP_ZERO' Liang Li (3): mm: support hugetlb free page reportin

[RFC PATCH 0/3 updated] add support for free hugepage reporting

2020-12-22 Thread Liang Li
lloon driver, changes for the virtio spec are needed. Before that, I need the feedback of the comunity about this new feature. This RFC is baed on my previous series: '[RFC v2 PATCH 0/4] speed up page allocation for __GFP_ZERO' Liang Li (3): mm: support hugetlb free page reportin

[RFC PATCH 3/3] mm: support free hugepage pre zero out

2020-12-21 Thread Liang Li
Williamson Cc: Michael S. Tsirkin Cc: Jason Wang Cc: Mike Kravetz Cc: Liang Li Signed-off-by: Liang Li --- mm/page_prezero.c | 17 + 1 file changed, 17 insertions(+) diff --git a/mm/page_prezero.c b/mm/page_prezero.c index c8ce720bfc54..dff4e0adf402 100644 --- a/mm

[RFC PATCH 2/3] virtio-balloon: add support for providing free huge page reports to host

2020-12-21 Thread Liang Li
n Cc: Alex Williamson Cc: Michael S. Tsirkin Cc: Jason Wang Cc: Mike Kravetz Cc: Liang Li Signed-off-by: Liang Li --- drivers/virtio/virtio_balloon.c | 61 + include/uapi/linux/virtio_balloon.h | 1 + 2 files changed, 62 insertions(+) diff --git a/drivers/v

[RFC PATCH 1/3] mm: support hugetlb free page reporting

2020-12-21 Thread Liang Li
David Hildenbrand Cc: Michal Hocko Cc: Andrew Morton Cc: Alex Williamson Cc: Michael S. Tsirkin Cc: Jason Wang Cc: Mike Kravetz Cc: Liang Li Signed-off-by: Liang Li --- include/linux/hugetlb.h| 3 + include/linux/page_reporting.h | 5 + mm/hugetlb.c | 29 +++

[RFC PATCH 0/3] add support for free hugepage reporting

2020-12-21 Thread Liang Li
the virtio_balloon driver, changes for the virtio spec are needed. Before that, I need the feedback of the comunity about this new feature. Liang Li (3): mm: support hugetlb free page reporting virtio-balloon: add support for providing free huge page reports to host mm: support f

Re: [Qemu-devel] [PATCH 0/2] buffer and delay backup COW write operation

2019-05-05 Thread Liang Li
On Tue, Apr 30, 2019 at 10:35:32AM +, Vladimir Sementsov-Ogievskiy wrote: > 28.04.2019 13:01, Liang Li wrote: > > If the backup target is a slow device like ceph rbd, the backup > > process will affect guest BLK write IO performance seriously, > > it's cause by the

[Qemu-devel] [PATCH 1/2] backup: buffer COW request and delay the write operation

2019-04-28 Thread Liang Li
Eric Blake Cc: Fam Zheng Signed-off-by: Liang Li --- block/backup.c | 117 ++--- 1 file changed, 104 insertions(+), 13 deletions(-) diff --git a/block/backup.c b/block/backup.c index 9988753249..d436f9e4ee 100644 --- a/block/backup.c +++ b/bloc

[Qemu-devel] [PATCH 2/2] qapi: add interface for setting backup cow buffer size

2019-04-28 Thread Liang Li
Cc: John Snow Cc: Kevin Wolf Cc: Max Reitz Cc: Wen Congyang Cc: Xie Changlong Cc: Markus Armbruster Cc: Eric Blake Cc: Fam Zheng Signed-off-by: Liang Li --- block/backup.c| 3 ++- block/replication.c | 2 +- blockdev.c| 5 + include/block/block_int.h

[Qemu-devel] [PATCH 0/2] buffer and delay backup COW write operation

2019-04-28 Thread Liang Li
Eric Blake Cc: Fam Zheng Liang Li (2): backup: buffer COW request and delay the write operation qapi: add interface for setting backup cow buffer size block/backup.c| 118 +- block/replication.c | 2 +- blockdev.c

Re: [Qemu-devel] [PATCH] vhost-user: fix qemu crash caused by failed backend

2018-10-29 Thread Liang Li
On Tue, Oct 02, 2018 at 01:54:25PM +0400, Marc-André Lureau wrote: > Hi > > On Thu, Sep 27, 2018 at 7:37 PM Liang Li wrote: > > > > During live migration, when stopping vhost-user device, 'vhost_dev_stop' > > will be called, 'vhost_dev_stop'

[Qemu-devel] [PATCH] migration: fix concurrent call of multifd_save_cleanup

2018-10-29 Thread Liang Li
Concurrent call of multifd_save_cleanup() is unsafe, it will lead to null pointer dereference. 'multifd_save_cleanup()' should not be called in multifd_new_send_channel_async(), move it to ram_save_cleanup() like other features do. Signed-off-by: Liang Li --- migration/migra

[Qemu-devel] [PATCH] vhost-user: fix qemu crash caused by failed backend

2018-09-27 Thread Liang Li
the vhost user backend failed, the 'CHR_EVENT_CLOSED' event will be triggerd, followed by the call chain chr_closed_bh()->vhost_user_stop()-> vhost_net_cleanup()->vhost_dev_cleanup() vhost_dev_cleanup will clear vhost_dev struct, so the later 'vhost_user_read' or &#

[Qemu-devel] [PATCH] vhost-user: fix qemu crash caused by failed backend

2018-09-27 Thread Liang Li
the vhost user backend failed, the 'CHR_EVENT_CLOSED' event will be triggerd, followed by the call chain chr_closed_bh()->vhost_user_stop()-> vhost_net_cleanup()->vhost_dev_cleanup() vhost_dev_cleanup will clear vhost_dev struct, so the later 'vhost_user_read' or &#

Re: [Qemu-devel] [PATCH V5] migration: add capability to bypass the shared memory

2018-06-27 Thread Liang Li
On Mon, Apr 16, 2018 at 11:00:11PM +0800, Lai Jiangshan wrote: > > migration/migration.c | 22 ++ > migration/migration.h | 1 + > migration/ram.c | 27 ++- > qapi/migration.json | 6 +- > 4 files changed, 46 insertions(+), 10 deletions(-)

Re: [Qemu-devel] [PATCH v2 3/3] virtio-balloon: add a timer to limit the free page report waiting time

2018-02-27 Thread Liang Li
On Tue, Feb 27, 2018 at 06:10:47PM +0800, Wei Wang wrote: > On 02/27/2018 08:50 AM, Michael S. Tsirkin wrote: > > On Mon, Feb 26, 2018 at 12:35:31PM +0800, Wei Wang wrote: > > > On 02/09/2018 08:15 PM, Dr. David Alan Gilbert wrote: > > > > * Wei Wang (wei.w.w...@intel.com) wrote: > > > > I think a

[Qemu-devel] [PATCH v2 resend] block/mirror: change the semantic of 'force' of block-job-cancel

2018-02-26 Thread Liang Li
in sync, and the command completes immediately), we can just improve the documentation to make the force flag obviously useful. Cc: Paolo Bonzini Cc: Jeff Cody Cc: Kevin Wolf Cc: Max Reitz Cc: Eric Blake Cc: John Snow Reported-by: Huaitong Han Signed-off-by: Huaitong Han Signed-off-by: Liang Li

[Qemu-devel] [PATCH v2] block/mirror: change the semantic of 'force' of block-job-cancel

2018-02-05 Thread Liang Li
in sync, and the command completes immediately), we can just improve the documentation to make the force flag obviously useful. Cc: Paolo Bonzini Cc: Jeff Cody Cc: Kevin Wolf Cc: Max Reitz Cc: Eric Blake Cc: John Snow Reported-by: Huaitong Han Signed-off-by: Huaitong Han Signed-off-by: Liang Li

Re: [Qemu-devel] [PATCH] block/mirror: change the semantic of 'force' of block-job-cancel

2018-02-05 Thread Liang Li
On Mon, Feb 05, 2018 at 02:28:55PM -0500, John Snow wrote: > > > On 01/31/2018 09:19 PM, Liang Li wrote: > > On Tue, Jan 30, 2018 at 03:18:31PM -0500, John Snow wrote: > >> > >> > >> On 01/30/2018 03:38 AM, Liang Li wrote: > >>> When doing dr

Re: [Qemu-devel] [PATCH] block/mirror: change the semantic of 'force' of block-job-cancel

2018-01-31 Thread Liang Li
On Tue, Jan 30, 2018 at 03:18:31PM -0500, John Snow wrote: > > > On 01/30/2018 03:38 AM, Liang Li wrote: >> When doing drive mirror to a low speed shared storage, if there was heavy >> BLK IO write workload in VM after the 'ready' event, drive mirror block job &

Re: [Qemu-devel] [PATCH] block/mirror: change the semantic of 'force' of block-job-cancel

2018-01-31 Thread Liang Li
On Tue, Jan 30, 2018 at 08:20:03AM -0600, Eric Blake wrote: > On 01/30/2018 02:38 AM, Liang Li wrote: >> When doing drive mirror to a low speed shared storage, if there was heavy >> BLK IO write workload in VM after the 'ready' event, drive mirror block job >> ca

[Qemu-devel] [PATCH] block/mirror: change the semantic of 'force' of block-job-cancel

2018-01-30 Thread Liang Li
antic. Cc: Paolo Bonzini Cc: Jeff Cody Cc: Kevin Wolf Cc: Max Reitz Cc: Eric Blake Cc: John Snow Reported-by: Huaitong Han Signed-off-by: Huaitong Han Signed-off-by: Liang Li --- block/mirror.c| 9 +++-- blockdev.c| 4 ++-- blockjob.c|

Re: [Qemu-devel] [PATCH] block/mirror: fix fail to cancel when VM has heavy BLK IO

2018-01-28 Thread Liang Li
On Fri, Jan 26, 2018 at 08:04:08AM -0600, Eric Blake wrote: > On 01/26/2018 12:46 AM, Liang Li wrote: > > The current QMP command is: > > > > { 'command': 'block-job-cancel', 'data': { 'device': 'str', '*force': &g

Re: [Qemu-devel] [PATCH] block/mirror: fix fail to cancel when VM has heavy BLK IO

2018-01-25 Thread Liang Li
On Thu, Jan 25, 2018 at 08:48:22AM -0600, Eric Blake wrote: > On 01/24/2018 10:59 PM, Liang Li wrote: > >> > >> There's ongoing work on adding async mirroring; this may be a better > >> solution to the issue you are seeing. > >> > >> https://l

Re: [Qemu-devel] [PATCH] block/mirror: fix fail to cancel when VM has heavy BLK IO

2018-01-24 Thread Liang Li
On Wed, Jan 24, 2018 at 01:16:39PM -0600, Eric Blake wrote: > On 01/24/2018 12:17 AM, Liang Li wrote: > > We found that when doing drive mirror to a low speed shared storage, > > if there was heavy BLK IO write workload in VM after the 'ready' event, > > drive mir

[Qemu-devel] [PATCH] block/mirror: fix fail to cancel when VM has heavy BLK IO

2018-01-23 Thread Liang Li
ixed this issue. Cc: Paolo Bonzini Cc: Jeff Cody Cc: Kevin Wolf Cc: Max Reitz Signed-off-by: Huaitong Han Signed-off-by: Liang Li --- block/mirror.c | 10 -- 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index c9badc1..3bc49a5 100644 --- a

[Qemu-devel] [PATCH] hbitmap: fix missing restore count when finish deserialization

2018-01-18 Thread Liang Li
The .count of HBitmap is forgot to set in function hbitmap_deserialize_finish, let's set it to the right value. Cc: Vladimir Sementsov-Ogievskiy Cc: Fam Zheng Cc: Max Reitz Cc: John Snow Signed-off-by: weiping zhang Signed-off-by: Liang Li --- util/hbitmap.c | 1 + 1 file chang

[Qemu-devel] [PATCH resend] hbitmap: fix missing restore count when finish deserialization

2018-01-18 Thread Liang Li
The .count of HBitmap is forgot to set in function hbitmap_deserialize_finish, let's set it to the right value. Cc: Vladimir Sementsov-Ogievskiy Cc: Fam Zheng Cc: Max Reitz Cc: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Weiping Zhang Signed-off-by: Lia

[Qemu-devel] [PATCH v4 qemu 6/6] migration: skip unused pages during live migration

2017-01-11 Thread Liang Li
unused pages will not be filtered out in this case. The current implementation can not work with post copy, if post copy is enabled, we simply ignore the unused pages. Will make it work later. Signed-off-by: Liang Li --- migration/ram.c | 86

[Qemu-devel] [PATCH v4 qemu 5/6] kvm.c: Add two new arch specific functions

2017-01-11 Thread Liang Li
Add a new function to get the vm's max pfn and a new function to filter out the holes in the undressed free page bitmap to get a tight free page bitmap. They are implemented on X86 and should be implemented on other arches for live migration optimization. Signed-off-by: Liang Li --- in

[Qemu-devel] [PATCH v4 qemu 4/6] bitmap: Add a new bitmap_move function

2017-01-11 Thread Liang Li
Sometimes, it is need to move a portion of bitmap to another place in a large bitmap, if overlap happens, the bitmap_copy can't not work correctly, we need a new function to do this work. Signed-off-by: Liang Li Reviewed-by: Dr. David Alan Gilbert --- include/qemu/bitmap.h

[Qemu-devel] [PATCH v4 qemu 0/6] Fast (de)inflating & fast live migration

2017-01-11 Thread Liang Li
ct from vq instead of separate variables. * Use two separate APIs to request free pages and query the status. * Changed the virtio balloon interface. * Addressed some of the comments of v1. Liang Li (6): virtio-balloon: update linux head file virtio-balloon: speed up inflating & def

[Qemu-devel] [PATCH v4 qemu 3/6] balloon: get unused page info from guest

2017-01-11 Thread Liang Li
ned-off-by: Liang Li --- balloon.c | 47 +++- hw/virtio/virtio-balloon.c | 149 - include/hw/virtio/virtio-balloon.h | 18 - include/sysemu/balloon.h | 18 - 4 files changed, 227 insertions(+), 5

[Qemu-devel] [PATCH v4 qemu 2/6] virtio-balloon: speed up inflating & deflating process

2017-01-11 Thread Liang Li
ture, inflating the balloon to 7GB of a 8GB idle guest only takes 590ms, the performance improvement is about 85%. TODO: optimize stage a by allocating/freeing a chunk of pages instead of a single page at a time. Signed-off-by: Liang Li Suggested-by: Michael S. Tsirkin --- hw/virtio/vi

[Qemu-devel] [PATCH v4 qemu 1/6] virtio-balloon: update linux head file

2017-01-11 Thread Liang Li
Update the linux head file to keep consistent with kernel side. The new definition will be used in the following patches. Signed-off-by: Liang Li --- include/standard-headers/linux/virtio_balloon.h | 34 + 1 file changed, 34 insertions(+) diff --git a/include/standard

[Qemu-devel] [PATCH v6 kernel 5/5] virtio-balloon: tell host vm's unused page info

2016-12-20 Thread Liang Li
nused pages, this is very helpful to reduce the network traffic and speed up the live migration process. Signed-off-by: Liang Li Cc: Andrew Morton Cc: Mel Gorman Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah Cc: Dave Hansen Cc: Andrea Arcangeli Cc: David Hilden

[Qemu-devel] [PATCH v6 kernel 3/5] virtio-balloon: speed up inflate/deflate process

2016-12-20 Thread Liang Li
new feature, inflating the balloon to 7GB of a 8GB idle guest only takes 590ms, the performance improvement is about 85%. TODO: optimize stage a by allocating/freeing a chunk of pages instead of a single page at a time. Signed-off-by: Liang Li Suggested-by: Michael S. Tsirkin Cc: Michael S. Tsirkin

[Qemu-devel] [PATCH v6 kernel 4/5] virtio-balloon: define flags and head for host request vq

2016-12-20 Thread Liang Li
hypervisor can get some of guest's runtime information through this virtual queue too, e.g. the guest's unused page information, which can be used for live migration optimization. Signed-off-by: Liang Li Cc: Andrew Morton Cc: Mel Gorman Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc

[Qemu-devel] [PATCH v6 kernel 2/5] virtio-balloon: define new feature bit and head struct

2016-12-20 Thread Liang Li
Add a new feature which supports sending the page information with range array. The current implementation uses PFNs array, which is not very efficient. Using ranges can improve the performance of inflating/deflating significantly. Signed-off-by: Liang Li Cc: Michael S. Tsirkin Cc: Paolo

[Qemu-devel] [PATCH v6 kernel 1/5] virtio-balloon: rework deflate to add page to a list

2016-12-20 Thread Liang Li
pfn|length} down the road. balloon_pfn_to_page() can be removed because it's useless. Signed-off-by: Liang Li Signed-off-by: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah Cc: Dave Hansen Cc: Andrea Arcangeli Cc: David Hildenbrand --- drivers/virtio/virtio_balloo

[Qemu-devel] [PATCH v6 kernel 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-12-20 Thread Liang Li
ew way to determine the page bitmap size. * Use a unified way to send the free page information with the bitmap * Address the issues referred in MST's comments Liang Li (5): virtio-balloon: rework deflate to add page to a list virtio-balloon: define new feature bit and head struct

[Qemu-devel] [PATCH kernel v5 4/5] virtio-balloon: define flags and head for host request vq

2016-11-30 Thread Liang Li
hypervisor can get some of guest's runtime information through this virtual queue too, e.g. the guest's unused page information, which can be used for live migration optimization. Signed-off-by: Liang Li Cc: Andrew Morton Cc: Mel Gorman Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc

[Qemu-devel] [PATCH kernel v5 3/5] virtio-balloon: speed up inflate/deflate process

2016-11-30 Thread Liang Li
ture, inflating the balloon to 7GB of a 8GB idle guest only takes 590ms, the performance improvement is about 85%. TODO: optimize stage a by allocating/freeing a chunk of pages instead of a single page at a time. Signed-off-by: Liang Li Suggested-by: Michael S. Tsirkin Cc: Michael S. Tsirkin Cc: P

[Qemu-devel] [PATCH kernel v5 2/5] virtio-balloon: define new feature bit and head struct

2016-11-30 Thread Liang Li
about the page bitmap. e.g. the page size, page bitmap length and start pfn. Signed-off-by: Liang Li Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah Cc: Dave Hansen --- include/uapi/linux/virtio_balloon.h | 19 +++ 1 file changed, 19 insertions(+) diff

[Qemu-devel] [PATCH kernel v5 5/5] virtio-balloon: tell host vm's unused page info

2016-11-30 Thread Liang Li
nused pages, this is very helpful to reduce the network traffic and speed up the live migration process. Signed-off-by: Liang Li Cc: Andrew Morton Cc: Mel Gorman Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah Cc: Dave Hansen --- drivers/virtio/virtio_balloon.c

[Qemu-devel] [PATCH kernel v5 1/5] virtio-balloon: rework deflate to add page to a list

2016-11-30 Thread Liang Li
of the PFN array, which will allow faster notifications using a bitmap down the road. balloon_pfn_to_page() can be removed because it's useless. Signed-off-by: Liang Li Signed-off-by: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah Cc: Dave Hansen --- driver

[Qemu-devel] [PATCH kernel v5 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-11-30 Thread Liang Li
d the free page information with the bitmap * Address the issues referred in MST's comments Liang Li (5): virtio-balloon: rework deflate to add page to a list virtio-balloon: define new feature bit and head struct virtio-balloon: speed up inflate/deflate process virtio-balloon: define fla

[Qemu-devel] [PATCH kernel v4 7/7] virtio-balloon: tell host vm's unused page info

2016-11-01 Thread Liang Li
ation process. Signed-off-by: Liang Li Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah Cc: Dave Hansen --- drivers/virtio/virtio_balloon.c | 128 +--- 1 file changed, 121 insertions(+), 7 deletions(-) diff --git a/drivers/v

[Qemu-devel] [PATCH kernel v4 6/7] virtio-balloon: define flags and head for host request vq

2016-11-01 Thread Liang Li
hypervisor can get some of guest's runtime information through this virtual queue too, e.g. the guest's unused page information, which can be used for live migration optimization. Signed-off-by: Liang Li Cc: Andrew Morton Cc: Mel Gorman Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc

[Qemu-devel] [PATCH kernel v4 5/7] mm: add the related functions to get unused page

2016-11-01 Thread Liang Li
can be corrected by the dirty page logging mechanism. Signed-off-by: Liang Li Cc: Andrew Morton Cc: Mel Gorman Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah Cc: Dave Hansen --- include/linux/mm.h | 2 ++ mm/page_alloc.c| 85

[Qemu-devel] [PATCH kernel v4 4/7] virtio-balloon: speed up inflate/deflate process

2016-11-01 Thread Liang Li
ture, inflating the balloon to 7GB of a 8GB idle guest only takes 590ms, the performance improvement is about 85%. TODO: optimize stage a by allocating/freeing a chunk of pages instead of a single page at a time. Signed-off-by: Liang Li Suggested-by: Michael S. Tsirkin Cc: Michael S. Tsirkin Cc: P

[Qemu-devel] [PATCH kernel v4 3/7] mm: add a function to get the max pfn

2016-11-01 Thread Liang Li
Expose the function to get the max pfn, so it can be used in the virtio-balloon device driver. Simply include the 'linux/bootmem.h' is not enough, if the device driver is built to a module, directly refer the max_pfn lead to build failed. Signed-off-by: Liang Li Cc: Andrew Morton Cc:

[Qemu-devel] [PATCH kernel v4 2/7] virtio-balloon: define new feature bit and head struct

2016-11-01 Thread Liang Li
about the page bitmap. e.g. the page size, page bitmap length and start pfn. Signed-off-by: Liang Li Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah Cc: Dave Hansen --- include/uapi/linux/virtio_balloon.h | 19 +++ 1 file changed, 19 insertions(+) diff

[Qemu-devel] [PATCH kernel v4 0/7] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-11-01 Thread Liang Li
che. * Put some structures to uapi head file. * Use a new way to determine the page bitmap size. * Use a unified way to send the free page information with the bitmap * Address the issues referred in MST's comments Liang Li (7): virtio-balloon: rework deflate to add page to a l

[Qemu-devel] [PATCH kernel v4 1/7] virtio-balloon: rework deflate to add page to a list

2016-11-01 Thread Liang Li
of the PFN array, which will allow faster notifications using a bitmap down the road. balloon_pfn_to_page() can be removed because it's useless. Signed-off-by: Liang Li Signed-off-by: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah Cc: Dave Hansen --- driver

[Qemu-devel] [PATCH qemu v3 6/6] migration: skip free pages during live migration

2016-10-21 Thread Liang Li
will not be filtered out in this case. The current implementation can not work with post copy, if post copy is enabled, we simply ignore the free pages. Will make it work later. Signed-off-by: Liang Li --- migration/ram.c | 86 + 1 file

[Qemu-devel] [PATCH qemu v3 1/6] virtio-balloon: update linux head file

2016-10-21 Thread Liang Li
Update the new feature bit definition for virtio balloon and the page bitmap header, request header struct to keep consistent with kernel side. Signed-off-by: Liang Li --- include/standard-headers/linux/virtio_balloon.h | 41 + 1 file changed, 41 insertions(+) diff

[Qemu-devel] [PATCH qemu v3 5/6] kvm: Add two new arch specific functions

2016-10-21 Thread Liang Li
Add a new function to get the vm's max pfn and a new function to filter out the holes in the undressed free page bitmap to get a tight free page bitmap. They are implemented on X86 and should be implemented on other arches for live migration optimization. Signed-off-by: Liang Li Reviewed-b

[Qemu-devel] [PATCH qemu v3 4/6] bitmap: Add a new bitmap_move function

2016-10-21 Thread Liang Li
Sometimes, it is need to move a portion of bitmap to another place in a large bitmap, if overlap happens, the bitmap_copy can't not work correctly, we need a new function to do this work. Signed-off-by: Liang Li --- include/qemu/bitmap.h | 13 + 1 file changed, 13 inser

[Qemu-devel] [PATCH qemu v3 0/6] Fast (de)inflating & fast live migration

2016-10-21 Thread Liang Li
io balloon interface. * Addressed some of the comments of v1. Liang Li (6): virtio-balloon: update linux head file virtio-balloon: speed up inflating & deflating process balloon: get free page info from guest bitmap: Add a new bitmap_move function kvm: Add two new arch specific functio

[Qemu-devel] [PATCH qemu v3 3/6] balloon: get free page info from guest

2016-10-21 Thread Liang Li
Add a new feature to get the free page information from guest, the free page information is saved in a bitmap. Please note that 'free page' means page is free sometime after host set the value of request ID and before it receive response with the same ID. Signed-off-by: Liang Li ---

[Qemu-devel] [PATCH qemu v3 2/6] virtio-balloon: speed up inflating & deflating process

2016-10-21 Thread Liang Li
ture, inflating the balloon to 7GB of a 8GB idle guest only takes 590ms, the performance improvement is about 85%. TODO: optimize stage a by allocating/freeing a chunk of pages instead of a single page at a time. Signed-off-by: Liang Li Suggested-by: Michael S. Tsirkin --- hw/virtio/virtio-balloon.c |

[Qemu-devel] [RESEND PATCH v3 kernel 5/7] mm: add the related functions to get unused page

2016-10-20 Thread Liang Li
corrected by the dirty page logging mechanism. Signed-off-by: Liang Li Cc: Andrew Morton Cc: Mel Gorman Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah --- include/linux/mm.h | 2 ++ mm/page_alloc.c| 84 ++ 2

[Qemu-devel] [RESEND PATCH v3 kernel 6/7] virtio-balloon: define feature bit and head for misc virt queue

2016-10-20 Thread Liang Li
VMM hypervisor can get some of guest's runtime information through this virtual queue, e.g. the guest's unused page information, which can be used for live migration optimization. Signed-off-by: Liang Li Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah --- in

[Qemu-devel] [RESEND PATCH v3 kernel 4/7] virtio-balloon: speed up inflate/deflate process

2016-10-20 Thread Liang Li
ture, inflating the balloon to 7GB of a 8GB idle guest only takes 590ms, the performance improvement is about 85%. TODO: optimize stage a by allocating/freeing a chunk of pages instead of a single page at a time. Signed-off-by: Liang Li Suggested-by: Michael S. Tsirkin Cc: Michael S. Tsirkin Cc: P

[Qemu-devel] [RESEND PATCH v3 kernel 7/7] virtio-balloon: tell host vm's unused page info

2016-10-20 Thread Liang Li
Support the request for vm's unused page information, response with a page bitmap. QEMU can make use of this bitmap and the dirty page logging mechanism to skip the transportation of these unused pages, this is very helpful to speed up the live migration process. Signed-off-by: Liang L

[Qemu-devel] [RESEND PATCH v3 kernel 3/7] mm: add a function to get the max pfn

2016-10-20 Thread Liang Li
Expose the function to get the max pfn, so it can be used in the virtio-balloon device driver. Simply include the 'linux/bootmem.h' is not enough, if the device driver is built to a module, directly refer the max_pfn lead to build failed. Signed-off-by: Liang Li Cc: Andrew Morton Cc:

[Qemu-devel] [RESEND PATCH v3 kernel 2/7] virtio-balloon: define new feature bit and page bitmap head

2016-10-20 Thread Liang Li
about the page bitmap. e.g. the page size, page bitmap length and start pfn. Signed-off-by: Liang Li Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah --- include/uapi/linux/virtio_balloon.h | 19 +++ 1 file changed, 19 insertions(+) diff --git a/include

[Qemu-devel] [RESEND PATCH v3 kernel 0/7] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-10-20 Thread Liang Li
ge information with the bitmap * Address the issues referred in MST's comments Liang Li (7): virtio-balloon: rework deflate to add page to a list virtio-balloon: define new feature bit and page bitmap head mm: add a function to get the max pfn virtio-balloon: speed up inflate/d

[Qemu-devel] [RESEND PATCH v3 kernel 1/7] virtio-balloon: rework deflate to add page to a list

2016-10-20 Thread Liang Li
Will allow faster notifications using a bitmap down the road. balloon_pfn_to_page() can be removed because it's useless. Signed-off-by: Liang Li Signed-off-by: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah --- drivers/virtio/virtio_balloon.c

[Qemu-devel] [PATCH repost] virtio-balloon: Remove needless precompiled directive

2016-08-08 Thread Liang Li
Since there in wrapper around madvise(), the virtio-balloon code is able to work without the precompiled directive, the directive can be removed. Signed-off-by: Liang Li Suggested-by: Thomas Huth Reviewd-by: Dr. David Alan Gilbert --- hw/virtio/virtio-balloon.c | 2 -- 1 file changed, 2

[Qemu-devel] [PATCH] migration: fix live migration failure with compression

2016-08-08 Thread Liang Li
Because of commit 11808bb0c422, which remove some condition checks of 'f->ops->writev_buffer', 'qemu_put_qemu_file' should be enhanced to clear the 'f_src->iovcnt', or 'f_src->iovcnt' may exceed the MAX_IOV_SIZE which will break live migra

[Qemu-devel] [PATCH v3 kernel 5/7] mm: add the related functions to get unused page

2016-08-07 Thread Liang Li
corrected by the dirty page logging mechanism. Signed-off-by: Liang Li Cc: Andrew Morton Cc: Mel Gorman Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah Cc: Dave Hansen --- include/linux/mm.h | 2 ++ mm/page_alloc.c| 84

[Qemu-devel] [PATCH v3 kernel 3/7] mm: add a function to get the max pfn

2016-08-07 Thread Liang Li
Expose the function to get the max pfn, so it can be used in the virtio-balloon device driver. Simply include the 'linux/bootmem.h' is not enough, if the device driver is built to a module, directly refer the max_pfn lead to build failed. Signed-off-by: Liang Li Cc: Andrew Morton Cc:

[Qemu-devel] [PATCH v3 kernel 7/7] virtio-balloon: tell host vm's unused page info

2016-08-07 Thread Liang Li
Support the request for vm's unused page information, response with a page bitmap. QEMU can make use of this bitmap and the dirty page logging mechanism to skip the transportation of these unused pages, this is very helpful to speed up the live migration process. Signed-off-by: Liang L

[Qemu-devel] [PATCH v3 kernel 2/7] virtio-balloon: define new feature bit and page bitmap head

2016-08-07 Thread Liang Li
about the page bitmap. e.g. the page size, page bitmap length and start pfn. Signed-off-by: Liang Li Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah Cc: Dave Hansen --- include/uapi/linux/virtio_balloon.h | 19 +++ 1 file changed, 19 insertions(+) diff

[Qemu-devel] [PATCH v3 kernel 6/7] virtio-balloon: define feature bit and head for misc virt queue

2016-08-07 Thread Liang Li
VMM hypervisor can get some of guest's runtime information through this virtual queue, e.g. the guest's unused page information, which can be used for live migration optimization. Signed-off-by: Liang Li Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah Cc: D

[Qemu-devel] [PATCH v3 kernel 1/7] virtio-balloon: rework deflate to add page to a list

2016-08-07 Thread Liang Li
Will allow faster notifications using a bitmap down the road. balloon_pfn_to_page() can be removed because it's useless. Signed-off-by: Liang Li Signed-off-by: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah Cc: Dave Hansen --- drivers/virtio/virtio_balloon.c

[Qemu-devel] [PATCH v3 kernel 4/7] virtio-balloon: speed up inflate/deflate process

2016-08-07 Thread Liang Li
ture, inflating the balloon to 7GB of a 8GB idle guest only takes 590ms, the performance improvement is about 85%. TODO: optimize stage a by allocating/freeing a chunk of pages instead of a single page at a time. Signed-off-by: Liang Li Suggested-by: Michael S. Tsirkin Cc: Michael S. Tsirkin Cc: P

[Qemu-devel] [PATCH v3 kernel 0/7] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-08-07 Thread Liang Li
page information with the bitmap * Address the issues referred in MST's comments Liang Li (7): virtio-balloon: rework deflate to add page to a list virtio-balloon: define new feature bit and page bitmap head mm: add a function to get the max pfn virtio-balloon: speed up inflate/d

[Qemu-devel] [PATCH v2 repost 6/7] mm: add the related functions to get free page info

2016-07-26 Thread Liang Li
Save the free page info into a page bitmap, will be used in virtio balloon device driver. Signed-off-by: Liang Li Cc: Andrew Morton Cc: Vlastimil Babka Cc: Mel Gorman Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah --- mm/page_alloc.c | 46

[Qemu-devel] [PATCH v2 repost 5/7] virtio-balloon: define feature bit and head for misc virt queue

2016-07-26 Thread Liang Li
VMM hypervisor can get some of guest's runtime information through this virtual queue, e.g. the guest's free page information, which can be used for live migration optimization. Signed-off-by: Liang Li Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah --- in

[Qemu-devel] [PATCH v2 repost 1/7] virtio-balloon: rework deflate to add page to a list

2016-07-26 Thread Liang Li
will allow faster notifications using a bitmap down the road. balloon_pfn_to_page() can be removed because it's useless. Signed-off-by: Liang Li Signed-off-by: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah --- drivers/virtio/virtio_balloon.c

[Qemu-devel] [PATCH v2 repost 7/7] virtio-balloon: tell host vm's free page info

2016-07-26 Thread Liang Li
Support the request for vm's free page information, response with a page bitmap. QEMU can make use of this free page bitmap to speed up live migration process by skipping process the free pages. Signed-off-by: Liang Li Cc: Michael S. Tsirkin Cc: Andrew Morton Cc: Vlastimil Babka Cc

[Qemu-devel] [PATCH v2 repost 3/7] mm: add a function to get the max pfn

2016-07-26 Thread Liang Li
Expose the function to get the max pfn, so it can be used in the virtio-balloon device driver. Signed-off-by: Liang Li Cc: Andrew Morton Cc: Vlastimil Babka Cc: Mel Gorman Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah --- mm/page_alloc.c | 6 ++ 1 file

[Qemu-devel] [PATCH v2 repost 0/7] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-07-26 Thread Liang Li
Changes from v1 to v2: * Abandon the patch for dropping page cache. * Put some structures to uapi head file. * Use a new way to determine the page bitmap size. * Use a unified way to send the free page information with the bitmap * Address the issues referred in MST's co

[Qemu-devel] [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process

2016-07-26 Thread Liang Li
ture, inflating the balloon to 7GB of a 8GB idle guest only takes 590ms, the performance improvement is about 85%. TODO: optimize stage a by allocating/freeing a chunk of pages instead of a single page at a time. Signed-off-by: Liang Li Suggested-by: Michael S. Tsirkin Cc: Michael S. Tsirkin Cc: An

[Qemu-devel] [PATCH v2 repost 2/7] virtio-balloon: define new feature bit and page bitmap head

2016-07-26 Thread Liang Li
about the page bitmap. e.g. the page size, page bitmap length and start pfn. Signed-off-by: Liang Li Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah --- include/uapi/linux/virtio_balloon.h | 19 +++ 1 file changed, 19 insertions(+) diff --git a/include

[Qemu-devel] [QEMU v2 5/9] balloon: get free page info from guest

2016-07-14 Thread Liang Li
Add a new feature to get the free page information from guest, the free page information is saved in a bitmap. Please note that 'free page' means page is free sometime after host set the value of request ID and before it receive response with the same ID. Signed-off-by: Liang Li ---

  1   2   3   4   >