Re: [PATCH 4/6] hugetlb: avoid allocation failed when page reporting is on going

2021-01-10 Thread Liang Li
> > > Please don't use this email address for me anymore. Either use > > > alexander.du...@gmail.com or alexanderdu...@fb.com. I am getting > > > bounces when I reply to this thread because of the old address. > > > > No problem. > > > > > > diff --git a/mm/hugetlb.c b/mm/hugetlb.c > > > > index

Re: [PATCH 3/6] hugetlb: add free page reporting support

2021-01-10 Thread Liang Li
On Fri, Jan 8, 2021 at 6:04 AM Mike Kravetz wrote: > > On 1/5/21 7:49 PM, Liang Li wrote: > > hugetlb manages its page in hstate's free page list, not in buddy > > system, this patch try to make it works for hugetlbfs. It canbe > > used for memory overcommit in virtualizat

Re: [PATCH 3/6] hugetlb: add free page reporting support

2021-01-10 Thread Liang Li
> >> On Tue 05-01-21 22:49:21, Liang Li wrote: > >>> hugetlb manages its page in hstate's free page list, not in buddy > >>> system, this patch try to make it works for hugetlbfs. It canbe > >>> used for memory overcommit in virtualization and hugetlb

Re: [PATCH 4/6] hugetlb: avoid allocation failed when page reporting is on going

2021-01-06 Thread Liang Li
> > Page reporting isolates free pages temporarily when reporting > > free pages information. It will reduce the actual free pages > > and may cause application failed for no enough available memory. > > This patch try to solve this issue, when there is no free page > > and page repoting is on

Re: [PATCH 3/6] hugetlb: add free page reporting support

2021-01-06 Thread Liang Li
On Thu, Jan 7, 2021 at 12:08 AM Michal Hocko wrote: > > On Tue 05-01-21 22:49:21, Liang Li wrote: > > hugetlb manages its page in hstate's free page list, not in buddy > > system, this patch try to make it works for hugetlbfs. It canbe > > used for memory overcommit in vi

Re: [PATCH 2/6] mm: let user decide page reporting option

2021-01-06 Thread Liang Li
> > enum { > > PAGE_REPORTING_IDLE = 0, > > @@ -44,7 +45,7 @@ __page_reporting_request(struct page_reporting_dev_info > > *prdev) > > * now we are limiting this to running no more than once every > > * couple of seconds. > > */ > > -

Re: [PATCH 1/6] mm: Add batch size for free page reporting

2021-01-06 Thread Liang Li
> So you are going to need a lot more explanation for this. Page > reporting already had the concept of batching as you could only scan > once every 2 seconds as I recall. Thus the "PAGE_REPORTING_DELAY". The > change you are making doesn't make any sense without additional > context. The reason

Re: [PATCH 0/6] hugetlbfs: support free page reporting

2021-01-06 Thread Liang Li
On Wed, Jan 6, 2021 at 5:41 PM David Hildenbrand wrote: > > On 06.01.21 04:46, Liang Li wrote: > > A typical usage of hugetlbfs it's to reserve amount of memory > > during the kernel booting stage, and the reserved pages are > > unlikely to return to the buddy syste

[PATCH 6/6] hugetlb: support free hugepage pre zero out

2021-01-05 Thread Liang Li
Williamson Cc: Michael S. Tsirkin Cc: Jason Wang Cc: Liang Li Signed-off-by: Liang Li --- include/linux/page-flags.h | 12 ++ mm/Kconfig | 10 ++ mm/huge_memory.c | 3 +- mm/hugetlb.c | 243 + mm/memory.c

[PATCH 5/6] virtio-balloon: reporting hugetlb free page to host

2021-01-05 Thread Liang Li
add support for reporting free hugepage to host when guest use hugetlbfs. Cc: Alexander Duyck Cc: Mel Gorman Cc: Andrea Arcangeli Cc: Dan Williams Cc: Dave Hansen Cc: David Hildenbrand Cc: Michal Hocko Cc: Andrew Morton Cc: Alex Williamson Cc: Michael S. Tsirkin Cc: Liang Li Signed-off

[PATCH 3/6] hugetlb: add free page reporting support

2021-01-05 Thread Liang Li
: David Hildenbrand Cc: Michal Hocko Cc: Andrew Morton Cc: Alex Williamson Cc: Michael S. Tsirkin Cc: Liang Li Signed-off-by: Liang Li --- include/linux/hugetlb.h| 3 + include/linux/page_reporting.h | 4 + mm/Kconfig | 1 + mm/hugetlb.c

[PATCH 4/6] hugetlb: avoid allocation failed when page reporting is on going

2021-01-05 Thread Liang Li
it is done. Cc: Alexander Duyck Cc: Mel Gorman Cc: Andrea Arcangeli Cc: Dan Williams Cc: Dave Hansen Cc: David Hildenbrand Cc: Michal Hocko Cc: Andrew Morton Cc: Alex Williamson Cc: Michael S. Tsirkin Cc: Liang Li Signed-off-by: Liang Li --- include/linux/hugetlb.h | 2 ++ mm/hugetlb.c

[PATCH 2/6] mm: let user decide page reporting option

2021-01-05 Thread Liang Li
Hildenbrand Cc: Michal Hocko Cc: Andrew Morton Cc: Alex Williamson Cc: Michael S. Tsirkin Cc: Liang Li Signed-off-by: Liang Li --- drivers/virtio/virtio_balloon.c | 3 +++ include/linux/page_reporting.h | 3 +++ mm/page_reporting.c | 13 + mm/page_reporting.h

[PATCH 1/6] mm: Add batch size for free page reporting

2021-01-05 Thread Liang Li
threshold to control the waking up of reporting worker. Cc: Alexander Duyck Cc: Mel Gorman Cc: Andrea Arcangeli Cc: Dan Williams Cc: Dave Hansen Cc: David Hildenbrand Cc: Michal Hocko Cc: Andrew Morton Cc: Alex Williamson Cc: Michael S. Tsirkin Cc: Liang Li Signed-off-by: Liang Li

[PATCH 0/6] hugetlbfs: support free page reporting

2021-01-05 Thread Liang Li
of the 'buddy free page pre zero out' feature brings, I remove it from this serier. Liang Li (6): mm: Add batch size for free page reporting mm: let user decide page reporting option hugetlb: add free page reporting support hugetlb: avoid allocation failed when page reporting is on going

Re: [RFC v2 PATCH 0/4] speed up page allocation for __GFP_ZERO

2021-01-05 Thread Liang Li
> >> That‘s mostly already existing scheduling logic, no? (How many vms can I > >> put onto a specific machine eventually) > > > > It depends on how the scheduling component is designed. Yes, you can put > > 10 VMs with 4C8G(4CPU, 8G RAM) on a host and 20 VMs with 2C4G on > > another one. But if

Re: [RFC v2 PATCH 4/4] mm: pre zero out free pages to speed up page allocation for __GFP_ZERO

2021-01-05 Thread Liang Li
On Tue, Jan 5, 2021 at 5:30 PM David Hildenbrand wrote: > > On 05.01.21 10:20, Michal Hocko wrote: > > On Mon 04-01-21 15:00:31, Dave Hansen wrote: > >> On 1/4/21 12:11 PM, David Hildenbrand wrote: > Yeah, it certainly can't be the default, but it *is* useful for > thing where we know

Re: [RFC v2 PATCH 0/4] speed up page allocation for __GFP_ZERO

2021-01-04 Thread Liang Li
> >>> In our production environment, there are three main applications have such > >>> requirement, one is QEMU [creating a VM with SR-IOV passthrough device], > >>> anther other two are DPDK related applications, DPDK OVS and SPDK vhost, > >>> for best performance, they populate memory when

Re: [RFC v2 PATCH 0/4] speed up page allocation for __GFP_ZERO

2021-01-04 Thread Liang Li
On Mon, Jan 4, 2021 at 8:56 PM Michal Hocko wrote: > > On Mon 21-12-20 11:25:22, Liang Li wrote: > [...] > > Security > > > > This is a weak version of "introduce init_on_alloc=1 and init_on_free=1 > > boot options", which zero out page in a asy

Re: [RFC v2 PATCH 0/4] speed up page allocation for __GFP_ZERO

2021-01-04 Thread Liang Li
> > Win or not depends on its effect. For our case, it solves the issue > > that we faced, so it can be thought as a win for us. If others don't > > have the issue we faced, the result will be different, maybe they will > > be affected by the side effect of this feature. I think this is your > >

Re: [RFC PATCH 1/3] mm: support hugetlb free page reporting

2020-12-23 Thread Liang Li
> >>> +static int > >>> +hugepage_reporting_cycle(struct page_reporting_dev_info *prdev, > >>> + struct hstate *h, unsigned int nid, > >>> + struct scatterlist *sgl, unsigned int *offset) > >>> +{ > >>> + struct list_head *list =

Re: [RFC PATCH 1/3] mm: support hugetlb free page reporting

2020-12-23 Thread Liang Li
> > > > + spin_lock_irq(_lock); > > > > + > > > > + if (huge_page_order(h) > MAX_ORDER) > > > > + budget = HUGEPAGE_REPORTING_CAPACITY; > > > > + else > > > > + budget = HUGEPAGE_REPORTING_CAPACITY * 32; > > > > > > Wouldn't huge_page_order always be

Re: [RFC v2 PATCH 0/4] speed up page allocation for __GFP_ZERO

2020-12-23 Thread Liang Li
On Wed, Dec 23, 2020 at 4:41 PM David Hildenbrand wrote: > > [...] > > >> I was rather saying that for security it's of little use IMHO. > >> Application/VM start up time might be improved by using huge pages (and > >> pre-zeroing these). Free page reporting might be improved by using > >>

Re: [RFC PATCH 1/3] mm: support hugetlb free page reporting

2020-12-22 Thread Liang Li
> On 12/21/20 11:46 PM, Liang Li wrote: > > Free page reporting only supports buddy pages, it can't report the > > free pages reserved for hugetlbfs case. On the other hand, hugetlbfs > > is a good choice for a system with a huge amount of RAM, because it > > ca

Re: [RFC PATCH 1/3] mm: support hugetlb free page reporting

2020-12-22 Thread Liang Li
> On 12/22/20 11:59 AM, Alexander Duyck wrote: > > On Mon, Dec 21, 2020 at 11:47 PM Liang Li > > wrote: > >> + > >> + if (huge_page_order(h) > MAX_ORDER) > >> + budget = HUGEPAGE_REPORTING_CAPACITY; > >> + else >

Re: [RFC PATCH 1/3] mm: support hugetlb free page reporting

2020-12-22 Thread Liang Li
> > +hugepage_reporting_cycle(struct page_reporting_dev_info *prdev, > > +struct hstate *h, unsigned int nid, > > +struct scatterlist *sgl, unsigned int *offset) > > +{ > > + struct list_head *list = >hugepage_freelists[nid]; > > +

Re: [RFC v2 PATCH 0/4] speed up page allocation for __GFP_ZERO

2020-12-22 Thread Liang Li
> > = > > QEMU use 4K pages, THP is off > > round1 round2 round3 > > w/o this patch:23.5s 24.7s 24.6s > > w/ this patch: 10.2s 10.3s 11.2s > > > > QEMU use 4K pages, THP is on > >

Re: [RFC v2 PATCH 0/4] speed up page allocation for __GFP_ZERO

2020-12-22 Thread Liang Li
https://static.sched.com/hosted_files/kvmforum2020/51/The%20Practice%20Method%20to%20Speed%20Up%2010x%20Boot-up%20Time%20for%20Guest%20in%20Alibaba%20Cloud.pdf > > > > and the flowing link is mine: > >

Re: [RFC PATCH 3/3] mm: support free hugepage pre zero out

2020-12-22 Thread Liang Li
> > Free page reporting in virtio-balloon doesn't give you any guarantees > > regarding zeroing of pages. Take a look at the QEMU implementation - > > e.g., with vfio all reports are simply ignored. > > > > Also, I am not sure if mangling such details ("zeroing of pages") into > > the page

Re: [RFC PATCH 2/3] virtio-balloon: add support for providing free huge page reports to host

2020-12-22 Thread Liang Li
On Tue, Dec 22, 2020 at 4:28 PM David Hildenbrand wrote: > > On 22.12.20 08:48, Liang Li wrote: > > Free page reporting only supports buddy pages, it can't report the > > free pages reserved for hugetlbfs case. On the other hand, hugetlbfs > > The virtio-balloon free

Re: [RFC v2 PATCH 0/4] speed up page allocation for __GFP_ZERO

2020-12-22 Thread Liang Li
On Tue, Dec 22, 2020 at 4:47 PM David Hildenbrand wrote: > > On 21.12.20 17:25, Liang Li wrote: > > The first version can be found at: https://lkml.org/lkml/2020/4/12/42 > > > > Zero out the page content usually happens when allocating pages with > > the flag

[RFC PATCH 0/3 updated] add support for free hugepage reporting

2020-12-22 Thread Liang Li
for the virtio spec are needed. Before that, I need the feedback of the comunity about this new feature. This RFC is baed on my previous series: '[RFC v2 PATCH 0/4] speed up page allocation for __GFP_ZERO' Liang Li (3): mm: support hugetlb free page reporting virtio-balloon: add support

[RFC PATCH 0/3 updated] add support for free hugepage reporting

2020-12-22 Thread Liang Li
for the virtio spec are needed. Before that, I need the feedback of the comunity about this new feature. This RFC is baed on my previous series: '[RFC v2 PATCH 0/4] speed up page allocation for __GFP_ZERO' Liang Li (3): mm: support hugetlb free page reporting virtio-balloon: add support

[RFC PATCH 3/3] mm: support free hugepage pre zero out

2020-12-21 Thread Liang Li
Williamson Cc: Michael S. Tsirkin Cc: Jason Wang Cc: Mike Kravetz Cc: Liang Li Signed-off-by: Liang Li --- mm/page_prezero.c | 17 + 1 file changed, 17 insertions(+) diff --git a/mm/page_prezero.c b/mm/page_prezero.c index c8ce720bfc54..dff4e0adf402 100644 --- a/mm

[RFC PATCH 2/3] virtio-balloon: add support for providing free huge page reports to host

2020-12-21 Thread Liang Li
: Alex Williamson Cc: Michael S. Tsirkin Cc: Jason Wang Cc: Mike Kravetz Cc: Liang Li Signed-off-by: Liang Li --- drivers/virtio/virtio_balloon.c | 61 + include/uapi/linux/virtio_balloon.h | 1 + 2 files changed, 62 insertions(+) diff --git a/drivers/virtio

[RFC PATCH 1/3] mm: support hugetlb free page reporting

2020-12-21 Thread Liang Li
Hildenbrand Cc: Michal Hocko Cc: Andrew Morton Cc: Alex Williamson Cc: Michael S. Tsirkin Cc: Jason Wang Cc: Mike Kravetz Cc: Liang Li Signed-off-by: Liang Li --- include/linux/hugetlb.h| 3 + include/linux/page_reporting.h | 5 + mm/hugetlb.c | 29 mm

[RFC PATCH 0/3] add support for free hugepage reporting

2020-12-21 Thread Liang Li
, changes for the virtio spec are needed. Before that, I need the feedback of the comunity about this new feature. Liang Li (3): mm: support hugetlb free page reporting virtio-balloon: add support for providing free huge page reports to host mm: support free hugepage pre zero out

[RFC v2 PATCH 0/4] speed up page allocation for __GFP_ZERO

2020-12-21 Thread Liang Li
to reduce cache pollution. To make the whole function works, support of pre zero out free huge pages should be added for hugetlbfs, I will send another patch for it. Liang Li (4): mm: let user decide page reporting option mm: pre zero out free pages to speed up page allocation for __GFP_ZERO

[RFC v2 PATCH 3/4] mm: let user decide page reporting option

2020-12-21 Thread Liang Li
Hildenbrand Cc: Michal Hocko Cc: Andrew Morton Cc: Alex Williamson Cc: Michael S. Tsirkin Signed-off-by: Liang Li --- drivers/virtio/virtio_balloon.c | 3 +++ include/linux/page_reporting.h | 3 +++ mm/page_reporting.c | 18 ++ mm/page_reporting.h

[RFC v2 PATCH 4/4] mm: pre zero out free pages to speed up page allocation for __GFP_ZERO

2020-12-21 Thread Liang Li
nbrand Cc: Michal Hocko Cc: Andrew Morton Cc: Alex Williamson Cc: Michael S. Tsirkin Signed-off-by: Liang Li --- include/linux/highmem.h| 31 +++- include/linux/page-flags.h | 16 +- include/trace/events/mmflags.h | 7 + mm/Kconfig | 10 ++ m

[RFC v2 PATCH 2/4] mm: Add batch size for free page reporting

2020-12-21 Thread Liang Li
to control the waking up of reporting worker. Cc: Alexander Duyck Cc: Mel Gorman Cc: Andrea Arcangeli Cc: Dan Williams Cc: Dave Hansen Cc: David Hildenbrand Cc: Michal Hocko Cc: Andrew Morton Cc: Alex Williamson Cc: Michael S. Tsirkin Signed-off-by: Liang Li --- mm/page_reporting.c | 2 ++ mm

[RFC v2 PATCH 1/4] mm: make page reporing worker works better for low order page

2020-12-21 Thread Liang Li
voluntarily if needed. Cc: Alexander Duyck Cc: Mel Gorman Cc: Andrea Arcangeli Cc: Dan Williams Cc: Dave Hansen Cc: David Hildenbrand Cc: Michal Hocko Cc: Andrew Morton Cc: Alex Williamson Cc: Michael S. Tsirkin Signed-off-by: Liang Li --- mm/page_reporting.c | 35

[PATCH RFC 4/4] VMX: Expose the LA57 feature to VM

2016-12-29 Thread Liang Li
This patch exposes 5 level page table feature to the VM, at the same time, the canonical virtual address checking is extended to support both 48-bits and 57-bits address width, it's the prerequisite to support 5 level paging guest. Signed-off-by: Liang Li <liang.z...@intel.com> Cc:

[PATCH RFC 4/4] VMX: Expose the LA57 feature to VM

2016-12-29 Thread Liang Li
This patch exposes 5 level page table feature to the VM, at the same time, the canonical virtual address checking is extended to support both 48-bits and 57-bits address width, it's the prerequisite to support 5 level paging guest. Signed-off-by: Liang Li Cc: Thomas Gleixner Cc: Ingo Molnar Cc

[PATCH RFC 3/4] KVM: MMU: Add 5 level EPT & Shadow page table support.

2016-12-29 Thread Liang Li
once the hardware supports, and this is not a good choice because 5 level EPT requires more memory access comparing to use 4 level EPT. The right thing is to use 5 level EPT only when it's needed, will change in the future version. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Thomas Gl

[PATCH RFC 2/4] KVM: MMU: Rename PT64_ROOT_LEVEL to PT64_ROOT_4LEVEL

2016-12-29 Thread Liang Li
Now we have 4 level page table and 5 level page table in 64 bits long mode, let's rename the PT64_ROOT_LEVEL to PT64_ROOT_4LEVEL, then we can use PT64_ROOT_5LEVEL for 5 level page table, it's helpful to make the code more clear. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Thomas Gl

[PATCH RFC 3/4] KVM: MMU: Add 5 level EPT & Shadow page table support.

2016-12-29 Thread Liang Li
once the hardware supports, and this is not a good choice because 5 level EPT requires more memory access comparing to use 4 level EPT. The right thing is to use 5 level EPT only when it's needed, will change in the future version. Signed-off-by: Liang Li Cc: Thomas Gleixner Cc: Ingo Molnar Cc

[PATCH RFC 2/4] KVM: MMU: Rename PT64_ROOT_LEVEL to PT64_ROOT_4LEVEL

2016-12-29 Thread Liang Li
Now we have 4 level page table and 5 level page table in 64 bits long mode, let's rename the PT64_ROOT_LEVEL to PT64_ROOT_4LEVEL, then we can use PT64_ROOT_5LEVEL for 5 level page table, it's helpful to make the code more clear. Signed-off-by: Liang Li Cc: Thomas Gleixner Cc: Ingo Molnar Cc

[PATCH RFC 0/4] 5-level EPT

2016-12-29 Thread Liang Li
level page table, with both the EPT and shadow page support. I just covered the booting process, the guest can boot successfully. Some parts of this patchset can be improved. Any comments on the design or the patches would be appreciated. Liang Li (4): x86: Add the new CPUID and CR4 bits for 5

[PATCH RFC 1/4] x86: Add the new CPUID and CR4 bits for 5 level page table

2016-12-29 Thread Liang Li
-by: Liang Li <liang.z...@intel.com> Cc: Thomas Gleixner <t...@linutronix.de> Cc: Ingo Molnar <mi...@redhat.com> Cc: Kirill A. Shutemov <kirill.shute...@linux.intel.com> Cc: Dave Hansen <dave.han...@linux.intel.com> Cc: Xiao Guangrong <guangrong.x...@linux.in

[PATCH RFC 1/4] x86: Add the new CPUID and CR4 bits for 5 level page table

2016-12-29 Thread Liang Li
-by: Liang Li Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Kirill A. Shutemov Cc: Dave Hansen Cc: Xiao Guangrong Cc: Paolo Bonzini Cc: "Radim Kr??m" --- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/uapi/asm/processor-flags.h | 2 ++ 2 files changed, 3 insertion

[PATCH RFC 0/4] 5-level EPT

2016-12-29 Thread Liang Li
level page table, with both the EPT and shadow page support. I just covered the booting process, the guest can boot successfully. Some parts of this patchset can be improved. Any comments on the design or the patches would be appreciated. Liang Li (4): x86: Add the new CPUID and CR4 bits for 5

[PATCH v6 kernel 2/5] virtio-balloon: define new feature bit and head struct

2016-12-20 Thread Liang Li
Add a new feature which supports sending the page information with range array. The current implementation uses PFNs array, which is not very efficient. Using ranges can improve the performance of inflating/deflating significantly. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Mic

[PATCH v6 kernel 4/5] virtio-balloon: define flags and head for host request vq

2016-12-20 Thread Liang Li
. And the hypervisor can get some of guest's runtime information through this virtual queue too, e.g. the guest's unused page information, which can be used for live migration optimization. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Mel

[PATCH v6 kernel 2/5] virtio-balloon: define new feature bit and head struct

2016-12-20 Thread Liang Li
Add a new feature which supports sending the page information with range array. The current implementation uses PFNs array, which is not very efficient. Using ranges can improve the performance of inflating/deflating significantly. Signed-off-by: Liang Li Cc: Michael S. Tsirkin Cc: Paolo

[PATCH v6 kernel 4/5] virtio-balloon: define flags and head for host request vq

2016-12-20 Thread Liang Li
. And the hypervisor can get some of guest's runtime information through this virtual queue too, e.g. the guest's unused page information, which can be used for live migration optimization. Signed-off-by: Liang Li Cc: Andrew Morton Cc: Mel Gorman Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia

[PATCH v6 kernel 5/5] virtio-balloon: tell host vm's unused page info

2016-12-20 Thread Liang Li
pages, this is very helpful to reduce the network traffic and speed up the live migration process. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Mel Gorman <mgor...@techsingularity.net> Cc: Michael S. Tsirkin <m...@redhat.c

[PATCH v6 kernel 5/5] virtio-balloon: tell host vm's unused page info

2016-12-20 Thread Liang Li
pages, this is very helpful to reduce the network traffic and speed up the live migration process. Signed-off-by: Liang Li Cc: Andrew Morton Cc: Mel Gorman Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah Cc: Dave Hansen Cc: Andrea Arcangeli Cc: David Hildenbrand

[PATCH v6 kernel 1/5] virtio-balloon: rework deflate to add page to a list

2016-12-20 Thread Liang Li
pfn|length} down the road. balloon_pfn_to_page() can be removed because it's useless. Signed-off-by: Liang Li <liang.z...@intel.com> Signed-off-by: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bonzini <pbonz...@redhat.com> Cc: Cornelia Huck <cornelia.h...@de.ibm.com> Cc: Amit

[PATCH v6 kernel 1/5] virtio-balloon: rework deflate to add page to a list

2016-12-20 Thread Liang Li
pfn|length} down the road. balloon_pfn_to_page() can be removed because it's useless. Signed-off-by: Liang Li Signed-off-by: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah Cc: Dave Hansen Cc: Andrea Arcangeli Cc: David Hildenbrand --- drivers/virtio/virtio_balloon.c |

[PATCH v6 kernel 3/5] virtio-balloon: speed up inflate/deflate process

2016-12-20 Thread Liang Li
new feature, inflating the balloon to 7GB of a 8GB idle guest only takes 590ms, the performance improvement is about 85%. TODO: optimize stage a by allocating/freeing a chunk of pages instead of a single page at a time. Signed-off-by: Liang Li <liang.z...@intel.com> Suggested-by: Michael S.

[PATCH v6 kernel 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-12-20 Thread Liang Li
* Use a unified way to send the free page information with the bitmap * Address the issues referred in MST's comments Liang Li (5): virtio-balloon: rework deflate to add page to a list virtio-balloon: define new feature bit and head struct virtio-balloon: speed up inflate/deflate process

[PATCH v6 kernel 3/5] virtio-balloon: speed up inflate/deflate process

2016-12-20 Thread Liang Li
new feature, inflating the balloon to 7GB of a 8GB idle guest only takes 590ms, the performance improvement is about 85%. TODO: optimize stage a by allocating/freeing a chunk of pages instead of a single page at a time. Signed-off-by: Liang Li Suggested-by: Michael S. Tsirkin Cc: Michael S. Tsirkin

[PATCH v6 kernel 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-12-20 Thread Liang Li
* Use a unified way to send the free page information with the bitmap * Address the issues referred in MST's comments Liang Li (5): virtio-balloon: rework deflate to add page to a list virtio-balloon: define new feature bit and head struct virtio-balloon: speed up inflate/deflate process

[PATCH kernel v5 2/5] virtio-balloon: define new feature bit and head struct

2016-11-30 Thread Liang Li
about the page bitmap. e.g. the page size, page bitmap length and start pfn. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bonzini <pbonz...@redhat.com> Cc: Cornelia Huck <cornelia.h...@de.ibm.com> Cc: Amit Shah <ami

[PATCH kernel v5 5/5] virtio-balloon: tell host vm's unused page info

2016-11-30 Thread Liang Li
pages, this is very helpful to reduce the network traffic and speed up the live migration process. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Mel Gorman <mgor...@techsingularity.net> Cc: Michael S. Tsirkin <m...@redhat.c

[PATCH kernel v5 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-11-30 Thread Liang Li
e bitmap * Address the issues referred in MST's comments Liang Li (5): virtio-balloon: rework deflate to add page to a list virtio-balloon: define new feature bit and head struct virtio-balloon: speed up inflate/deflate process virtio-balloon: define flags and head for host request vq virtio

[PATCH kernel v5 4/5] virtio-balloon: define flags and head for host request vq

2016-11-30 Thread Liang Li
. And the hypervisor can get some of guest's runtime information through this virtual queue too, e.g. the guest's unused page information, which can be used for live migration optimization. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Mel

[PATCH kernel v5 2/5] virtio-balloon: define new feature bit and head struct

2016-11-30 Thread Liang Li
about the page bitmap. e.g. the page size, page bitmap length and start pfn. Signed-off-by: Liang Li Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah Cc: Dave Hansen --- include/uapi/linux/virtio_balloon.h | 19 +++ 1 file changed, 19 insertions(+) diff

[PATCH kernel v5 5/5] virtio-balloon: tell host vm's unused page info

2016-11-30 Thread Liang Li
pages, this is very helpful to reduce the network traffic and speed up the live migration process. Signed-off-by: Liang Li Cc: Andrew Morton Cc: Mel Gorman Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah Cc: Dave Hansen --- drivers/virtio/virtio_balloon.c | 126

[PATCH kernel v5 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-11-30 Thread Liang Li
e bitmap * Address the issues referred in MST's comments Liang Li (5): virtio-balloon: rework deflate to add page to a list virtio-balloon: define new feature bit and head struct virtio-balloon: speed up inflate/deflate process virtio-balloon: define flags and head for host request vq virtio

[PATCH kernel v5 4/5] virtio-balloon: define flags and head for host request vq

2016-11-30 Thread Liang Li
. And the hypervisor can get some of guest's runtime information through this virtual queue too, e.g. the guest's unused page information, which can be used for live migration optimization. Signed-off-by: Liang Li Cc: Andrew Morton Cc: Mel Gorman Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia

[PATCH kernel v5 1/5] virtio-balloon: rework deflate to add page to a list

2016-11-30 Thread Liang Li
ead of the PFN array, which will allow faster notifications using a bitmap down the road. balloon_pfn_to_page() can be removed because it's useless. Signed-off-by: Liang Li <liang.z...@intel.com> Signed-off-by: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bonzini <pbonz...@redhat.com&

[PATCH kernel v5 3/5] virtio-balloon: speed up inflate/deflate process

2016-11-30 Thread Liang Li
ture, inflating the balloon to 7GB of a 8GB idle guest only takes 590ms, the performance improvement is about 85%. TODO: optimize stage a by allocating/freeing a chunk of pages instead of a single page at a time. Signed-off-by: Liang Li <liang.z...@intel.com> Suggested-by: Michael S. Tsirkin <m.

[PATCH kernel v5 1/5] virtio-balloon: rework deflate to add page to a list

2016-11-30 Thread Liang Li
ead of the PFN array, which will allow faster notifications using a bitmap down the road. balloon_pfn_to_page() can be removed because it's useless. Signed-off-by: Liang Li Signed-off-by: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah Cc: Dave Hansen --- drivers/vir

[PATCH kernel v5 3/5] virtio-balloon: speed up inflate/deflate process

2016-11-30 Thread Liang Li
ture, inflating the balloon to 7GB of a 8GB idle guest only takes 590ms, the performance improvement is about 85%. TODO: optimize stage a by allocating/freeing a chunk of pages instead of a single page at a time. Signed-off-by: Liang Li Suggested-by: Michael S. Tsirkin Cc: Michael S. Tsirkin Cc: P

[PATCH kernel v4 4/7] virtio-balloon: speed up inflate/deflate process

2016-11-02 Thread Liang Li
ture, inflating the balloon to 7GB of a 8GB idle guest only takes 590ms, the performance improvement is about 85%. TODO: optimize stage a by allocating/freeing a chunk of pages instead of a single page at a time. Signed-off-by: Liang Li <liang.z...@intel.com> Suggested-by: Michael S. Tsirkin <m.

[PATCH kernel v4 4/7] virtio-balloon: speed up inflate/deflate process

2016-11-02 Thread Liang Li
ture, inflating the balloon to 7GB of a 8GB idle guest only takes 590ms, the performance improvement is about 85%. TODO: optimize stage a by allocating/freeing a chunk of pages instead of a single page at a time. Signed-off-by: Liang Li Suggested-by: Michael S. Tsirkin Cc: Michael S. Tsirkin Cc: P

[PATCH kernel v4 6/7] virtio-balloon: define flags and head for host request vq

2016-11-02 Thread Liang Li
. And the hypervisor can get some of guest's runtime information through this virtual queue too, e.g. the guest's unused page information, which can be used for live migration optimization. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Mel

[PATCH kernel v4 5/7] mm: add the related functions to get unused page

2016-11-02 Thread Liang Li
can be corrected by the dirty page logging mechanism. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Andrew Morton <a...@linux-foundation.org> Cc: Mel Gorman <mgor...@techsingularity.net> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bonzini <pbonz...@re

[PATCH kernel v4 3/7] mm: add a function to get the max pfn

2016-11-02 Thread Liang Li
Expose the function to get the max pfn, so it can be used in the virtio-balloon device driver. Simply include the 'linux/bootmem.h' is not enough, if the device driver is built to a module, directly refer the max_pfn lead to build failed. Signed-off-by: Liang Li <liang.z...@intel.com> Cc:

[PATCH kernel v4 5/7] mm: add the related functions to get unused page

2016-11-02 Thread Liang Li
can be corrected by the dirty page logging mechanism. Signed-off-by: Liang Li Cc: Andrew Morton Cc: Mel Gorman Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah Cc: Dave Hansen --- include/linux/mm.h | 2 ++ mm/page_alloc.c| 85

[PATCH kernel v4 3/7] mm: add a function to get the max pfn

2016-11-02 Thread Liang Li
Expose the function to get the max pfn, so it can be used in the virtio-balloon device driver. Simply include the 'linux/bootmem.h' is not enough, if the device driver is built to a module, directly refer the max_pfn lead to build failed. Signed-off-by: Liang Li Cc: Andrew Morton Cc: Mel Gorman

[PATCH kernel v4 6/7] virtio-balloon: define flags and head for host request vq

2016-11-02 Thread Liang Li
. And the hypervisor can get some of guest's runtime information through this virtual queue too, e.g. the guest's unused page information, which can be used for live migration optimization. Signed-off-by: Liang Li Cc: Andrew Morton Cc: Mel Gorman Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia

[PATCH kernel v4 7/7] virtio-balloon: tell host vm's unused page info

2016-11-02 Thread Liang Li
process. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bonzini <pbonz...@redhat.com> Cc: Cornelia Huck <cornelia.h...@de.ibm.com> Cc: Amit Shah <amit.s...@redhat.com> Cc: Dave Hansen <dave.han...@intel.com> --- d

[PATCH kernel v4 7/7] virtio-balloon: tell host vm's unused page info

2016-11-02 Thread Liang Li
process. Signed-off-by: Liang Li Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah Cc: Dave Hansen --- drivers/virtio/virtio_balloon.c | 128 +--- 1 file changed, 121 insertions(+), 7 deletions(-) diff --git a/drivers/virtio

[PATCH kernel v4 0/7] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-11-02 Thread Liang Li
api head file. * Use a new way to determine the page bitmap size. * Use a unified way to send the free page information with the bitmap * Address the issues referred in MST's comments Liang Li (7): virtio-balloon: rework deflate to add page to a list virtio-balloon: define new fea

[PATCH kernel v4 2/7] virtio-balloon: define new feature bit and head struct

2016-11-02 Thread Liang Li
about the page bitmap. e.g. the page size, page bitmap length and start pfn. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bonzini <pbonz...@redhat.com> Cc: Cornelia Huck <cornelia.h...@de.ibm.com> Cc: Amit Shah <ami

[PATCH kernel v4 0/7] Extend virtio-balloon for fast (de)inflating & fast live migration

2016-11-02 Thread Liang Li
api head file. * Use a new way to determine the page bitmap size. * Use a unified way to send the free page information with the bitmap * Address the issues referred in MST's comments Liang Li (7): virtio-balloon: rework deflate to add page to a list virtio-balloon: define new fea

[PATCH kernel v4 2/7] virtio-balloon: define new feature bit and head struct

2016-11-02 Thread Liang Li
about the page bitmap. e.g. the page size, page bitmap length and start pfn. Signed-off-by: Liang Li Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah Cc: Dave Hansen --- include/uapi/linux/virtio_balloon.h | 19 +++ 1 file changed, 19 insertions(+) diff

[PATCH kernel v4 1/7] virtio-balloon: rework deflate to add page to a list

2016-11-02 Thread Liang Li
ead of the PFN array, which will allow faster notifications using a bitmap down the road. balloon_pfn_to_page() can be removed because it's useless. Signed-off-by: Liang Li <liang.z...@intel.com> Signed-off-by: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bonzini <pbonz...@redhat.com&

[PATCH kernel v4 1/7] virtio-balloon: rework deflate to add page to a list

2016-11-02 Thread Liang Li
ead of the PFN array, which will allow faster notifications using a bitmap down the road. balloon_pfn_to_page() can be removed because it's useless. Signed-off-by: Liang Li Signed-off-by: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah Cc: Dave Hansen --- drivers/vir

[RESEND PATCH v3 kernel 6/7] virtio-balloon: define feature bit and head for misc virt queue

2016-10-21 Thread Liang Li
. And the VMM hypervisor can get some of guest's runtime information through this virtual queue, e.g. the guest's unused page information, which can be used for live migration optimization. Signed-off-by: Liang Li <liang.z...@intel.com> Cc: Michael S. Tsirkin <m...@redhat.com> Cc: Paolo Bo

[RESEND PATCH v3 kernel 4/7] virtio-balloon: speed up inflate/deflate process

2016-10-21 Thread Liang Li
ture, inflating the balloon to 7GB of a 8GB idle guest only takes 590ms, the performance improvement is about 85%. TODO: optimize stage a by allocating/freeing a chunk of pages instead of a single page at a time. Signed-off-by: Liang Li <liang.z...@intel.com> Suggested-by: Michael S. Tsirkin <m.

[RESEND PATCH v3 kernel 3/7] mm: add a function to get the max pfn

2016-10-21 Thread Liang Li
Expose the function to get the max pfn, so it can be used in the virtio-balloon device driver. Simply include the 'linux/bootmem.h' is not enough, if the device driver is built to a module, directly refer the max_pfn lead to build failed. Signed-off-by: Liang Li <liang.z...@intel.com> Cc:

[RESEND PATCH v3 kernel 7/7] virtio-balloon: tell host vm's unused page info

2016-10-21 Thread Liang Li
Support the request for vm's unused page information, response with a page bitmap. QEMU can make use of this bitmap and the dirty page logging mechanism to skip the transportation of these unused pages, this is very helpful to speed up the live migration process. Signed-off-by: Liang Li <lian

[RESEND PATCH v3 kernel 6/7] virtio-balloon: define feature bit and head for misc virt queue

2016-10-21 Thread Liang Li
. And the VMM hypervisor can get some of guest's runtime information through this virtual queue, e.g. the guest's unused page information, which can be used for live migration optimization. Signed-off-by: Liang Li Cc: Michael S. Tsirkin Cc: Paolo Bonzini Cc: Cornelia Huck Cc: Amit Shah --- include/uapi

[RESEND PATCH v3 kernel 4/7] virtio-balloon: speed up inflate/deflate process

2016-10-21 Thread Liang Li
ture, inflating the balloon to 7GB of a 8GB idle guest only takes 590ms, the performance improvement is about 85%. TODO: optimize stage a by allocating/freeing a chunk of pages instead of a single page at a time. Signed-off-by: Liang Li Suggested-by: Michael S. Tsirkin Cc: Michael S. Tsirkin Cc: P

[RESEND PATCH v3 kernel 3/7] mm: add a function to get the max pfn

2016-10-21 Thread Liang Li
Expose the function to get the max pfn, so it can be used in the virtio-balloon device driver. Simply include the 'linux/bootmem.h' is not enough, if the device driver is built to a module, directly refer the max_pfn lead to build failed. Signed-off-by: Liang Li Cc: Andrew Morton Cc: Mel Gorman

[RESEND PATCH v3 kernel 7/7] virtio-balloon: tell host vm's unused page info

2016-10-21 Thread Liang Li
Support the request for vm's unused page information, response with a page bitmap. QEMU can make use of this bitmap and the dirty page logging mechanism to skip the transportation of these unused pages, this is very helpful to speed up the live migration process. Signed-off-by: Liang Li Cc

  1   2   3   >