RE: [virtio-dev] Re: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process

2016-08-01 Thread Li, Liang Z
> > It's only small because it makes you rescan the free list. > > So maybe you should do something else. > > I looked at it a bit. Instead of scanning the free list, how about > > scanning actual page structures? If page is unused, pass it to host. > > Solves the problem of rescanning multiple tim

Re: [virtio-dev] Re: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process

2016-07-29 Thread Dave Hansen
On 07/28/2016 02:51 PM, Michael S. Tsirkin wrote: >> > If 1MB is too big, how about 512K, or 256K? 32K seems too small. >> > > It's only small because it makes you rescan the free list. > So maybe you should do something else. > I looked at it a bit. Instead of scanning the free list, how about >

RE: [virtio-dev] Re: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process

2016-07-28 Thread Li, Liang Z
> > > On Wed, Jul 27, 2016 at 09:03:21AM -0700, Dave Hansen wrote: > > > > On 07/26/2016 06:23 PM, Liang Li wrote: > > > > > + vb->pfn_limit = VIRTIO_BALLOON_PFNS_LIMIT; > > > > > + vb->pfn_limit = min(vb->pfn_limit, get_max_pfn()); > > > > > + vb->bmap_len = ALIGN(vb->pfn_limit, BITS_P

RE: [virtio-dev] Re: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process

2016-07-28 Thread Li, Liang Z
> On Thu, Jul 28, 2016 at 06:36:18AM +, Li, Liang Z wrote: > > > > > This ends up doing a 1MB kmalloc() right? That seems a _bit_ big. > > > > > How big was the pfn buffer before? > > > > > > > > Yes, it is if the max pfn is more than 32GB. > > > > The size of the pfn buffer use before is 256*

RE: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process

2016-07-28 Thread Li, Liang Z
> On Thu, Jul 28, 2016 at 03:06:37AM +, Li, Liang Z wrote: > > > > + * VIRTIO_BALLOON_PFNS_LIMIT is used to limit the size of page > > > > +bitmap > > > > + * to prevent a very large page bitmap, there are two reasons for this: > > > > + * 1) to save memory. > > > > + * 2) allocate a large bitm

Re: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process

2016-07-28 Thread Michael S. Tsirkin
On Thu, Jul 28, 2016 at 03:06:37AM +, Li, Liang Z wrote: > > > + * VIRTIO_BALLOON_PFNS_LIMIT is used to limit the size of page bitmap > > > + * to prevent a very large page bitmap, there are two reasons for this: > > > + * 1) to save memory. > > > + * 2) allocate a large bitmap may fail. > > >

Re: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process

2016-07-28 Thread Michael S. Tsirkin
On Thu, Jul 28, 2016 at 03:30:09AM +, Li, Liang Z wrote: > > Subject: Re: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate > > process > > > > On Wed, Jul 27, 2016 at 09:03:21AM -0700, Dave Hansen wrote: > > > On 07/26/2016 06:23 PM, Liang Li wro

Re: [virtio-dev] Re: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process

2016-07-28 Thread Michael S. Tsirkin
On Thu, Jul 28, 2016 at 06:36:18AM +, Li, Liang Z wrote: > > > > This ends up doing a 1MB kmalloc() right? That seems a _bit_ big. > > > > How big was the pfn buffer before? > > > > > > Yes, it is if the max pfn is more than 32GB. > > > The size of the pfn buffer use before is 256*4 = 1024 Byt

RE: [virtio-dev] Re: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process

2016-07-27 Thread Li, Liang Z
> > > This ends up doing a 1MB kmalloc() right? That seems a _bit_ big. > > > How big was the pfn buffer before? > > > > Yes, it is if the max pfn is more than 32GB. > > The size of the pfn buffer use before is 256*4 = 1024 Bytes, it's too > > small, and it's the main reason for bad performance. >

RE: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process

2016-07-27 Thread Li, Liang Z
> > +/* > > + * VIRTIO_BALLOON_PFNS_LIMIT is used to limit the size of page bitmap > > + * to prevent a very large page bitmap, there are two reasons for this: > > + * 1) to save memory. > > + * 2) allocate a large bitmap may fail. > > + * > > + * The actual limit of pfn is determined by: > > + * p

RE: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process

2016-07-27 Thread Li, Liang Z
> Subject: Re: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate > process > > On Wed, Jul 27, 2016 at 09:03:21AM -0700, Dave Hansen wrote: > > On 07/26/2016 06:23 PM, Liang Li wrote: > > > + vb->pfn_limit = VIRTIO_BALLOON_PFNS_LIMIT; > > &

RE: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process

2016-07-27 Thread Li, Liang Z
> > + * VIRTIO_BALLOON_PFNS_LIMIT is used to limit the size of page bitmap > > + * to prevent a very large page bitmap, there are two reasons for this: > > + * 1) to save memory. > > + * 2) allocate a large bitmap may fail. > > + * > > + * The actual limit of pfn is determined by: > > + * pfn_limit

Re: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process

2016-07-27 Thread Michael S. Tsirkin
On Thu, Jul 28, 2016 at 01:13:35AM +, Li, Liang Z wrote: > > Subject: Re: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate > > process > > > > On 07/26/2016 06:23 PM, Liang Li wrote: > > > + vb->pfn_limit = VIRTIO_BALLOON_PFNS_LIMIT; >

RE: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process

2016-07-27 Thread Li, Liang Z
> Subject: Re: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate > process > > On 07/26/2016 06:23 PM, Liang Li wrote: > > + vb->pfn_limit = VIRTIO_BALLOON_PFNS_LIMIT; > > + vb->pfn_limit = min(vb->pfn_limit, get_max_pfn()); > >

Re: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process

2016-07-27 Thread Michael S. Tsirkin
On Wed, Jul 27, 2016 at 09:23:33AM +0800, Liang Li wrote: > The implementation of the current virtio-balloon is not very > efficient, the time spends on different stages of inflating > the balloon to 7GB of a 8GB idle guest: > > a. allocating pages (6.5%) > b. sending PFNs to host (68.3%) > c. add

Re: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process

2016-07-27 Thread Michael S. Tsirkin
On Wed, Jul 27, 2016 at 09:03:21AM -0700, Dave Hansen wrote: > On 07/26/2016 06:23 PM, Liang Li wrote: > > + vb->pfn_limit = VIRTIO_BALLOON_PFNS_LIMIT; > > + vb->pfn_limit = min(vb->pfn_limit, get_max_pfn()); > > + vb->bmap_len = ALIGN(vb->pfn_limit, BITS_PER_LONG) / > > +BITS_PER

Re: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process

2016-07-27 Thread Michael S. Tsirkin
On Wed, Jul 27, 2016 at 09:23:33AM +0800, Liang Li wrote: > The implementation of the current virtio-balloon is not very > efficient, the time spends on different stages of inflating > the balloon to 7GB of a 8GB idle guest: > > a. allocating pages (6.5%) > b. sending PFNs to host (68.3%) > c. add

Re: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process

2016-07-27 Thread Dave Hansen
On 07/26/2016 06:23 PM, Liang Li wrote: > + vb->pfn_limit = VIRTIO_BALLOON_PFNS_LIMIT; > + vb->pfn_limit = min(vb->pfn_limit, get_max_pfn()); > + vb->bmap_len = ALIGN(vb->pfn_limit, BITS_PER_LONG) / > + BITS_PER_BYTE + 2 * sizeof(unsigned long); > + hdr_len = sizeof(str

[PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process

2016-07-26 Thread Liang Li
The implementation of the current virtio-balloon is not very efficient, the time spends on different stages of inflating the balloon to 7GB of a 8GB idle guest: a. allocating pages (6.5%) b. sending PFNs to host (68.3%) c. address translation (6.1%) d. madvise (19%) It takes about 4126ms for the