Re: [PATCH v11 4/6] mm: function to offer a page block on the free list

2017-06-20 Thread Rik van Riel
On Tue, 2017-06-20 at 21:26 +0300, Michael S. Tsirkin wrote: > On Tue, Jun 20, 2017 at 01:29:00PM -0400, Rik van Riel wrote: > > I agree with that.  Let me go into some more detail of > > what Nitesh is implementing: > > > > 1) In arch_free_page, the being-freed page i

Re: [PATCH v11 4/6] mm: function to offer a page block on the free list

2017-06-20 Thread Rik van Riel
On Tue, 2017-06-20 at 18:49 +0200, David Hildenbrand wrote: > On 20.06.2017 18:44, Rik van Riel wrote: > > Nitesh Lal (on the CC list) is working on a way > > to efficiently batch recently freed pages for > > free page hinting to the hypervisor. > > > > If th

Re: [PATCH v11 4/6] mm: function to offer a page block on the free list

2017-06-20 Thread Rik van Riel
On Mon, 2017-06-12 at 07:10 -0700, Dave Hansen wrote: > The hypervisor is going to throw away the contents of these pages, > right?  As soon as the spinlock is released, someone can allocate a > page, and put good data in it.  What keeps the hypervisor from > throwing > away good data? That looks

Re: [Qemu-devel] [RFC qemu 0/4] A PV solution for live migration optimization

2016-03-09 Thread Rik van Riel
On Wed, 2016-03-09 at 20:04 +0300, Roman Kagan wrote: > On Wed, Mar 09, 2016 at 05:41:39PM +0200, Michael S. Tsirkin wrote: > > On Wed, Mar 09, 2016 at 05:28:54PM +0300, Roman Kagan wrote: > > > For (1) I've been trying to make a point that skipping clean > > > pages is > > > much more likely to re

Re: [PATCH v15 16/16] unfair qspinlock: a queue based unfair lock

2015-04-09 Thread Rik van Riel
On 04/09/2015 10:13 AM, Peter Zijlstra wrote: > On Thu, Apr 09, 2015 at 09:16:24AM -0400, Rik van Riel wrote: >> On 04/09/2015 03:01 AM, Peter Zijlstra wrote: >>> On Wed, Apr 08, 2015 at 02:32:19PM -0400, Waiman Long wrote: >>>> For a virtual guest with the qspinlo

Re: [PATCH v15 16/16] unfair qspinlock: a queue based unfair lock

2015-04-09 Thread Rik van Riel
On 04/09/2015 03:01 AM, Peter Zijlstra wrote: > On Wed, Apr 08, 2015 at 02:32:19PM -0400, Waiman Long wrote: >> For a virtual guest with the qspinlock patch, a simple unfair byte lock >> will be used if PV spinlock is not configured in or the hypervisor >> isn't either KVM or Xen. The byte lock wor

Re: [RFC] Implement Batched (group) ticket lock

2014-05-28 Thread Rik van Riel
On 05/28/2014 06:19 PM, Linus Torvalds wrote: > If somebody has a P4 still, that's likely the worst case by far. I'm sure cmpxchg isn't the only thing making P4 the worst case :) -- All rights reversed ___ Virtualization mailing list Virtualization@li

Re: [RFC] Implement Batched (group) ticket lock

2014-05-28 Thread Rik van Riel
On 05/28/2014 08:16 AM, Raghavendra K T wrote: This patch looks very promising. > TODO: > - we need an intelligent way to nullify the effect of batching for baremetal > (because extra cmpxchg is not required). On (larger?) NUMA systems, the unfairness may be a nice performance benefit, reducing

Re: [PATCH v10 0/5] make balloon pages movable by compaction

2012-09-17 Thread Rik van Riel
On 09/17/2012 06:15 PM, Andrew Morton wrote: On Mon, 17 Sep 2012 13:38:15 -0300 Rafael Aquini wrote: Memory fragmentation introduced by ballooning might reduce significantly the number of 2MB contiguous memory blocks that can be used within a guest, thus imposing performance penalties associat

Re: [PATCH] Add a page cache-backed balloon device driver.

2012-09-10 Thread Rik van Riel
On 09/10/2012 01:37 PM, Mike Waychison wrote: On Mon, Sep 10, 2012 at 5:05 AM, Michael S. Tsirkin wrote: Also can you pls answer Avi's question? How is overcommit managed? Overcommit in our deployments is managed using memory cgroups on the host. This allows us to have very directed polici

Re: [PATCH v9 0/5] make balloon pages movable by compaction

2012-08-26 Thread Rik van Riel
On 08/26/2012 03:58 AM, Michael S. Tsirkin wrote: On Sat, Aug 25, 2012 at 02:24:55AM -0300, Rafael Aquini wrote: Memory fragmentation introduced by ballooning might reduce significantly the number of 2MB contiguous memory blocks that can be used within a guest, thus imposing performance penaltie

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rik van Riel
On 08/23/2012 07:36 PM, Michael S. Tsirkin wrote: ---> virtio-balloon: replace page->lru list with page->private. The point is to free up page->lru for use by compaction. Warning: completely untested, will provide tested version if we agree on this direction. A singly linked list is not goin

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rik van Riel
On 08/23/2012 01:28 PM, Rafael Aquini wrote: What about keep a conter at virtio_balloon structure on how much pages are isolated from balloon's list and check it at leak time? if the counter gets > 0 than we can safely put leak_balloon() to wait until balloon page list gets completely refilled.

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rik van Riel
On 08/23/2012 11:54 AM, Michael S. Tsirkin wrote: On Thu, Aug 23, 2012 at 12:21:29PM -0300, Rafael Aquini wrote: On Thu, Aug 23, 2012 at 04:53:29PM +0300, Michael S. Tsirkin wrote: On Thu, Aug 23, 2012 at 10:06:07AM -0300, Rafael Aquini wrote: On Thu, Aug 23, 2012 at 03:34:32PM +0300, Michael

Re: [PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-15 Thread Rik van Riel
On 08/14/2012 05:38 PM, Michael S. Tsirkin wrote: And even ignoring that, global pointer to a device is an ugly hack and ugly hacks tend to explode. And even ignoring estetics, and if we decide we are fine with a single balloon, it needs to fail gracefully not crash like it does now. Fair eno

Re: [PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-14 Thread Rik van Riel
On 08/14/2012 04:54 PM, Michael S. Tsirkin wrote: To clarify, the global state that this patch adds, is ugly even if we didn't support multiple balloons yet. So I don't think I can accept such a patch. Rusty has a final word here, maybe he thinks differently. Before deciding that "does not sup

Re: [PATCH v6 3/3] mm: add vm event counters for balloon pages compaction

2012-08-08 Thread Rik van Riel
future, we'll be able to see it... Signed-off-by: Rafael Aquini Acked-by: Rik van Riel -- All rights reversed ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 2/3] virtio_balloon: introduce migration primitives to balloon pages

2012-08-08 Thread Rik van Riel
(spinlock): special protection to balloon pages list against concurrent list handling operations; Signed-off-by: Rafael Aquini Reviewed-by: Rik van Riel -- All rights reversed ___ Virtualization mailing list Virtualization

Re: [PATCH v6 1/3] mm: introduce compaction and migration for virtio ballooned pages

2012-08-08 Thread Rik van Riel
: Rafael Aquini Reviewed-by: Rik van Riel -- All rights reversed ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v5 1/3] mm: introduce compaction and migration for virtio ballooned pages

2012-08-06 Thread Rik van Riel
On 08/06/2012 03:00 PM, Rafael Aquini wrote: On Mon, Aug 06, 2012 at 02:38:23PM -0400, Rik van Riel wrote: On 08/06/2012 09:56 AM, Rafael Aquini wrote: @@ -846,6 +861,21 @@ static int unmap_and_move(new_page_t get_new_page, unsigned long private, goto out; rc

Re: [PATCH v5 1/3] mm: introduce compaction and migration for virtio ballooned pages

2012-08-06 Thread Rik van Riel
On 08/06/2012 09:56 AM, Rafael Aquini wrote: @@ -846,6 +861,21 @@ static int unmap_and_move(new_page_t get_new_page, unsigned long private, goto out; rc = __unmap_and_move(page, newpage, force, offlining, mode); + + if (unlikely(is_balloon_page(newpage)&&

Re: [PATCH v4 1/3] mm: introduce compaction and migration for virtio ballooned pages

2012-08-01 Thread Rik van Riel
On 07/23/2012 02:19 PM, Rafael Aquini wrote: In a glance, I believe this whole dance you're suggesting might just be too much of an overcomplication, and the best approach would be simply teaching the hotplug bits about the ballooned corner case just like it's being done to compaction/migration.

Re: [PATCH v4 1/3] mm: introduce compaction and migration for virtio ballooned pages

2012-08-01 Thread Rik van Riel
On 07/22/2012 10:33 PM, Minchan Kim wrote: IMHO, better approach is that after we can get complete free pageblocks by compaction or reclaim, move balloon pages into that pageblocks and make that blocks to unmovable. It can prevent fragmentation and it makes current or future code don't need to c

Re: [PATCH] Add a page cache-backed balloon device driver.

2012-06-26 Thread Rik van Riel
On 06/26/2012 05:31 PM, Frank Swiderski wrote: On Tue, Jun 26, 2012 at 1:40 PM, Rik van Riel wrote: The code looks good to me, my only worry is the code duplication. We now have 5 balloon drivers, for 4 hypervisors, all implementing everything from scratch... Do you have any

Re: [PATCH] Add a page cache-backed balloon device driver.

2012-06-26 Thread Rik van Riel
On 06/26/2012 04:32 PM, Frank Swiderski wrote: This implementation of a virtio balloon driver uses the page cache to "store" pages that have been released to the host. The communication (outside of target counts) is one way--the guest notifies the host when it adds a page to the page cache, allo

Re: [PATCH 1/4] mm: introduce compaction and migration for virtio ballooned pages

2012-06-26 Thread Rik van Riel
On 06/25/2012 07:25 PM, Rafael Aquini wrote: This patch introduces helper functions that teach compaction and migration bits how to cope with pages which are part of a guest memory balloon, in order to make them movable by memory compaction procedures. Signed-off-by: Rafael Aquini The function

Re: [PATCH 2/3] kvm hypervisor : Add hypercalls to support pv-ticketlock

2011-01-22 Thread Rik van Riel
On 01/22/2011 01:14 AM, Srivatsa Vaddagiri wrote: > Also it may be possible for the pv-ticketlocks to track owning vcpu and make > use > of a yield-to interface as further optimization to avoid the > "others-get-more-time" problem, but Peterz rightly pointed that PI would be a > better solution t

Re: [PATCH 2/3] kvm hypervisor : Add hypercalls to support pv-ticketlock

2011-01-21 Thread Rik van Riel
On 01/21/2011 09:02 AM, Srivatsa Vaddagiri wrote: > On Thu, Jan 20, 2011 at 09:56:27AM -0800, Jeremy Fitzhardinge wrote: >>> The key here is not to >>> sleep when waiting for locks (as implemented by current patch-series, which >>> can >>> put other VMs at an advantage by giving them more time t

Re: [patch 0/6] Guest page hinting version 7.

2009-04-02 Thread Rik van Riel
Jeremy Fitzhardinge wrote: > The more complex host policy decisions of how to balance overall > memory use system-wide are much in the same for both mechanisms. Not at all. Page hinting is just an optimization to host swapping, where IO can be avoided on many of the pages that hit the end of the

Re: [patch 0/6] Guest page hinting version 7.

2009-04-02 Thread Rik van Riel
Nick Piggin wrote: > On Friday 03 April 2009 06:06:31 Rik van Riel wrote: > >> Ballooning has a simpler mechanism, but relies on an >> as-of-yet undiscovered policy. >> >> Having experienced a zillion VM corner cases over the >> last decade and a bit,

Re: [patch 0/6] Guest page hinting version 7.

2009-04-02 Thread Rik van Riel
Martin Schwidefsky wrote: > The benefits are the same but the algorithmic complexity is reduced. > The patch to the memory management has complexity in itself but from a > 1000 feet standpoint guest page hinting is simpler, no? Page hinting has a complex, but well understood, mechanism and simple

Re: [patch 6/6] Guest page hinting: s390 support.

2009-04-01 Thread Rik van Riel
e states called block content states. > Signed-off-by: Martin Schwidefsky Acked-by: Rik van Riel -- All rights reversed. ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/virtualization

Re: [patch 5/6] Guest page hinting: minor fault optimization.

2009-04-01 Thread Rik van Riel
n question. Therefore it is important to > avoid the state transitions when possible. > Signed-off-by: Martin Schwidefsky Acked-by: Rik van Riel -- All rights reversed. ___ Virtualization mailing list Virtualization@lists.linux-foundat

Re: [patch 4/6] Guest page hinting: writable page table entries.

2009-04-01 Thread Rik van Riel
Martin Schwidefsky wrote: > On Wed, 01 Apr 2009 09:25:34 -0400 > Rik van Riel wrote: > >> Martin Schwidefsky wrote: >> >> This code has me stumped. Does it mean that if a page already >> has the PageWritable bit set (and count_ok stays 0), we will >> alwa

Re: [patch 4/6] Guest page hinting: writable page table entries.

2009-04-01 Thread Rik van Riel
Martin Schwidefsky wrote: This code has me stumped. Does it mean that if a page already has the PageWritable bit set (and count_ok stays 0), we will always mark the page as volatile? How does that work out on !s390? > /** > + * __page_check_writable() - check page state for new writable pte >

Re: [patch 3/6] Guest page hinting: mlocked pages.

2009-03-31 Thread Rik van Riel
le. > Signed-off-by: Martin Schwidefsky Acked-by: Rik van Riel -- All rights reversed. ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/virtualization

Re: [patch 2/6] Guest page hinting: volatile swap cache.

2009-03-31 Thread Rik van Riel
efsky Acked-by: Rik van Riel -- All rights reversed. ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/virtualization

Re: [patch 0/6] Guest page hinting version 7.

2009-03-30 Thread Rik van Riel
Jeremy Fitzhardinge wrote: > Rik van Riel wrote: >> Jeremy Fitzhardinge wrote: >> >>> That said, people have been looking at tracking block IO to work out >>> when it might be useful to try and share pages between guests under Xen. >> >> Tracking bloc

Re: [patch 0/6] Guest page hinting version 7.

2009-03-30 Thread Rik van Riel
Jeremy Fitzhardinge wrote: > That said, people have been looking at tracking block IO to work out > when it might be useful to try and share pages between guests under Xen. Tracking block IO seems like a bass-ackwards way to figure out what the contents of a memory page are. The KVM KSM code ha

Re: [patch 0/6] Guest page hinting version 7.

2009-03-29 Thread Rik van Riel
Martin Schwidefsky wrote: > On Fri, 27 Mar 2009 20:06:03 -0400 > Rik van Riel wrote: > >> Dave Hansen wrote: >>> On Fri, 2009-03-27 at 16:09 +0100, Martin Schwidefsky wrote: >>>> If the host picks one of the >>>> pages the guest can recreate, the h

Re: [patch 1/6] Guest page hinting: core + volatile page cache.

2009-03-29 Thread Rik van Riel
Martin Schwidefsky wrote: > On Fri, 27 Mar 2009 18:57:31 -0400 > Rik van Riel wrote: >> Martin Schwidefsky wrote: >>> There are some alternatives how this can be done, e.g. a global >>> lock, or lock per segment in the kernel page table, or the per page >

Re: [patch 0/6] Guest page hinting version 7.

2009-03-27 Thread Rik van Riel
Dave Hansen wrote: > On Fri, 2009-03-27 at 16:09 +0100, Martin Schwidefsky wrote: >> If the host picks one of the >> pages the guest can recreate, the host can throw it away instead of writing >> it to the paging device. Simple and elegant. > > Heh, simple and elegant for the hypervisor. But I'm

Re: [patch 1/6] Guest page hinting: core + volatile page cache.

2009-03-27 Thread Rik van Riel
Martin Schwidefsky wrote: > The major obstacles that need to get addressed: > * Concurrent page state changes: > To guard against concurrent page state updates some kind of lock > is needed. If page_make_volatile() has already done the 11 checks it > will issue the state change primitive. If

Re: [patch 0/4] [RFC] Another proportional weight IO controller

2008-11-06 Thread Rik van Riel
Peter Zijlstra wrote: > The only real issue I can see is with linear volumes, but those are > stupid anyway - non of the gains but all the risks. Linear volumes may well be the most common ones. People start out with the filesystems at a certain size, increasing onto a second (new) disk later, w

Re: [patch 0/4] [RFC] Another proportional weight IO controller

2008-11-06 Thread Rik van Riel
Peter Zijlstra wrote: > Nice, although I would think only doing the higher level devices makes > more sense than only doing the leafs. I'm not convinced. Say that you have two resource groups on a bunch of LVM volumes across two disks. If one of the resource groups only sends requests to one of

Re: [patch 0/6] Guest page hinting version 6.

2008-05-06 Thread Rik van Riel
On Tue, 06 May 2008 17:33:02 +0200 Martin Schwidefsky <[EMAIL PROTECTED]> wrote: > On Thu, 2008-03-13 at 16:57 +, Hugh Dickins wrote: > > It's very encouraging to see Jeremy and Rusty weighing in. I hope > > Zach will too, and I've added Andrea: their support would count a lot. > > You have Ni