Re: [PATCH v11 4/6] mm: function to offer a page block on the free list

2017-06-20 Thread Rik van Riel
On Tue, 2017-06-20 at 21:26 +0300, Michael S. Tsirkin wrote: > On Tue, Jun 20, 2017 at 01:29:00PM -0400, Rik van Riel wrote: > > I agree with that.  Let me go into some more detail of > > what Nitesh is implementing: > > > > 1) In arch_free_page, the being-freed page i

Re: [PATCH v11 4/6] mm: function to offer a page block on the free list

2017-06-20 Thread Rik van Riel
On Tue, 2017-06-20 at 18:49 +0200, David Hildenbrand wrote: > On 20.06.2017 18:44, Rik van Riel wrote: > > Nitesh Lal (on the CC list) is working on a way > > to efficiently batch recently freed pages for > > free page hinting to the hypervisor. > > > > If th

Re: [PATCH v11 4/6] mm: function to offer a page block on the free list

2017-06-20 Thread Rik van Riel
On Mon, 2017-06-12 at 07:10 -0700, Dave Hansen wrote: > The hypervisor is going to throw away the contents of these pages, > right?  As soon as the spinlock is released, someone can allocate a > page, and put good data in it.  What keeps the hypervisor from > throwing > away good data? That

Re: [Qemu-devel] [RFC qemu 0/4] A PV solution for live migration optimization

2016-03-09 Thread Rik van Riel
On Wed, 2016-03-09 at 20:04 +0300, Roman Kagan wrote: > On Wed, Mar 09, 2016 at 05:41:39PM +0200, Michael S. Tsirkin wrote: > > On Wed, Mar 09, 2016 at 05:28:54PM +0300, Roman Kagan wrote: > > > For (1) I've been trying to make a point that skipping clean > > > pages is > > > much more likely to

Re: [PATCH v15 16/16] unfair qspinlock: a queue based unfair lock

2015-04-09 Thread Rik van Riel
On 04/09/2015 10:13 AM, Peter Zijlstra wrote: On Thu, Apr 09, 2015 at 09:16:24AM -0400, Rik van Riel wrote: On 04/09/2015 03:01 AM, Peter Zijlstra wrote: On Wed, Apr 08, 2015 at 02:32:19PM -0400, Waiman Long wrote: For a virtual guest with the qspinlock patch, a simple unfair byte lock

Re: [PATCH v15 16/16] unfair qspinlock: a queue based unfair lock

2015-04-09 Thread Rik van Riel
On 04/09/2015 03:01 AM, Peter Zijlstra wrote: On Wed, Apr 08, 2015 at 02:32:19PM -0400, Waiman Long wrote: For a virtual guest with the qspinlock patch, a simple unfair byte lock will be used if PV spinlock is not configured in or the hypervisor isn't either KVM or Xen. The byte lock works

Re: [RFC] Implement Batched (group) ticket lock

2014-05-28 Thread Rik van Riel
On 05/28/2014 08:16 AM, Raghavendra K T wrote: This patch looks very promising. TODO: - we need an intelligent way to nullify the effect of batching for baremetal (because extra cmpxchg is not required). On (larger?) NUMA systems, the unfairness may be a nice performance benefit, reducing

Re: [RFC] Implement Batched (group) ticket lock

2014-05-28 Thread Rik van Riel
On 05/28/2014 06:19 PM, Linus Torvalds wrote: If somebody has a P4 still, that's likely the worst case by far. I'm sure cmpxchg isn't the only thing making P4 the worst case :) -- All rights reversed ___ Virtualization mailing list

Re: [PATCH v10 0/5] make balloon pages movable by compaction

2012-09-17 Thread Rik van Riel
On 09/17/2012 06:15 PM, Andrew Morton wrote: On Mon, 17 Sep 2012 13:38:15 -0300 Rafael Aquini aqu...@redhat.com wrote: Memory fragmentation introduced by ballooning might reduce significantly the number of 2MB contiguous memory blocks that can be used within a guest, thus imposing performance

Re: [PATCH] Add a page cache-backed balloon device driver.

2012-09-10 Thread Rik van Riel
On 09/10/2012 01:37 PM, Mike Waychison wrote: On Mon, Sep 10, 2012 at 5:05 AM, Michael S. Tsirkin m...@redhat.com wrote: Also can you pls answer Avi's question? How is overcommit managed? Overcommit in our deployments is managed using memory cgroups on the host. This allows us to have very

Re: [PATCH v9 0/5] make balloon pages movable by compaction

2012-08-26 Thread Rik van Riel
On 08/26/2012 03:58 AM, Michael S. Tsirkin wrote: On Sat, Aug 25, 2012 at 02:24:55AM -0300, Rafael Aquini wrote: Memory fragmentation introduced by ballooning might reduce significantly the number of 2MB contiguous memory blocks that can be used within a guest, thus imposing performance

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rik van Riel
On 08/23/2012 11:54 AM, Michael S. Tsirkin wrote: On Thu, Aug 23, 2012 at 12:21:29PM -0300, Rafael Aquini wrote: On Thu, Aug 23, 2012 at 04:53:29PM +0300, Michael S. Tsirkin wrote: On Thu, Aug 23, 2012 at 10:06:07AM -0300, Rafael Aquini wrote: On Thu, Aug 23, 2012 at 03:34:32PM +0300, Michael

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rik van Riel
On 08/23/2012 01:28 PM, Rafael Aquini wrote: What about keep a conter at virtio_balloon structure on how much pages are isolated from balloon's list and check it at leak time? if the counter gets 0 than we can safely put leak_balloon() to wait until balloon page list gets completely refilled.

Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility

2012-08-23 Thread Rik van Riel
On 08/23/2012 07:36 PM, Michael S. Tsirkin wrote: --- virtio-balloon: replace page-lru list with page-private. The point is to free up page-lru for use by compaction. Warning: completely untested, will provide tested version if we agree on this direction. A singly linked list is not going

Re: [PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-15 Thread Rik van Riel
On 08/14/2012 05:38 PM, Michael S. Tsirkin wrote: And even ignoring that, global pointer to a device is an ugly hack and ugly hacks tend to explode. And even ignoring estetics, and if we decide we are fine with a single balloon, it needs to fail gracefully not crash like it does now. Fair

Re: [PATCH v7 2/4] virtio_balloon: introduce migration primitives to balloon pages

2012-08-14 Thread Rik van Riel
On 08/14/2012 04:54 PM, Michael S. Tsirkin wrote: To clarify, the global state that this patch adds, is ugly even if we didn't support multiple balloons yet. So I don't think I can accept such a patch. Rusty has a final word here, maybe he thinks differently. Before deciding that does not

Re: [PATCH v6 1/3] mm: introduce compaction and migration for virtio ballooned pages

2012-08-08 Thread Rik van Riel
-by: Rafael Aquiniaqu...@redhat.com Reviewed-by: Rik van Riel r...@redhat.com -- All rights reversed ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v6 2/3] virtio_balloon: introduce migration primitives to balloon pages

2012-08-08 Thread Rik van Riel
(spinlock): special protection to balloon pages list against concurrent list handling operations; Signed-off-by: Rafael Aquiniaqu...@redhat.com Reviewed-by: Rik van Riel r...@redhat.com -- All rights reversed ___ Virtualization

Re: [PATCH v6 3/3] mm: add vm event counters for balloon pages compaction

2012-08-08 Thread Rik van Riel
in the future, we'll be able to see it... Signed-off-by: Rafael Aquiniaqu...@redhat.com Acked-by: Rik van Riel r...@redhat.com -- All rights reversed ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org

Re: [PATCH v5 1/3] mm: introduce compaction and migration for virtio ballooned pages

2012-08-06 Thread Rik van Riel
On 08/06/2012 09:56 AM, Rafael Aquini wrote: @@ -846,6 +861,21 @@ static int unmap_and_move(new_page_t get_new_page, unsigned long private, goto out; rc = __unmap_and_move(page, newpage, force, offlining, mode); + + if (unlikely(is_balloon_page(newpage) +

Re: [PATCH v4 1/3] mm: introduce compaction and migration for virtio ballooned pages

2012-08-01 Thread Rik van Riel
On 07/22/2012 10:33 PM, Minchan Kim wrote: IMHO, better approach is that after we can get complete free pageblocks by compaction or reclaim, move balloon pages into that pageblocks and make that blocks to unmovable. It can prevent fragmentation and it makes current or future code don't need to

Re: [PATCH v4 1/3] mm: introduce compaction and migration for virtio ballooned pages

2012-08-01 Thread Rik van Riel
On 07/23/2012 02:19 PM, Rafael Aquini wrote: In a glance, I believe this whole dance you're suggesting might just be too much of an overcomplication, and the best approach would be simply teaching the hotplug bits about the ballooned corner case just like it's being done to

Re: [PATCH 1/4] mm: introduce compaction and migration for virtio ballooned pages

2012-06-26 Thread Rik van Riel
On 06/25/2012 07:25 PM, Rafael Aquini wrote: This patch introduces helper functions that teach compaction and migration bits how to cope with pages which are part of a guest memory balloon, in order to make them movable by memory compaction procedures. Signed-off-by: Rafael

Re: [PATCH] Add a page cache-backed balloon device driver.

2012-06-26 Thread Rik van Riel
On 06/26/2012 04:32 PM, Frank Swiderski wrote: This implementation of a virtio balloon driver uses the page cache to store pages that have been released to the host. The communication (outside of target counts) is one way--the guest notifies the host when it adds a page to the page cache,

Re: [PATCH] Add a page cache-backed balloon device driver.

2012-06-26 Thread Rik van Riel
On 06/26/2012 05:31 PM, Frank Swiderski wrote: On Tue, Jun 26, 2012 at 1:40 PM, Rik van Rielr...@redhat.com wrote: The code looks good to me, my only worry is the code duplication. We now have 5 balloon drivers, for 4 hypervisors, all implementing everything from scratch... Do you have any

Re: [PATCH 2/3] kvm hypervisor : Add hypercalls to support pv-ticketlock

2011-01-22 Thread Rik van Riel
On 01/22/2011 01:14 AM, Srivatsa Vaddagiri wrote: Also it may be possible for the pv-ticketlocks to track owning vcpu and make use of a yield-to interface as further optimization to avoid the others-get-more-time problem, but Peterz rightly pointed that PI would be a better solution there

Re: [PATCH 2/3] kvm hypervisor : Add hypercalls to support pv-ticketlock

2011-01-21 Thread Rik van Riel
On 01/21/2011 09:02 AM, Srivatsa Vaddagiri wrote: On Thu, Jan 20, 2011 at 09:56:27AM -0800, Jeremy Fitzhardinge wrote: The key here is not to sleep when waiting for locks (as implemented by current patch-series, which can put other VMs at an advantage by giving them more time than they are

Re: [patch 0/6] Guest page hinting version 7.

2009-04-02 Thread Rik van Riel
Martin Schwidefsky wrote: The benefits are the same but the algorithmic complexity is reduced. The patch to the memory management has complexity in itself but from a 1000 feet standpoint guest page hinting is simpler, no? Page hinting has a complex, but well understood, mechanism and simple

Re: [patch 0/6] Guest page hinting version 7.

2009-04-02 Thread Rik van Riel
Nick Piggin wrote: On Friday 03 April 2009 06:06:31 Rik van Riel wrote: Ballooning has a simpler mechanism, but relies on an as-of-yet undiscovered policy. Having experienced a zillion VM corner cases over the last decade and a bit, I think I prefer a complex mechanism over complex

Re: [patch 0/6] Guest page hinting version 7.

2009-04-02 Thread Rik van Riel
Jeremy Fitzhardinge wrote: The more complex host policy decisions of how to balance overall memory use system-wide are much in the same for both mechanisms. Not at all. Page hinting is just an optimization to host swapping, where IO can be avoided on many of the pages that hit the end of the

Re: [patch 4/6] Guest page hinting: writable page table entries.

2009-04-01 Thread Rik van Riel
Martin Schwidefsky wrote: This code has me stumped. Does it mean that if a page already has the PageWritable bit set (and count_ok stays 0), we will always mark the page as volatile? How does that work out on !s390? /** + * __page_check_writable() - check page state for new writable pte +

Re: [patch 4/6] Guest page hinting: writable page table entries.

2009-04-01 Thread Rik van Riel
Martin Schwidefsky wrote: On Wed, 01 Apr 2009 09:25:34 -0400 Rik van Riel r...@redhat.com wrote: Martin Schwidefsky wrote: This code has me stumped. Does it mean that if a page already has the PageWritable bit set (and count_ok stays 0), we will always mark the page as volatile? How

Re: [patch 5/6] Guest page hinting: minor fault optimization.

2009-04-01 Thread Rik van Riel
information is in question. Therefore it is important to avoid the state transitions when possible. Signed-off-by: Martin Schwidefsky schwidef...@de.ibm.com Acked-by: Rik van Riel r...@redhat.com -- All rights reversed. ___ Virtualization mailing

Re: [patch 6/6] Guest page hinting: s390 support.

2009-04-01 Thread Rik van Riel
host page states called block content states. Signed-off-by: Martin Schwidefsky schwidef...@de.ibm.com Acked-by: Rik van Riel r...@redhat.com -- All rights reversed. ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https

Re: [patch 2/6] Guest page hinting: volatile swap cache.

2009-03-31 Thread Rik van Riel
-by: Martin Schwidefsky schwidef...@de.ibm.com Acked-by: Rik van Riel r...@redhat.com -- All rights reversed. ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/virtualization

Re: [patch 3/6] Guest page hinting: mlocked pages.

2009-03-31 Thread Rik van Riel
they need to be stable. Signed-off-by: Martin Schwidefsky schwidef...@de.ibm.com Acked-by: Rik van Riel r...@redhat.com -- All rights reversed. ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linux-foundation.org

Re: [patch 0/6] Guest page hinting version 7.

2009-03-30 Thread Rik van Riel
Jeremy Fitzhardinge wrote: That said, people have been looking at tracking block IO to work out when it might be useful to try and share pages between guests under Xen. Tracking block IO seems like a bass-ackwards way to figure out what the contents of a memory page are. The KVM KSM code has

Re: [patch 0/6] Guest page hinting version 7.

2009-03-30 Thread Rik van Riel
Jeremy Fitzhardinge wrote: Rik van Riel wrote: Jeremy Fitzhardinge wrote: That said, people have been looking at tracking block IO to work out when it might be useful to try and share pages between guests under Xen. Tracking block IO seems like a bass-ackwards way to figure out what

Re: [patch 1/6] Guest page hinting: core + volatile page cache.

2009-03-29 Thread Rik van Riel
Martin Schwidefsky wrote: On Fri, 27 Mar 2009 18:57:31 -0400 Rik van Riel r...@redhat.com wrote: Martin Schwidefsky wrote: There are some alternatives how this can be done, e.g. a global lock, or lock per segment in the kernel page table, or the per page bit PG_arch_1 if it is still

Re: [patch 1/6] Guest page hinting: core + volatile page cache.

2009-03-27 Thread Rik van Riel
Martin Schwidefsky wrote: The major obstacles that need to get addressed: * Concurrent page state changes: To guard against concurrent page state updates some kind of lock is needed. If page_make_volatile() has already done the 11 checks it will issue the state change primitive. If in

Re: [patch 0/4] [RFC] Another proportional weight IO controller

2008-11-06 Thread Rik van Riel
Peter Zijlstra wrote: Nice, although I would think only doing the higher level devices makes more sense than only doing the leafs. I'm not convinced. Say that you have two resource groups on a bunch of LVM volumes across two disks. If one of the resource groups only sends requests to one of

Re: [patch 0/4] [RFC] Another proportional weight IO controller

2008-11-06 Thread Rik van Riel
Peter Zijlstra wrote: The only real issue I can see is with linear volumes, but those are stupid anyway - non of the gains but all the risks. Linear volumes may well be the most common ones. People start out with the filesystems at a certain size, increasing onto a second (new) disk later,

Re: [patch 0/6] Guest page hinting version 6.

2008-05-06 Thread Rik van Riel
On Tue, 06 May 2008 17:33:02 +0200 Martin Schwidefsky [EMAIL PROTECTED] wrote: On Thu, 2008-03-13 at 16:57 +, Hugh Dickins wrote: It's very encouraging to see Jeremy and Rusty weighing in. I hope Zach will too, and I've added Andrea: their support would count a lot. You have Nick on