Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-02-24 Thread Alex Williamson
On Thu, 2011-02-24 at 14:34 +0200, Avi Kivity wrote: > On 02/23/2011 11:46 PM, Alex Williamson wrote: > > > > > > But kvm_arch_flush_shadow() takes mmu_lock currently, so that needs > > > fixing. > > > > Hmm, I tried to follow the example in the !npages path just above this > > that does: > > > >

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-02-24 Thread Avi Kivity
On 02/24/2011 02:34 PM, Avi Kivity wrote: Hmm, your patch dereferences kvm->memslots without rcu_dereference(). That's a mortal (as in oops) sin. Sorry, that's wrong, all those places are under ->slots_lock. Note that sticking an rcu_dereference() blindly may or may not work - we might e

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-02-24 Thread Avi Kivity
On 02/23/2011 11:46 PM, Alex Williamson wrote: > > But kvm_arch_flush_shadow() takes mmu_lock currently, so that needs > fixing. Hmm, I tried to follow the example in the !npages path just above this that does: rcu_assign_pointer() synchronize_srcu_expedited() kvm_arch_flush_shadow() Do we h

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-02-23 Thread Alex Williamson
On Mon, 2011-01-31 at 17:18 -0200, Marcelo Tosatti wrote: > On Mon, Jan 24, 2011 at 10:37:56PM -0700, Alex Williamson wrote: > > On Mon, 2011-01-24 at 08:44 -0700, Alex Williamson wrote: > > > I'll look at how we might be > > > able to allocate slots on demand. Thanks, > > > > Here's a first cut

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-31 Thread Marcelo Tosatti
On Mon, Jan 24, 2011 at 10:37:56PM -0700, Alex Williamson wrote: > On Mon, 2011-01-24 at 08:44 -0700, Alex Williamson wrote: > > I'll look at how we might be > > able to allocate slots on demand. Thanks, > > Here's a first cut just to see if this looks agreeable. This allows the > slot array to

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-27 Thread Avi Kivity
On 01/27/2011 11:29 AM, Michael S. Tsirkin wrote: On Thu, Jan 27, 2011 at 11:28:12AM +0200, Avi Kivity wrote: > On 01/27/2011 11:26 AM, Michael S. Tsirkin wrote: > >> >Right. That's why I say that sorting by size might not be optimal. > >> >Maybe a cache ... > >> > >> Why would it not

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-27 Thread Michael S. Tsirkin
On Thu, Jan 27, 2011 at 11:28:12AM +0200, Avi Kivity wrote: > On 01/27/2011 11:26 AM, Michael S. Tsirkin wrote: > >> >Right. That's why I say that sorting by size might not be optimal. > >> >Maybe a cache ... > >> > >> Why would it not be optimal? > >> > >> If you have 16GB RAM in two slots and

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-27 Thread Michael S. Tsirkin
On Thu, Jan 27, 2011 at 11:26:19AM +0200, Michael S. Tsirkin wrote: > On Thu, Jan 27, 2011 at 11:21:47AM +0200, Avi Kivity wrote: > > On 01/26/2011 02:08 PM, Michael S. Tsirkin wrote: > > >> > > > >> >I just mean that once you fault you map sptes and then you can use them > > >> >without exits.

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-27 Thread Avi Kivity
On 01/27/2011 11:26 AM, Michael S. Tsirkin wrote: > >Right. That's why I say that sorting by size might not be optimal. > >Maybe a cache ... > > Why would it not be optimal? > > If you have 16GB RAM in two slots and a few megabytes here and there > scattered in some slots, you have three ord

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-27 Thread Michael S. Tsirkin
On Thu, Jan 27, 2011 at 11:21:47AM +0200, Avi Kivity wrote: > On 01/26/2011 02:08 PM, Michael S. Tsirkin wrote: > >> > > >> >I just mean that once you fault you map sptes and then you can use them > >> >without exits. mmio will cause exits each time. Right? > >> > >> The swapper scanning sptes

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-27 Thread Avi Kivity
On 01/26/2011 02:08 PM, Michael S. Tsirkin wrote: > > > >I just mean that once you fault you map sptes and then you can use them > >without exits. mmio will cause exits each time. Right? > > The swapper scanning sptes, ksmd, khugepaged, and swapping can all > cause a page to be unmapped. T

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-26 Thread Michael S. Tsirkin
On Wed, Jan 26, 2011 at 11:54:21AM +0200, Avi Kivity wrote: > On 01/26/2011 11:39 AM, Michael S. Tsirkin wrote: > >On Wed, Jan 26, 2011 at 11:23:21AM +0200, Avi Kivity wrote: > >> On 01/26/2011 11:20 AM, Michael S. Tsirkin wrote: > >> >On Wed, Jan 26, 2011 at 11:17:11AM +0200, Avi Kivity wrote: >

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-26 Thread Avi Kivity
On 01/26/2011 11:39 AM, Michael S. Tsirkin wrote: On Wed, Jan 26, 2011 at 11:23:21AM +0200, Avi Kivity wrote: > On 01/26/2011 11:20 AM, Michael S. Tsirkin wrote: > >On Wed, Jan 26, 2011 at 11:17:11AM +0200, Avi Kivity wrote: > >> On 01/25/2011 07:58 PM, Michael S. Tsirkin wrote: > >> >On

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-26 Thread Michael S. Tsirkin
On Wed, Jan 26, 2011 at 11:23:21AM +0200, Avi Kivity wrote: > On 01/26/2011 11:20 AM, Michael S. Tsirkin wrote: > >On Wed, Jan 26, 2011 at 11:17:11AM +0200, Avi Kivity wrote: > >> On 01/25/2011 07:58 PM, Michael S. Tsirkin wrote: > >> >On Tue, Jan 25, 2011 at 07:33:40PM +0200, Avi Kivity wrote: >

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-26 Thread Avi Kivity
On 01/25/2011 08:00 PM, Michael S. Tsirkin wrote: > > > >I see now. Yes, a flag in spte would help. changes in slots would then > >have to update all these flags. > > That's easy, we drop all sptes. Ah, right. Hmm cpu has no flag to distinguish mmio sptes somehow already? No. Allocated-b

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-26 Thread Avi Kivity
On 01/26/2011 11:20 AM, Michael S. Tsirkin wrote: On Wed, Jan 26, 2011 at 11:17:11AM +0200, Avi Kivity wrote: > On 01/25/2011 07:58 PM, Michael S. Tsirkin wrote: > >On Tue, Jan 25, 2011 at 07:33:40PM +0200, Avi Kivity wrote: > >> On 01/25/2011 04:59 PM, Michael S. Tsirkin wrote: > >> >On

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-26 Thread Avi Kivity
On 01/25/2011 07:43 PM, Alex Williamson wrote: On Tue, 2011-01-25 at 19:11 +0200, Avi Kivity wrote: > On 01/25/2011 04:57 PM, Alex Williamson wrote: > > On Tue, 2011-01-25 at 12:23 +0200, Avi Kivity wrote: > > > On 01/25/2011 07:37 AM, Alex Williamson wrote: > > > > On Mon, 2011-01-2

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-26 Thread Michael S. Tsirkin
On Wed, Jan 26, 2011 at 11:17:11AM +0200, Avi Kivity wrote: > On 01/25/2011 07:58 PM, Michael S. Tsirkin wrote: > >On Tue, Jan 25, 2011 at 07:33:40PM +0200, Avi Kivity wrote: > >> On 01/25/2011 04:59 PM, Michael S. Tsirkin wrote: > >> >On Tue, Jan 25, 2011 at 04:53:44PM +0200, Avi Kivity wrote: >

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-26 Thread Avi Kivity
On 01/25/2011 07:58 PM, Michael S. Tsirkin wrote: On Tue, Jan 25, 2011 at 07:33:40PM +0200, Avi Kivity wrote: > On 01/25/2011 04:59 PM, Michael S. Tsirkin wrote: > >On Tue, Jan 25, 2011 at 04:53:44PM +0200, Avi Kivity wrote: > >> For the other lookups, which we > >> believe will succeed,

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-26 Thread Jan Kiszka
On 2011-01-25 20:13, Alex Williamson wrote: > On Tue, 2011-01-25 at 17:35 +0100, Jan Kiszka wrote: >> On 2011-01-25 15:53, Avi Kivity wrote: >>> On 01/25/2011 04:41 PM, Alex Williamson wrote: > > > > > > kvm: Allow memory slot array to grow on demand > > > > Remove fixed

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-25 Thread Alex Williamson
On Tue, 2011-01-25 at 17:35 +0100, Jan Kiszka wrote: > On 2011-01-25 15:53, Avi Kivity wrote: > > On 01/25/2011 04:41 PM, Alex Williamson wrote: > >> > > > >> > > > >> > > kvm: Allow memory slot array to grow on demand > >> > > > >> > > Remove fixed KVM_MEMORY_SLOTS limit, allowing the slot

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-25 Thread Michael S. Tsirkin
On Tue, Jan 25, 2011 at 07:34:18PM +0200, Avi Kivity wrote: > On 01/25/2011 05:23 PM, Michael S. Tsirkin wrote: > >On Tue, Jan 25, 2011 at 04:58:41PM +0200, Avi Kivity wrote: > >> On 01/25/2011 04:55 PM, Michael S. Tsirkin wrote: > >> >> > >> >> We can't make it unbounded in the kernel, since

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-25 Thread Michael S. Tsirkin
On Tue, Jan 25, 2011 at 07:33:40PM +0200, Avi Kivity wrote: > On 01/25/2011 04:59 PM, Michael S. Tsirkin wrote: > >On Tue, Jan 25, 2011 at 04:53:44PM +0200, Avi Kivity wrote: > >> For the other lookups, which we > >> believe will succeed, we can assume the probablity of a match is > >> related t

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-25 Thread Alex Williamson
On Tue, 2011-01-25 at 19:11 +0200, Avi Kivity wrote: > On 01/25/2011 04:57 PM, Alex Williamson wrote: > > On Tue, 2011-01-25 at 12:23 +0200, Avi Kivity wrote: > > > On 01/25/2011 07:37 AM, Alex Williamson wrote: > > > > On Mon, 2011-01-24 at 08:44 -0700, Alex Williamson wrote: > > > > > I'll

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-25 Thread Avi Kivity
On 01/25/2011 05:23 PM, Michael S. Tsirkin wrote: On Tue, Jan 25, 2011 at 04:58:41PM +0200, Avi Kivity wrote: > On 01/25/2011 04:55 PM, Michael S. Tsirkin wrote: > >> > >> We can't make it unbounded in the kernel, since a malicious user > >> could start creating an infinite amount of memo

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-25 Thread Avi Kivity
On 01/25/2011 04:59 PM, Michael S. Tsirkin wrote: On Tue, Jan 25, 2011 at 04:53:44PM +0200, Avi Kivity wrote: > For the other lookups, which we > believe will succeed, we can assume the probablity of a match is > related to the slot size, and sort the slots by page count. Unlikely to be true

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-25 Thread Avi Kivity
On 01/25/2011 04:57 PM, Alex Williamson wrote: On Tue, 2011-01-25 at 12:23 +0200, Avi Kivity wrote: > On 01/25/2011 07:37 AM, Alex Williamson wrote: > > On Mon, 2011-01-24 at 08:44 -0700, Alex Williamson wrote: > > > I'll look at how we might be > > > able to allocate slots on demand.

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-25 Thread Jan Kiszka
On 2011-01-25 15:53, Avi Kivity wrote: > On 01/25/2011 04:41 PM, Alex Williamson wrote: >> > > >> > > >> > > kvm: Allow memory slot array to grow on demand >> > > >> > > Remove fixed KVM_MEMORY_SLOTS limit, allowing the slot array >> > > to grow on demand. Private slots are now allocated

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-25 Thread Michael S. Tsirkin
On Tue, Jan 25, 2011 at 04:58:41PM +0200, Avi Kivity wrote: > On 01/25/2011 04:55 PM, Michael S. Tsirkin wrote: > >> > >> We can't make it unbounded in the kernel, since a malicious user > >> could start creating an infinite amount of memory slots, pinning > >> unbounded kernel memory. > > > >Ho

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-25 Thread Michael S. Tsirkin
On Tue, Jan 25, 2011 at 04:53:44PM +0200, Avi Kivity wrote: > For the other lookups, which we > believe will succeed, we can assume the probablity of a match is > related to the slot size, and sort the slots by page count. Unlikely to be true for assigned device BARs. -- MST -- To unsubscribe fr

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-25 Thread Avi Kivity
On 01/25/2011 04:55 PM, Michael S. Tsirkin wrote: > > We can't make it unbounded in the kernel, since a malicious user > could start creating an infinite amount of memory slots, pinning > unbounded kernel memory. How about keeping the slots in userspace memory, access them with copy from user

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-25 Thread Alex Williamson
On Tue, 2011-01-25 at 12:23 +0200, Avi Kivity wrote: > On 01/25/2011 07:37 AM, Alex Williamson wrote: > > On Mon, 2011-01-24 at 08:44 -0700, Alex Williamson wrote: > > > I'll look at how we might be > > > able to allocate slots on demand. Thanks, > > > > Here's a first cut just to see if this lo

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-25 Thread Avi Kivity
On 01/25/2011 04:46 PM, Alex Williamson wrote: On Tue, 2011-01-25 at 12:20 +0200, Avi Kivity wrote: > On 01/24/2011 11:32 AM, Marcelo Tosatti wrote: > > On Fri, Jan 21, 2011 at 04:48:02PM -0700, Alex Williamson wrote: > > > When doing device assignment, we use cpu_register_physical_memory(

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-25 Thread Michael S. Tsirkin
On Tue, Jan 25, 2011 at 12:20:03PM +0200, Avi Kivity wrote: > On 01/24/2011 11:32 AM, Marcelo Tosatti wrote: > >On Fri, Jan 21, 2011 at 04:48:02PM -0700, Alex Williamson wrote: > >> When doing device assignment, we use cpu_register_physical_memory() to > >> directly map the qemu mmap of the devic

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-25 Thread Avi Kivity
On 01/25/2011 04:41 PM, Alex Williamson wrote: > > > > > > kvm: Allow memory slot array to grow on demand > > > > Remove fixed KVM_MEMORY_SLOTS limit, allowing the slot array > > to grow on demand. Private slots are now allocated at the > > front instead of the end. Only x86 seems to

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-25 Thread Avi Kivity
On 01/25/2011 04:45 PM, Michael S. Tsirkin wrote: Also, is there any time where we need to scan all slots on data path? All guest mmio accesses. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a me

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-25 Thread Alex Williamson
On Tue, 2011-01-25 at 12:20 +0200, Avi Kivity wrote: > On 01/24/2011 11:32 AM, Marcelo Tosatti wrote: > > On Fri, Jan 21, 2011 at 04:48:02PM -0700, Alex Williamson wrote: > > > When doing device assignment, we use cpu_register_physical_memory() to > > > directly map the qemu mmap of the device re

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-25 Thread Michael S. Tsirkin
On Tue, Jan 25, 2011 at 07:41:32AM -0700, Alex Williamson wrote: > On Tue, 2011-01-25 at 08:36 +0100, Jan Kiszka wrote: > > On 2011-01-25 06:37, Alex Williamson wrote: > > > On Mon, 2011-01-24 at 08:44 -0700, Alex Williamson wrote: > > >> I'll look at how we might be > > >> able to allocate slots o

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-25 Thread Alex Williamson
On Tue, 2011-01-25 at 08:36 +0100, Jan Kiszka wrote: > On 2011-01-25 06:37, Alex Williamson wrote: > > On Mon, 2011-01-24 at 08:44 -0700, Alex Williamson wrote: > >> I'll look at how we might be > >> able to allocate slots on demand. Thanks, > > > > Here's a first cut just to see if this looks ag

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-25 Thread Avi Kivity
On 01/24/2011 11:32 AM, Marcelo Tosatti wrote: On Fri, Jan 21, 2011 at 04:48:02PM -0700, Alex Williamson wrote: > When doing device assignment, we use cpu_register_physical_memory() to > directly map the qemu mmap of the device resource into the address > space of the guest. The unadvertised

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-25 Thread Avi Kivity
On 01/25/2011 07:37 AM, Alex Williamson wrote: On Mon, 2011-01-24 at 08:44 -0700, Alex Williamson wrote: > I'll look at how we might be > able to allocate slots on demand. Thanks, Here's a first cut just to see if this looks agreeable. This allows the slot array to grow on demand. This work

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-24 Thread Jan Kiszka
On 2011-01-25 06:37, Alex Williamson wrote: > On Mon, 2011-01-24 at 08:44 -0700, Alex Williamson wrote: >> I'll look at how we might be >> able to allocate slots on demand. Thanks, > > Here's a first cut just to see if this looks agreeable. This allows the > slot array to grow on demand. This w

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-24 Thread Alex Williamson
On Mon, 2011-01-24 at 08:44 -0700, Alex Williamson wrote: > I'll look at how we might be > able to allocate slots on demand. Thanks, Here's a first cut just to see if this looks agreeable. This allows the slot array to grow on demand. This works with current userspace, as well as userspace triv

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-24 Thread Alex Williamson
On Mon, 2011-01-24 at 15:16 +0100, Jan Kiszka wrote: > On 2011-01-24 10:32, Marcelo Tosatti wrote: > > On Fri, Jan 21, 2011 at 04:48:02PM -0700, Alex Williamson wrote: > >> When doing device assignment, we use cpu_register_physical_memory() to > >> directly map the qemu mmap of the device resource

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-24 Thread Jan Kiszka
On 2011-01-24 10:32, Marcelo Tosatti wrote: > On Fri, Jan 21, 2011 at 04:48:02PM -0700, Alex Williamson wrote: >> When doing device assignment, we use cpu_register_physical_memory() to >> directly map the qemu mmap of the device resource into the address >> space of the guest. The unadvertised fea

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-24 Thread Marcelo Tosatti
On Fri, Jan 21, 2011 at 04:48:02PM -0700, Alex Williamson wrote: > When doing device assignment, we use cpu_register_physical_memory() to > directly map the qemu mmap of the device resource into the address > space of the guest. The unadvertised feature of the register physical > memory code path

Re: [RFC PATCH 0/2] Expose available KVM free memory slot count to help avoid aborts

2011-01-22 Thread Michael S. Tsirkin
On Fri, Jan 21, 2011 at 04:48:02PM -0700, Alex Williamson wrote: > When doing device assignment, we use cpu_register_physical_memory() to > directly map the qemu mmap of the device resource into the address > space of the guest. The unadvertised feature of the register physical > memory code path