Re: [PATCH] vhost-net: switch to smp barriers

2010-02-07 Thread Michael S. Tsirkin
On Mon, Feb 01, 2010 at 07:21:02PM +0200, Michael S. Tsirkin wrote:
> vhost-net only uses memory barriers to control SMP effects
> (communication with userspace potentially running on a different CPU),
> so it should use SMP barriers and not mandatory barriers for memory
> access ordering, as suggested by Documentation/memory-barriers.txt
> 
> Signed-off-by: Michael S. Tsirkin 


Rusty, any feedback on this one?
Thanks!

> ---
>  drivers/vhost/vhost.c |   10 +-
>  1 files changed, 5 insertions(+), 5 deletions(-)
> 
> The above applies on top of net-next-2.6. Does not seem to give any
> measureable performance gain, but seems to generate less code
> and I think it's better to use correct APIs.
> 
> diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
> index c8c25db..6eb1525 100644
> --- a/drivers/vhost/vhost.c
> +++ b/drivers/vhost/vhost.c
> @@ -685,7 +685,7 @@ int vhost_log_write(struct vhost_virtqueue *vq, struct 
> vhost_log *log,
>   int i, r;
>  
>   /* Make sure data written is seen before log. */
> - wmb();
> + smp_wmb();
>   for (i = 0; i < log_num; ++i) {
>   u64 l = min(log[i].len, len);
>   r = log_write(vq->log_base, log[i].addr, l);
> @@ -884,7 +884,7 @@ unsigned vhost_get_vq_desc(struct vhost_dev *dev, struct 
> vhost_virtqueue *vq,
>   return vq->num;
>  
>   /* Only get avail ring entries after they have been exposed by guest. */
> - rmb();
> + smp_rmb();
>  
>   /* Grab the next descriptor number they're advertising, and increment
>* the index we've seen. */
> @@ -996,14 +996,14 @@ int vhost_add_used(struct vhost_virtqueue *vq, unsigned 
> int head, int len)
>   return -EFAULT;
>   }
>   /* Make sure buffer is written before we update index. */
> - wmb();
> + smp_wmb();
>   if (put_user(vq->last_used_idx + 1, &vq->used->idx)) {
>   vq_err(vq, "Failed to increment used idx");
>   return -EFAULT;
>   }
>   if (unlikely(vq->log_used)) {
>   /* Make sure data is seen before log. */
> - wmb();
> + smp_wmb();
>   log_write(vq->log_base, vq->log_addr + sizeof *vq->used->ring *
> (vq->last_used_idx % vq->num),
> sizeof *vq->used->ring);
> @@ -1060,7 +1060,7 @@ bool vhost_enable_notify(struct vhost_virtqueue *vq)
>   }
>   /* They could have slipped one in as we were doing that: make
>* sure it's written, then check again. */
> - mb();
> + smp_mb();
>   r = get_user(avail_idx, &vq->avail->idx);
>   if (r) {
>   vq_err(vq, "Failed to check avail idx at %p: %d\n",
> -- 
> 1.6.6.144.g5c3af
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 02/11] iommu-api: Add iommu_map and iommu_unmap functions

2010-02-07 Thread Avi Kivity

On 01/28/2010 01:37 PM, Joerg Roedel wrote:

These two functions provide support for mapping and
unmapping physical addresses to io virtual addresses. The
difference to the iommu_(un)map_range() is that the new
functions take a gfp_order parameter instead of a size. This
allows the IOMMU backend implementations to detect easier if
a given range can be mapped by larger page sizes.
These new functions should replace the old ones in the long
term.
   


These seem to be less flexible in the long term.  Sure, it is easier for 
the backend to map to multiple page sizes if your iommu supports 
arbitrary power-of-two page sizes, but we should make the APIs easier 
for the callers, not the backend.



--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] vhost-net: switch to smp barriers

2010-02-07 Thread Avi Kivity

On 02/01/2010 07:21 PM, Michael S. Tsirkin wrote:

vhost-net only uses memory barriers to control SMP effects
(communication with userspace potentially running on a different CPU),
so it should use SMP barriers and not mandatory barriers for memory
access ordering, as suggested by Documentation/memory-barriers.txt

   


A UP guest running on an SMP host still needs those barriers.

--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v2 00/21] qemu-kvm: Hook cleanups and extended use of upstream code

2010-02-07 Thread Gleb Natapov
On Wed, Feb 03, 2010 at 09:53:25AM +0100, Jan Kiszka wrote:
> This version addresses the feedback on v2, namely:
> - assert( || thread>) on low-level
>   load/save registers
> - fixed mpstate initialization
> 
With those patched, doing "info cpus" in monitor kill the guest, which,
unfortunately, means that reworked VCPU state writeback is still not so
easy to use correctly.

--
Gleb.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] vhost-net: switch to smp barriers

2010-02-07 Thread Michael S. Tsirkin
On Sun, Feb 07, 2010 at 11:42:29AM +0200, Avi Kivity wrote:
> On 02/01/2010 07:21 PM, Michael S. Tsirkin wrote:
>> vhost-net only uses memory barriers to control SMP effects
>> (communication with userspace potentially running on a different CPU),
>> so it should use SMP barriers and not mandatory barriers for memory
>> access ordering, as suggested by Documentation/memory-barriers.txt
>>
>>
>
> A UP guest running on an SMP host still needs those barriers.

Correct. And since vhost net is running host-side, smp_XX
barriers will do exactly the right thing, right?

> -- 
> error compiling committee.c: too many arguments to function
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] vhost-net: switch to smp barriers

2010-02-07 Thread Avi Kivity

On 02/07/2010 11:44 AM, Michael S. Tsirkin wrote:

On Sun, Feb 07, 2010 at 11:42:29AM +0200, Avi Kivity wrote:
   

On 02/01/2010 07:21 PM, Michael S. Tsirkin wrote:
 

vhost-net only uses memory barriers to control SMP effects
(communication with userspace potentially running on a different CPU),
so it should use SMP barriers and not mandatory barriers for memory
access ordering, as suggested by Documentation/memory-barriers.txt


   

A UP guest running on an SMP host still needs those barriers.
 

Correct. And since vhost net is running host-side, smp_XX
barriers will do exactly the right thing, right?
   


Right, of course.  Mixed up virtio and vhost.

--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 02/11] iommu-api: Add iommu_map and iommu_unmap functions

2010-02-07 Thread Joerg Roedel
On Sun, Feb 07, 2010 at 11:38:30AM +0200, Avi Kivity wrote:
> On 01/28/2010 01:37 PM, Joerg Roedel wrote:
>> These two functions provide support for mapping and
>> unmapping physical addresses to io virtual addresses. The
>> difference to the iommu_(un)map_range() is that the new
>> functions take a gfp_order parameter instead of a size. This
>> allows the IOMMU backend implementations to detect easier if
>> a given range can be mapped by larger page sizes.
>> These new functions should replace the old ones in the long
>> term.
>>
>
> These seem to be less flexible in the long term.  Sure, it is easier for  
> the backend to map to multiple page sizes if your iommu supports  
> arbitrary power-of-two page sizes, but we should make the APIs easier  
> for the callers, not the backend.

These functions are as flexible as the old ones which just tok a size.
The benefit of the new interface is that is makes the ability of the
IOMMU to map the area with a large page (an get the benefit of fewer
hardware tlb walks) visible to the caller. With the old interface the
caller is tempted to just map ist whole area using 4kb page sizes.
It gives a bit more complexity to the caller, thats right. But given
that the page allocator in Linux always returns pages which are aligned
to its size takes a lot of that complexity away.

Joerg

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: KVM problems with Xeon L5530

2010-02-07 Thread Avi Kivity

On 01/28/2010 01:02 PM, Matteo Ghezzi wrote:

Upgraded to 2.6.33-rc5, same qemu-kvm version, same hardware.
The Vnc screen is still black and the virtual machine, but now I don't
have anymore messages in the logs but directly at standard output:

KVM internal error. Suberror: 2
extra data[0]: 8010
extra data[1]: 8b0d

rip 014a rflags 00010002
cs c000 (000c/ p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
   


That is basically the same error.

The guest runs the vga vios and hangs.  Please try with a production 
processor.


--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 02/11] iommu-api: Add iommu_map and iommu_unmap functions

2010-02-07 Thread Avi Kivity

On 02/07/2010 12:50 PM, Joerg Roedel wrote:

On Sun, Feb 07, 2010 at 11:38:30AM +0200, Avi Kivity wrote:
   

On 01/28/2010 01:37 PM, Joerg Roedel wrote:
 

These two functions provide support for mapping and
unmapping physical addresses to io virtual addresses. The
difference to the iommu_(un)map_range() is that the new
functions take a gfp_order parameter instead of a size. This
allows the IOMMU backend implementations to detect easier if
a given range can be mapped by larger page sizes.
These new functions should replace the old ones in the long
term.

   

These seem to be less flexible in the long term.  Sure, it is easier for
the backend to map to multiple page sizes if your iommu supports
arbitrary power-of-two page sizes, but we should make the APIs easier
for the callers, not the backend.
 

These functions are as flexible as the old ones which just tok a size.
The benefit of the new interface is that is makes the ability of the
IOMMU to map the area with a large page (an get the benefit of fewer
hardware tlb walks) visible to the caller. With the old interface the
caller is tempted to just map ist whole area using 4kb page sizes.
It gives a bit more complexity to the caller, thats right. But given
that the page allocator in Linux always returns pages which are aligned
to its size takes a lot of that complexity away.

   


You are right - I was thinking of the kvm use case which is range 
oriented, but the ordinary kernel interfaces are gfp_order oriented.


--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v2 00/21] qemu-kvm: Hook cleanups and extended use of upstream code

2010-02-07 Thread Jan Kiszka
Gleb Natapov wrote:
> On Wed, Feb 03, 2010 at 09:53:25AM +0100, Jan Kiszka wrote:
>> This version addresses the feedback on v2, namely:
>> - assert( || thread>) on low-level
>>   load/save registers
>> - fixed mpstate initialization
>>
> With those patched, doing "info cpus" in monitor kill the guest, which,
> unfortunately, means that reworked VCPU state writeback is still not so
> easy to use correctly.
> 

Regression of patch 12 ("qemu-kvm: Use upstream kvm_vcpu_dirty"), will
be fixed in v3.

Thanks,
Jan

---

diff --git a/qemu-kvm.c b/qemu-kvm.c
index 6a72b89..72c2ca0 100644
--- a/qemu-kvm.c
+++ b/qemu-kvm.c
@@ -1532,13 +1532,13 @@ static void do_kvm_cpu_synchronize_state(void *_env)
 CPUState *env = _env;
 
 kvm_arch_save_regs(env);
+env->kvm_vcpu_dirty = 1;
 }
 
 void kvm_cpu_synchronize_state(CPUState *env)
 {
 if (!env->kvm_vcpu_dirty) {
 on_vcpu(env, do_kvm_cpu_synchronize_state, env);
-env->kvm_vcpu_dirty = 1;
 }
 }
 



signature.asc
Description: OpenPGP digital signature


Re: [PATCH 01/10] arch/ia64: Fix continuation line formats

2010-02-07 Thread Avi Kivity

On 02/02/2010 09:22 AM, Joe Perches wrote:

String constants that are continued on subsequent lines with \
are not good.
   


Applied, thanks.

--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 4/11] arch/x86/kvm: Correct NULL test

2010-02-07 Thread Avi Kivity

On 02/06/2010 10:43 AM, Julia Lawall wrote:

From: Julia Lawall

msr was tested above, so the second test is not needed.

   


Applied, thanks.

--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v2 00/21] qemu-kvm: Hook cleanups and extended use of upstream code

2010-02-07 Thread Gleb Natapov
On Sun, Feb 07, 2010 at 12:28:49PM +0100, Jan Kiszka wrote:
> Gleb Natapov wrote:
> > On Wed, Feb 03, 2010 at 09:53:25AM +0100, Jan Kiszka wrote:
> >> This version addresses the feedback on v2, namely:
> >> - assert( || thread>) on low-level
> >>   load/save registers
> >> - fixed mpstate initialization
> >>
> > With those patched, doing "info cpus" in monitor kill the guest, which,
> > unfortunately, means that reworked VCPU state writeback is still not so
> > easy to use correctly.
> > 
> 
> Regression of patch 12 ("qemu-kvm: Use upstream kvm_vcpu_dirty"), will
> be fixed in v3.
> 
Looks better now indeed. Thanks.

--
Gleb.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: KVM: VMX: update cr0 read shadow when deactivating cr0.ts passthrough

2010-02-07 Thread Avi Kivity

On 02/01/2010 06:48 PM, Marcelo Tosatti wrote:

On fpu deactivation, the cr0 read shadow is not properly updated, since
it assumes vcpu->arch.cr0 contains the guest visible cr0 value before
guest had control of cr0.ts.

This is not true, since cr0 has been decached (from vmx_fpu_deactivate
itself or somewhere else).

Fix by unconditionally updating cr0 read shadow (this is not a hot path,
in comparison with entry/exit).

Fixes FC8 64 install.
   


Applied, thanks.

--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 7/8] KVM: Optimize kvm_read_cr[04]_bits()

2010-02-07 Thread Avi Kivity

On 02/05/2010 10:26 AM, Jan Kiszka wrote:

Avi Kivity wrote:
   

'mask' is always a constant, so we can check whether it includes a bit that
might be owned by the guest very cheaply, and avoid the decache call.  Saves
a few hundred bytes of module text.
 

-no-kvm-irqchip -smp 2 is broken for my Linux guests since this commit.
Their user space applications receive #UD and things fall apart.

   


Does 2c8232fc help?

--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 05/11] kvm: Introduce kvm_host_page_size

2010-02-07 Thread Avi Kivity

On 01/28/2010 01:37 PM, Joerg Roedel wrote:

This patch introduces a generic function to find out the
host page size for a given gfn. This function is needed by
the kvm iommu code. This patch also simplifies the x86
host_mapping_level function.
   


Applied to kvm.git since this makes sense independently of the rest of 
your patchset.


--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 06/11] kvm: Change kvm_iommu_map_pages to map large pages

2010-02-07 Thread Avi Kivity

On 01/29/2010 12:24 AM, Marcelo Tosatti wrote:

On Thu, Jan 28, 2010 at 12:37:57PM +0100, Joerg Roedel wrote:
   

This patch changes the implementation of of
kvm_iommu_map_pages to map the pages with the host page size
into the io virtual address space.

Signed-off-by: Joerg Roedel
---
  virt/kvm/iommu.c |  106 ++---
  1 files changed, 84 insertions(+), 22 deletions(-)

diff --git a/virt/kvm/iommu.c b/virt/kvm/iommu.c
index 65a5143..92a434d 100644
--- a/virt/kvm/iommu.c
+++ b/virt/kvm/iommu.c
@@ -32,12 +32,27 @@ static int kvm_iommu_unmap_memslots(struct kvm *kvm);
  static void kvm_iommu_put_pages(struct kvm *kvm,
gfn_t base_gfn, unsigned long npages);

+static pfn_t kvm_pin_pages(struct kvm *kvm, struct kvm_memory_slot *slot,
+  gfn_t gfn, unsigned long size)
+{
+   gfn_t end_gfn;
+   pfn_t pfn;
+
+   pfn = gfn_to_pfn_memslot(kvm, slot, gfn);
 

If gfn_to_pfn_memslot returns pfn of bad_page, you might create a
large iommu translation for it?
   


How could it return a bad_page?  The whole thing is called for a valid slot.

--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 06/11] kvm: Change kvm_iommu_map_pages to map large pages

2010-02-07 Thread Avi Kivity

On 02/05/2010 01:01 PM, Joerg Roedel wrote:



Yes, addresses the concern.
 

Are there any further objections against this patchset? If not it would
be cool if you could give me some acks for the kvm specific parts of
this patchset.
   


There are two ways we can get the kvm bits in:

- you publish a git tree excluding the kvm patches, I merge your tree, 
then apply the kvm specific patches.  This is best for avoiding 
merge-time conflicts if I get other patches for the same area.

- you push everything including the kvm bits.

I'm fine with both, though prefer the former.  If you choose the latter, 
ACK for the kvm patches.


--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 3/8] KVM: Activate fpu on clts

2010-02-07 Thread Avi Kivity

On 02/04/2010 07:56 PM, Avi Kivity wrote:

vmx.c doesn't initialize kvm_x86_ops->fpu_activate as far as I see.



Gaak.  Well, that's obviously unintended.



I committed a patch to fix this.

--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 02/18] KVM: PPC: Enable MMIO to do 64 bits, fprs and qprs

2010-02-07 Thread Avi Kivity

On 02/04/2010 05:55 PM, Alexander Graf wrote:

Right now MMIO access can only happen for GPRs and is at most 32 bit wide.
That's actually enough for almost all types of hardware out there.

Unfortunately, the guest I was using used FPU writes to MMIO regions, so
it ended up writing 64 bit MMIOs using FPRs and QPRs.

So let's add code to handle those odd cases too.

Signed-off-by: Alexander Graf
---
  arch/powerpc/include/asm/kvm.h |7 +++
  arch/powerpc/include/asm/kvm_ppc.h |2 +-
  arch/powerpc/kvm/powerpc.c |   24 ++--
  3 files changed, 30 insertions(+), 3 deletions(-)

diff --git a/arch/powerpc/include/asm/kvm.h b/arch/powerpc/include/asm/kvm.h
index 81f3b0b..548376c 100644
--- a/arch/powerpc/include/asm/kvm.h
+++ b/arch/powerpc/include/asm/kvm.h
@@ -77,4 +77,11 @@ struct kvm_debug_exit_arch {
  struct kvm_guest_debug_arch {
  };

+#define REG_MASK   0x001f
+#define REG_EXT_MASK   0xffe0
+#define REG_GPR0x
+#define REG_FPR0x0020
+#define REG_QPR0x0040
+#define REG_FQPR   0x0060
   


These names seem too generic to belong in asm/kvm.h - some application 
could use the same names.  Please add a KVM_ prefix.


--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 03/18] KVM: PPC: Teach MMIO Signedness

2010-02-07 Thread Avi Kivity

On 02/04/2010 05:55 PM, Alexander Graf wrote:

The guest I was trying to get to run uses the LHA and LHAU instructions.
Those instructions basically do a load, but also sign extend the result.

Since we need to fill our registers by hand when doing MMIO, we also need
to sign extend manually.

This patch implements sign extended MMIO and the LHA(U) instructions.

@@ -300,6 +300,25 @@ static void kvmppc_complete_mmio_load(struct kvm_vcpu 
*vcpu,
}
}

+   if (vcpu->arch.mmio_sign_extend) {
+   switch (run->mmio.len) {
+#ifdef CONFIG_PPC64
+   case 4:
+   if (gpr&  0x8000)
+   gpr |= 0xULL;
+   break;
   


Wouldn't

  gpr = (s64)(gpr << 32) >> 32;

work?  Not sure if >> is guaranteed to sign extend.


--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 7/8] KVM: Optimize kvm_read_cr[04]_bits()

2010-02-07 Thread Jan Kiszka
Avi Kivity wrote:
> On 02/05/2010 10:26 AM, Jan Kiszka wrote:
>> Avi Kivity wrote:
>>   
>>> 'mask' is always a constant, so we can check whether it includes a
>>> bit that
>>> might be owned by the guest very cheaply, and avoid the decache
>>> call.  Saves
>>> a few hundred bytes of module text.
>>>  
>> -no-kvm-irqchip -smp 2 is broken for my Linux guests since this commit.
>> Their user space applications receive #UD and things fall apart.
>>
>>
> 
> Does 2c8232fc help?
> 

Nope, unfortunately.

Jan



signature.asc
Description: OpenPGP digital signature


Re: [PATCH 7/8] KVM: Optimize kvm_read_cr[04]_bits()

2010-02-07 Thread Avi Kivity

On 02/07/2010 02:33 PM, Jan Kiszka wrote:

Avi Kivity wrote:
   

On 02/05/2010 10:26 AM, Jan Kiszka wrote:
 

Avi Kivity wrote:

   

'mask' is always a constant, so we can check whether it includes a
bit that
might be owned by the guest very cheaply, and avoid the decache
call.  Saves
a few hundred bytes of module text.

 

-no-kvm-irqchip -smp 2 is broken for my Linux guests since this commit.
Their user space applications receive #UD and things fall apart.


   

Does 2c8232fc help?

 

Nope, unfortunately.

   


Which specific guests exhibit the behaviour?

--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 06/11] kvm: Change kvm_iommu_map_pages to map large pages

2010-02-07 Thread Joerg Roedel
On Sun, Feb 07, 2010 at 02:22:35PM +0200, Avi Kivity wrote:
> On 02/05/2010 01:01 PM, Joerg Roedel wrote:
>>
>>> Yes, addresses the concern.
>>>  
>> Are there any further objections against this patchset? If not it would
>> be cool if you could give me some acks for the kvm specific parts of
>> this patchset.
>>
>
> There are two ways we can get the kvm bits in:
>
> - you publish a git tree excluding the kvm patches, I merge your tree,  
> then apply the kvm specific patches.  This is best for avoiding  
> merge-time conflicts if I get other patches for the same area.
> - you push everything including the kvm bits.
>
> I'm fine with both, though prefer the former.  If you choose the latter,  
> ACK for the kvm patches.

Hmm, the patch set is self-contained and hard to split. We had the same
problem with the initial iommu-api merge. Since I have further updates
for the AMD IOMMU driver I think its best to send the these patches and
my updates directly to Linus after the KVM updates are merged in the
next merge window. Thanks for your ACKs :-)

Joerg

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 18/18] KVM: PPC: Implement Paired Single emulation

2010-02-07 Thread Avi Kivity

On 02/04/2010 05:55 PM, Alexander Graf wrote:

The one big thing about the Gekko is paired singles.

Paired singles are an extension to the instruction set, that adds 32 single
precision floating point registers (qprs), some SPRs to modify the behavior
of paired singled operations and instructions to deal with qprs to the
instruction set.

Unfortunately, it also changes semantics of existing operations that affect
single values in FPRs. In most cases they get mirrored to the coresponding
QPR.

Thanks to that we need to emulate all FPU operations and all the new paired
single operations too.

In order to achieve that, we take the guest's instruction, rip out the
parameters, put in our own and execute the very same instruction, but also
fix up the QPR values along the way.

That way we can execute paired single FPU operations without implementing a
soft fpu.

   


A little frightening.  How many instructions are there?  Maybe we can 
just have an array of all of them followed by a return instruction, so 
we don't jit code.



static void call_fpu_inst(u32 inst, u64 *out, u64 *in1, u64 *in2, u64 *in3,
+ u32 *cr, u32 *fpscr)
+{
+   u32 cr_val = 0;
+   u32 *call_stack;
+   u64 inout[5] = { 0, 0, 0, 0, 0 };
+
+   if (fpscr)
+   inout[0] = *fpscr;
+   if (in1)
+   inout[1] = *in1;
+   if (in2)
+   inout[2] = *in2;
+   if (in3)
+   inout[3] = *in3;
+   if (cr)
+   cr_val = *cr;
+
+   dprintk(KERN_INFO "FPU Emulator 0x%x ( 0x%llx, 0x%llx, 0x%llx )", inst,
+   inout[1], inout[2], inout[3]);
+
+   call_stack =&kvmppc_call_stack[(smp_processor_id() * 2)];
+   call_stack[0] = inst;
+   /* call_stack[1] is INS_BLR */
+
   


Would be easier on the cache to do this per-cpu?

--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 00/18] KVM: PPC: Virtualize Gekko guests

2010-02-07 Thread Avi Kivity

On 02/04/2010 05:55 PM, Alexander Graf wrote:

In an effort to get KVM on PPC more useful for other userspace users than
Qemu, I figured it'd be a nice idea to implement virtualization of the
Gekko CPU.

The Gekko is the CPU used in the GameCube. In a slightly more modern
fashion it lives on in the Wii today.

Using this patch set and a modified version of Dolphin, I was able to
virtualize simple GameCube demos on a 970MP system.

As always, while getting this to run I stumbled across several broken
parts and fixed them as they came up. So expect some bug fixes in this
patch set too.
   


This is halfway into emulation rather than virtualization.  What does 
performance look like when running fpu intensive applications?


I might have missed it, but I didn't see the KVM_CAP and save/restore 
support for this.


--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v2 01/21] qemu-kvm: Drop vmport changes

2010-02-07 Thread Avi Kivity

On 02/03/2010 10:53 AM, Jan Kiszka wrote:

This attempt to make vmport KVM compatible is half-broken and is
scheduled to be replaced by proper upstream support.
   


Does "scheduled" mean you have patches for adding 
cpu_synchronize_state() to vmport?


--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 7/8] KVM: Optimize kvm_read_cr[04]_bits()

2010-02-07 Thread Jan Kiszka
Avi Kivity wrote:
> On 02/07/2010 02:33 PM, Jan Kiszka wrote:
>> Avi Kivity wrote:
>>   
>>> On 02/05/2010 10:26 AM, Jan Kiszka wrote:
>>> 
 Avi Kivity wrote:

   
> 'mask' is always a constant, so we can check whether it includes a
> bit that
> might be owned by the guest very cheaply, and avoid the decache
> call.  Saves
> a few hundred bytes of module text.
>
>  
 -no-kvm-irqchip -smp 2 is broken for my Linux guests since this commit.
 Their user space applications receive #UD and things fall apart.



>>> Does 2c8232fc help?
>>>
>>>  
>> Nope, unfortunately.
>>
>>
> 
> Which specific guests exhibit the behaviour?
> 

64-bit Linux, various kernels from 2.6.27 to 32. x86-32 appears to be
unaffected.

Jan



signature.asc
Description: OpenPGP digital signature


Re: [PATCH v2 14/21] qemu-kvm: Rework VCPU state writeback API

2010-02-07 Thread Avi Kivity

On 02/03/2010 10:53 AM, Jan Kiszka wrote:

This grand cleanup drops all reset and vmsave/load related
synchronization points in favor of four(!) generic hooks:

- cpu_synchronize_all_states in qemu_savevm_state_complete
   (initial sync from kernel before vmsave)
- cpu_synchronize_all_post_init in qemu_loadvm_state
   (writeback after vmload)
- cpu_synchronize_all_post_init in main after machine init
- cpu_synchronize_all_post_reset in qemu_system_reset
   (writeback after system reset)

These writeback points + the existing one of VCPU exec after
cpu_synchronize_state map on three levels of writeback:

- KVM_PUT_ASYNC_STATE (during runtime, other VCPUs continue to run)
   


Wouldn't that be SYNC_STATE (state that is modified by the current vcpu 
only)?



- KVM_PUT_RESET_STATE (on synchronous system reset, all VCPUs stopped)
- KVM_PUT_FULL_STATE  (on init or vmload, all VCPUs stopped as well)

This level is passed to the arch-specific VCPU state writing function
that will decide which concrete substates need to be written. That way,
no writer of load, save or reset functions that interact with in-kernel
KVM states will ever have to worry about synchronization again. That
also means that a lot of reasons for races, segfaults and deadlocks are
eliminated.

cpu_synchronize_state remains untouched, just as Anthony suggested. We
continue to need it before reading or writing of VCPU states that are
also tracked by in-kernel KVM subsystems.

Consequently, this patch removes many cpu_synchronize_state calls that
are now redundant, just like remaining explicit register syncs. It does
not touch qemu-kvm's special hooks for mpstate, vcpu_events, or tsc
loading. They will be cleaned up by individual patches.

   


I'm uneasy about this.  What are the rules for putting 
cpu_synchronize_state() now?



--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v2 14/21] qemu-kvm: Rework VCPU state writeback API

2010-02-07 Thread Jan Kiszka
Avi Kivity wrote:
> On 02/03/2010 10:53 AM, Jan Kiszka wrote:
>> This grand cleanup drops all reset and vmsave/load related
>> synchronization points in favor of four(!) generic hooks:
>>
>> - cpu_synchronize_all_states in qemu_savevm_state_complete
>>(initial sync from kernel before vmsave)
>> - cpu_synchronize_all_post_init in qemu_loadvm_state
>>(writeback after vmload)
>> - cpu_synchronize_all_post_init in main after machine init
>> - cpu_synchronize_all_post_reset in qemu_system_reset
>>(writeback after system reset)
>>
>> These writeback points + the existing one of VCPU exec after
>> cpu_synchronize_state map on three levels of writeback:
>>
>> - KVM_PUT_ASYNC_STATE (during runtime, other VCPUs continue to run)
>>
> 
> Wouldn't that be SYNC_STATE (state that is modified by the current vcpu
> only)?

It's async /wrt other VCPUs. They continue to run and may interact with
this VCPU while updating its state.

> 
>> - KVM_PUT_RESET_STATE (on synchronous system reset, all VCPUs stopped)
>> - KVM_PUT_FULL_STATE  (on init or vmload, all VCPUs stopped as well)
>>
>> This level is passed to the arch-specific VCPU state writing function
>> that will decide which concrete substates need to be written. That way,
>> no writer of load, save or reset functions that interact with in-kernel
>> KVM states will ever have to worry about synchronization again. That
>> also means that a lot of reasons for races, segfaults and deadlocks are
>> eliminated.
>>
>> cpu_synchronize_state remains untouched, just as Anthony suggested. We
>> continue to need it before reading or writing of VCPU states that are
>> also tracked by in-kernel KVM subsystems.
>>
>> Consequently, this patch removes many cpu_synchronize_state calls that
>> are now redundant, just like remaining explicit register syncs. It does
>> not touch qemu-kvm's special hooks for mpstate, vcpu_events, or tsc
>> loading. They will be cleaned up by individual patches.
>>
>>
> 
> I'm uneasy about this.  What are the rules for putting
> cpu_synchronize_state() now?

As before for code that accesses the state during runtime: Before you
read or write some bit of it, call cpu_synchronize_state().

Only reset and save/restore handlers do not have to worry about
synchronization anymore. It makes no sense to overload them with
arch-specific KVM knowledge about what shall be written and when.

Jan



signature.asc
Description: OpenPGP digital signature


Re: [PATCH v2 01/21] qemu-kvm: Drop vmport changes

2010-02-07 Thread Jan Kiszka
Avi Kivity wrote:
> On 02/03/2010 10:53 AM, Jan Kiszka wrote:
>> This attempt to make vmport KVM compatible is half-broken and is
>> scheduled to be replaced by proper upstream support.
>>
> 
> Does "scheduled" mean you have patches for adding
> cpu_synchronize_state() to vmport?

See patch 2 in this series, and Marcelo already queued that one for
upstream.

Jan



signature.asc
Description: OpenPGP digital signature


Re: [PATCH v2 14/21] qemu-kvm: Rework VCPU state writeback API

2010-02-07 Thread Avi Kivity

On 02/07/2010 03:51 PM, Jan Kiszka wrote:

Avi Kivity wrote:
   

On 02/03/2010 10:53 AM, Jan Kiszka wrote:
 

This grand cleanup drops all reset and vmsave/load related
synchronization points in favor of four(!) generic hooks:

- cpu_synchronize_all_states in qemu_savevm_state_complete
(initial sync from kernel before vmsave)
- cpu_synchronize_all_post_init in qemu_loadvm_state
(writeback after vmload)
- cpu_synchronize_all_post_init in main after machine init
- cpu_synchronize_all_post_reset in qemu_system_reset
(writeback after system reset)

These writeback points + the existing one of VCPU exec after
cpu_synchronize_state map on three levels of writeback:

- KVM_PUT_ASYNC_STATE (during runtime, other VCPUs continue to run)

   

Wouldn't that be SYNC_STATE (state that is modified by the current vcpu
only)?
 

It's async /wrt other VCPUs. They continue to run and may interact with
this VCPU while updating its state.
   


Well, to me it makes more sense to name them from the point of view of 
the vcpu that is doing the update.



I'm uneasy about this.  What are the rules for putting
cpu_synchronize_state() now?
 

As before for code that accesses the state during runtime: Before you
read or write some bit of it, call cpu_synchronize_state().

Only reset and save/restore handlers do not have to worry about
synchronization anymore. It makes no sense to overload them with
arch-specific KVM knowledge about what shall be written and when.
   


OK.

--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 05/11] kvm: Introduce kvm_host_page_size

2010-02-07 Thread Joerg Roedel
On Sun, Feb 07, 2010 at 02:09:36PM +0200, Avi Kivity wrote:
> On 01/28/2010 01:37 PM, Joerg Roedel wrote:
>> This patch introduces a generic function to find out the
>> host page size for a given gfn. This function is needed by
>> the kvm iommu code. This patch also simplifies the x86
>> host_mapping_level function.
>>
>
> Applied to kvm.git since this makes sense independently of the rest of  
> your patchset.

Yes, makes sense for this patch. I will rebase my branch once you pushed
it out.

Joerg

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v2 14/21] qemu-kvm: Rework VCPU state writeback API

2010-02-07 Thread Jan Kiszka
Avi Kivity wrote:
> On 02/07/2010 03:51 PM, Jan Kiszka wrote:
>> Avi Kivity wrote:
>>   
>>> On 02/03/2010 10:53 AM, Jan Kiszka wrote:
>>> 
 This grand cleanup drops all reset and vmsave/load related
 synchronization points in favor of four(!) generic hooks:

 - cpu_synchronize_all_states in qemu_savevm_state_complete
 (initial sync from kernel before vmsave)
 - cpu_synchronize_all_post_init in qemu_loadvm_state
 (writeback after vmload)
 - cpu_synchronize_all_post_init in main after machine init
 - cpu_synchronize_all_post_reset in qemu_system_reset
 (writeback after system reset)

 These writeback points + the existing one of VCPU exec after
 cpu_synchronize_state map on three levels of writeback:

 - KVM_PUT_ASYNC_STATE (during runtime, other VCPUs continue to run)


>>> Wouldn't that be SYNC_STATE (state that is modified by the current vcpu
>>> only)?
>>>  
>> It's async /wrt other VCPUs. They continue to run and may interact with
>> this VCPU while updating its state.
>>
> 
> Well, to me it makes more sense to name them from the point of view of
> the vcpu that is doing the update.

I'm open for a better name - except for "sync" as writebacks are always
synchronous from the POV of the modified VCPU. Is KVM_PUT_RUNTIME_STATE
clearer?

Jan



signature.asc
Description: OpenPGP digital signature


Re: [PATCH v2 14/21] qemu-kvm: Rework VCPU state writeback API

2010-02-07 Thread Avi Kivity

On 02/07/2010 04:26 PM, Jan Kiszka wrote:



Well, to me it makes more sense to name them from the point of view of
the vcpu that is doing the update.
 

I'm open for a better name - except for "sync" as writebacks are always
synchronous from the POV of the modified VCPU.


I meant that the state itself is synchronous from the POV of the vcpu.  
That is, rax can only be changed by the core, but INIT state may be 
changed by the other cores.



  Is KVM_PUT_RUNTIME_STATE
clearer?
   


They're all runtime.  I guess this issue will always be confusing, so 
best to slap a comment on top of the define to explain it.


--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 00/18] KVM: PPC: Virtualize Gekko guests

2010-02-07 Thread Alexander Graf

Am 07.02.2010 um 13:54 schrieb Avi Kivity :


On 02/04/2010 05:55 PM, Alexander Graf wrote:
In an effort to get KVM on PPC more useful for other userspace  
users than
Qemu, I figured it'd be a nice idea to implement virtualization of  
the

Gekko CPU.

The Gekko is the CPU used in the GameCube. In a slightly more modern
fashion it lives on in the Wii today.

Using this patch set and a modified version of Dolphin, I was able to
virtualize simple GameCube demos on a 970MP system.

As always, while getting this to run I stumbled across several broken
parts and fixed them as they came up. So expect some bug fixes in  
this

patch set too.



This is halfway into emulation rather than virtualization.  What  
does performance look like when running fpu intensive applications?


It is for the FPU. It is not for whatever runs on the CPU.

I haven't benchmarked things so far,

The only two choices I have to get this running is in-kernel emulation  
or userspace emulation. According to how x86 deals with things I  
suppose full state transition to userspace and continuing emulation  
there isn't considered a good idea. So I went with in-kernel.




I might have missed it, but I didn't see the KVM_CAP and save/ 
restore support for this.


Ah, cap again. Right. Mind if I send an patch on top of the set?

As far as save/restore goes, the ioctl to get/set fprs isn't even  
implemented (yet)! We're really off full state migration to/from  
userspace yet.


Alex
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 03/18] KVM: PPC: Teach MMIO Signedness

2010-02-07 Thread Alexander Graf


Am 07.02.2010 um 13:32 schrieb Avi Kivity :


On 02/04/2010 05:55 PM, Alexander Graf wrote:
The guest I was trying to get to run uses the LHA and LHAU  
instructions.
Those instructions basically do a load, but also sign extend the  
result.


Since we need to fill our registers by hand when doing MMIO, we  
also need

to sign extend manually.

This patch implements sign extended MMIO and the LHA(U) instructions.

@@ -300,6 +300,25 @@ static void kvmppc_complete_mmio_load(struct  
kvm_vcpu *vcpu,

 }
 }

+if (vcpu->arch.mmio_sign_extend) {
+switch (run->mmio.len) {
+#ifdef CONFIG_PPC64
+case 4:
+if (gpr&  0x8000)
+gpr |= 0xULL;
+break;



Wouldn't

 gpr = (s64)(gpr << 32) >> 32;

work?  Not sure if >> is guaranteed to sign extend.


Not sure either. The code as is is rather obvious imho, so I wouldn't  
want to replace it with anything that's even remotely magical.


Alex
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 02/18] KVM: PPC: Enable MMIO to do 64 bits, fprs and qprs

2010-02-07 Thread Alexander Graf


Am 07.02.2010 um 13:29 schrieb Avi Kivity :


On 02/04/2010 05:55 PM, Alexander Graf wrote:
Right now MMIO access can only happen for GPRs and is at most 32  
bit wide.

That's actually enough for almost all types of hardware out there.

Unfortunately, the guest I was using used FPU writes to MMIO  
regions, so

it ended up writing 64 bit MMIOs using FPRs and QPRs.

So let's add code to handle those odd cases too.

Signed-off-by: Alexander Graf
---
 arch/powerpc/include/asm/kvm.h |7 +++
 arch/powerpc/include/asm/kvm_ppc.h |2 +-
 arch/powerpc/kvm/powerpc.c |   24 ++--
 3 files changed, 30 insertions(+), 3 deletions(-)

diff --git a/arch/powerpc/include/asm/kvm.h b/arch/powerpc/include/ 
asm/kvm.h

index 81f3b0b..548376c 100644
--- a/arch/powerpc/include/asm/kvm.h
+++ b/arch/powerpc/include/asm/kvm.h
@@ -77,4 +77,11 @@ struct kvm_debug_exit_arch {
 struct kvm_guest_debug_arch {
 };

+#define REG_MASK0x001f
+#define REG_EXT_MASK0xffe0
+#define REG_GPR0x
+#define REG_FPR0x0020
+#define REG_QPR0x0040
+#define REG_FQPR0x0060



These names seem too generic to belong in asm/kvm.h - some  
application could use the same names.  Please add a KVM_ prefix.


Yes, will do.

Alex
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 18/18] KVM: PPC: Implement Paired Single emulation

2010-02-07 Thread Alexander Graf


Am 07.02.2010 um 13:50 schrieb Avi Kivity :


On 02/04/2010 05:55 PM, Alexander Graf wrote:

The one big thing about the Gekko is paired singles.

Paired singles are an extension to the instruction set, that adds  
32 single
precision floating point registers (qprs), some SPRs to modify the  
behavior
of paired singled operations and instructions to deal with qprs to  
the

instruction set.

Unfortunately, it also changes semantics of existing operations  
that affect
single values in FPRs. In most cases they get mirrored to the  
coresponding

QPR.

Thanks to that we need to emulate all FPU operations and all the  
new paired

single operations too.

In order to achieve that, we take the guest's instruction, rip out  
the
parameters, put in our own and execute the very same instruction,  
but also

fix up the QPR values along the way.

That way we can execute paired single FPU operations without  
implementing a

soft fpu.




A little frightening.  How many instructions are there?  Maybe we  
can just have an array of all of them followed by a return  
instruction, so we don't jit code.


There's all the instructions in the list, most can have the rc  
(compare) bit set to modify CC and iirc there were a couple ones with  
immediate values.


But maybe you're right. I probably could just always set rc and either  
ignore the result or use it. I could maybe find alternatives to  
immediate using instructions. Let me check this on the bus trip back  
from brussels.




static void call_fpu_inst(u32 inst, u64 *out, u64 *in1, u64 *in2,  
u64 *in3,

+  u32 *cr, u32 *fpscr)
+{
+u32 cr_val = 0;
+u32 *call_stack;
+u64 inout[5] = { 0, 0, 0, 0, 0 };
+
+if (fpscr)
+inout[0] = *fpscr;
+if (in1)
+inout[1] = *in1;
+if (in2)
+inout[2] = *in2;
+if (in3)
+inout[3] = *in3;
+if (cr)
+cr_val = *cr;
+
+dprintk(KERN_INFO "FPU Emulator 0x%x ( 0x%llx, 0x%llx, 0x 
%llx )", inst,

+inout[1], inout[2], inout[3]);
+
+call_stack =&kvmppc_call_stack[(smp_processor_id() * 2)];
+call_stack[0] = inst;
+/* call_stack[1] is INS_BLR */
+



Would be easier on the cache to do this per-cpu?


It is per-cpu. Or do you mean to actually use the PER_CPU definition?  
Is that guaranteed to be executable?


Alex
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 03/18] KVM: PPC: Teach MMIO Signedness

2010-02-07 Thread Avi Kivity

On 02/07/2010 05:51 PM, Alexander Graf wrote:

+if (vcpu->arch.mmio_sign_extend) {
+switch (run->mmio.len) {
+#ifdef CONFIG_PPC64
+case 4:
+if (gpr&  0x8000)
+gpr |= 0xULL;
+break;



Wouldn't

 gpr = (s64)(gpr << 32) >> 32;

work?  Not sure if >> is guaranteed to sign extend.



Not sure either. The code as is is rather obvious imho, so I wouldn't 
want to replace it with anything that's even remotely magical.




That's fair.

--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 18/18] KVM: PPC: Implement Paired Single emulation

2010-02-07 Thread Avi Kivity

On 02/07/2010 05:57 PM, Alexander Graf wrote:+
+dprintk(KERN_INFO "FPU Emulator 0x%x ( 0x%llx, 0x%llx, 0x%llx 
)", inst,

+inout[1], inout[2], inout[3]);
+
+call_stack =&kvmppc_call_stack[(smp_processor_id() * 2)];
+call_stack[0] = inst;
+/* call_stack[1] is INS_BLR */
+



Would be easier on the cache to do this per-cpu?


It is per-cpu. Or do you mean to actually use the PER_CPU definition? 
Is that guaranteed to be executable?


I meant, per-cpu vmalloc area, but it should be enough to have a per-cpu 
cache line.


--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 00/18] KVM: PPC: Virtualize Gekko guests

2010-02-07 Thread Avi Kivity

On 02/07/2010 05:49 PM, Alexander Graf wrote:

Am 07.02.2010 um 13:54 schrieb Avi Kivity :


On 02/04/2010 05:55 PM, Alexander Graf wrote:
In an effort to get KVM on PPC more useful for other userspace users 
than

Qemu, I figured it'd be a nice idea to implement virtualization of the
Gekko CPU.

The Gekko is the CPU used in the GameCube. In a slightly more modern
fashion it lives on in the Wii today.

Using this patch set and a modified version of Dolphin, I was able to
virtualize simple GameCube demos on a 970MP system.

As always, while getting this to run I stumbled across several broken
parts and fixed them as they came up. So expect some bug fixes in this
patch set too.



This is halfway into emulation rather than virtualization.  What does 
performance look like when running fpu intensive applications?


It is for the FPU. It is not for whatever runs on the CPU.

I haven't benchmarked things so far,

The only two choices I have to get this running is in-kernel emulation 
or userspace emulation. According to how x86 deals with things I 
suppose full state transition to userspace and continuing emulation 
there isn't considered a good idea. So I went with in-kernel.


It's not a good idea for the kernel either, if it happens all the time.  
If a typical Gekko application uses the fpu and the emulated 
instructions intensively, performance will suck badly (as in: qemu/tcg 
will be faster).


--
error compiling committee.c: too many arguments to function

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [Qemu-devel] [PATCH] Add assignment operation to config file parser..

2010-02-07 Thread Anthony Liguori

On 02/06/2010 12:59 PM, john cooper wrote:

This patch reworks support for both assignment and
append in the config file parser.  It was motivated
by comments received on the cpu model config file
format.

Commit dc9ca4ba27be4fe6a0284061b8f056c4364fb0d9
changed the behavior of "=" from assign to append.
This patch preserves the ability to append to an
option (however now via "+="), reverts "=" to its
previous behavior, and allows both to interoperate.

Signed-off-by: john cooper
   


This deviates from standard ini syntax which makes me a big 
uncomfortable with it.  Gerd, do you have an opinion?


Regards,

Anthony Liguori

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 03/18] KVM: PPC: Teach MMIO Signedness

2010-02-07 Thread Anthony Liguori

On 02/07/2010 06:32 AM, Avi Kivity wrote:

On 02/04/2010 05:55 PM, Alexander Graf wrote:

The guest I was trying to get to run uses the LHA and LHAU instructions.
Those instructions basically do a load, but also sign extend the result.

Since we need to fill our registers by hand when doing MMIO, we also 
need

to sign extend manually.

This patch implements sign extended MMIO and the LHA(U) instructions.

@@ -300,6 +300,25 @@ static void kvmppc_complete_mmio_load(struct 
kvm_vcpu *vcpu,

  }
  }

+if (vcpu->arch.mmio_sign_extend) {
+switch (run->mmio.len) {
+#ifdef CONFIG_PPC64
+case 4:
+if (gpr&  0x8000)
+gpr |= 0xULL;
+break;


Wouldn't

  gpr = (s64)(gpr << 32) >> 32;

work?  Not sure if >> is guaranteed to sign extend.


It's technically implementation dependent but I don't know of an 
implementation that doesn't sign extend.


Regards,

Anthony Liguori


--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] KVM test: Ensure multiple pre/post commands can run

2010-02-07 Thread Michael Goldish

- "Lucas Meneghel Rodrigues"  wrote:

> The way tests are currently defined, running unattended
> install + hugepages will allways skip unattended install
> setup step (coincidentally the tests on our test farm
> were working because previous executions of the unattended
> install script ran, leaving the environment prepared for
> unattended install).
> 
> So, make sure pre_commands on default tests.cfg file are
> additive, and the preprocessor splits the pre_command
> string, and executes pre/post commands in sequence.
> 
> Signed-off-by: Lucas Meneghel Rodrigues 

Why not just append ';' to each command?

For example:

pre_command = "scripts/unattended.py;"
...
pre_command += " scripts/hugepage.py;"

(the quotes can be omitted)

IMO this is simpler.

Also, by using "".split() you're not allowing commands that
contain spaces.

> ---
>  client/tests/kvm/kvm_preprocessing.py  |   29
> -
>  client/tests/kvm/tests_base.cfg.sample |4 ++--
>  2 files changed, 18 insertions(+), 15 deletions(-)
> 
> diff --git a/client/tests/kvm/kvm_preprocessing.py
> b/client/tests/kvm/kvm_preprocessing.py
> index 8a0c151..2e35d9f 100644
> --- a/client/tests/kvm/kvm_preprocessing.py
> +++ b/client/tests/kvm/kvm_preprocessing.py
> @@ -126,7 +126,7 @@ def postprocess_vm(test, params, env, name):
>  vm.destroy(gracefully = params.get("kill_vm_gracefully") ==
> "yes")
>  
>  
> -def process_command(test, params, env, command, command_timeout,
> +def process_command(test, params, env, commands, command_timeout,
>  command_noncritical):
>  """
>  Pre- or post- custom commands to be executed before/after a test
> is run
> @@ -134,22 +134,23 @@ def process_command(test, params, env, command,
> command_timeout,
>  @param test: An Autotest test object.
>  @param params: A dict containing all VM and image parameters.
>  @param env: The environment (a dict-like object).
> -@param command: Command to be run.
> +@param commands: List of commands to be run.
>  @param command_timeout: Timeout for command execution.
>  @param command_noncritical: If True test will not fail if command
> fails.
>  """
>  # Export environment vars
>  for k in params.keys():
>  os.putenv("KVM_TEST_%s" % k, str(params[k]))
> -# Execute command
> -try:
> -utils.system("cd %s; %s" % (test.bindir, command))
> -except error.CmdError, e:
> -logging.warn("Custom processing command '%s' failed, output
> is: %s",
> - command, str(e))
> -if not command_noncritical:
> -raise error.TestError("Custom processing command failed:
> %s" %
> -  str(e))
> +# Execute commands
> +for command in commands:
> +try:
> +utils.system("cd %s; %s" % (test.bindir, command))
> +except error.CmdError, e:
> +logging.warn("Custom processing command '%s' failed,
> output is: %s",
> + command, str(e))
> +if not command_noncritical:
> +raise error.TestError("Custom processing command
> failed: %s" %
> +  str(e))
>  
>  
>  def process(test, params, env, image_func, vm_func):
> @@ -220,7 +221,8 @@ def preprocess(test, params, env):
>  
>  # Execute any pre_commands
>  if params.get("pre_command"):
> -process_command(test, params, env,
> params.get("pre_command"),
> +pre_commands = params.get("pre_command").spit()
> +process_command(test, params, env, pre_commands,
>  int(params.get("pre_command_timeout",
> "600")),
>  params.get("pre_command_noncritical") ==
> "yes")
>  
> @@ -296,7 +298,8 @@ def postprocess(test, params, env):
>  
>  # Execute any post_commands
>  if params.get("post_command"):
> -process_command(test, params, env,
> params.get("post_command"),
> +post_commands = params.get("post_command").split()
> +process_command(test, params, env, post_commands,
>  int(params.get("post_command_timeout",
> "600")),
>  params.get("post_command_noncritical") ==
> "yes")
>  
> diff --git a/client/tests/kvm/tests_base.cfg.sample
> b/client/tests/kvm/tests_base.cfg.sample
> index 91daf48..8f88f3b 100644
> --- a/client/tests/kvm/tests_base.cfg.sample
> +++ b/client/tests/kvm/tests_base.cfg.sample
> @@ -66,7 +66,7 @@ variants:
>  kill_vm_gracefully = yes
>  kill_vm_on_error = yes
>  force_create_image = yes
> -pre_command = scripts/unattended.py
> +pre_command += " scripts/unattended.py"
>  floppy = "images/floppy.img"
>  extra_params += " -boot d"
>  nic_mode = user
> @@ -953,7 +953,7 @@ variants:
>  variants:
>  - @smallpages:
>  - hugepages:
> -pre_command = "/usr/bin/python scripts/hugepage.py
> /mnt/kvm_hugepage"
> 

Re: [PATCH] KVM: tests_base.cfg: major guest OS cleanup

2010-02-07 Thread Michael Goldish

- "Lucas Meneghel Rodrigues"  wrote:

> Major update and cleanup on guest OS definition:
> 
>  * Fedora 8, 9, 10 dropped
>  * RHEL versions bumped to the latest stable versions
>  * Windows 2008, Vista and 7 bumped to the last versions
>  * Some duplicate information re-organized

Looks like you dropped all step file tests, and I don't think
this is a good idea.  Keeping them is harmless, so I don't think
they should be dropped unless they're completely useless.
If you don't want them to run, keep them out of the sample test
sets, but by removing them entirely from the sample config file,
you're making it much harder for those who use them to keep
using them.

Of course, if we become quite confident that no one will ever want
to use step files again, getting rid of them isn't bad.  I do
however think that some people will keep using them, and I even
think it's a good idea to run a quick step file test as part of
our daily routine on autotest.virt.bos, just to make sure KVM
displays stuff properly and responds to keyboard input.

Regarding reorganizing duplicate information:
You don't need to get rid of step file tests in order to avoid
config code duplication.  You can reuse information like this:

- 11.32:
image_name = fc11-32
...
(common stuff)
...
install:
steps = Fedora-11-32.steps
(install specific stuff)
unattended_install:
unattended_file = unattended/something
(unattended_install specific stuff)

Regarding version bumping:
Since bumping guest OS versions breaks step file installs, I think
we should either keep the old versions next to the new ones in the
same config file, and exclude them from sample test sets, or keep
them in a separate config file, e.g. tests_base_old.cfg.sample
(or tests_base.cfg.sample.old?).
In any case, I don't think we should remove the old versions.
This also applies to the old Fedoras.

> 
> Signed-off-by: Lucas Meneghel Rodrigues 
> ---
>  client/tests/kvm/tests_base.cfg.sample  |  286
> ---
>  client/tests/kvm/unattended/winxp64.sif |   73 
>  2 files changed, 149 insertions(+), 210 deletions(-)
>  create mode 100644 client/tests/kvm/unattended/winxp64.sif
> 
> diff --git a/client/tests/kvm/tests_base.cfg.sample
> b/client/tests/kvm/tests_base.cfg.sample
> index 8f88f3b..110be98 100644
> --- a/client/tests/kvm/tests_base.cfg.sample
> +++ b/client/tests/kvm/tests_base.cfg.sample
> @@ -311,47 +311,8 @@ variants:
>  shell_prompt = "^\[.*\][\#\$]\s*$"
>  
>  variants:
> -- 8.32:
> -no setup
> -image_name = fc8-32
> -install:
> -steps = Fedora-8-i386.steps
> -cdrom = linux/Fedora-8-i386-DVD.iso
> -md5sum =
> dd6c79fddfff36d409d02242e7b10189
> -md5sum_1m =
> dabae451bb69fbbad0e505b25144b1f9
> -
> -- 8.64:
> -no setup
> -image_name = fc8-64
> -install:
> -steps = Fedora-8-64.steps
> -cdrom = linux/Fedora-8-x86_64-DVD.iso
> -md5sum =
> 2cb231a86709dec413425fd2f8bf5295
> -md5sum_1m =
> 145f6414e19492649a56c89f0a45e719
> -
> -- 9.32:
> -image_name = fc9-32
> -install:
> -steps = Fedora-9-i386.steps
> -cdrom = linux/Fedora-9-i386-DVD.iso
> -md5sum =
> 72601f685ea8c808c303353d8bf4d307
> -md5sum_1m =
> f24fa25689e5863f1b99984c6feb787f
> -
> -- 9.64:
> -image_name = fc9-64
> -install:
> -steps = Fedora-9-64.steps
> -cdrom = linux/Fedora-9-x86_64-DVD.iso
> -md5sum =
> 05b2ebeed273ec54d6f9ed3d61ea4c96
> -md5sum_1m =
> 9822ab5097e37e8fe306ef2192727db4
> -
>  - 11.32:
>  image_name = fc11-32
> -install:
> -steps = Fedora-11-32.steps
> -cdrom = linux/Fedora-11-i386-DVD.iso
> -md5sum =
> e3b1e2d1ba42aa4705fa5f41771b3927
> -md5sum_1m =
> dc8ddf90648c247339c721395aa49714
>  unattended_install:
>  cdrom = linux/Fedora-11-i386-DVD.iso
>  md5sum =
> e3b1e2d1ba42aa4705fa5f41771b3927
> @@ -363,12 +324,6 @@ variants:
>  
>  - 11.64:
>  image_name = fc11-64
> -install:
> -  

Re: [PATCH 06/11] kvm: Change kvm_iommu_map_pages to map large pages

2010-02-07 Thread Marcelo Tosatti
On Sun, Feb 07, 2010 at 02:18:19PM +0200, Avi Kivity wrote:
> On 01/29/2010 12:24 AM, Marcelo Tosatti wrote:
> >On Thu, Jan 28, 2010 at 12:37:57PM +0100, Joerg Roedel wrote:
> >>This patch changes the implementation of of
> >>kvm_iommu_map_pages to map the pages with the host page size
> >>into the io virtual address space.
> >>
> >>Signed-off-by: Joerg Roedel
> >>---
> >>  virt/kvm/iommu.c |  106 
> >> ++---
> >>  1 files changed, 84 insertions(+), 22 deletions(-)
> >>
> >>diff --git a/virt/kvm/iommu.c b/virt/kvm/iommu.c
> >>index 65a5143..92a434d 100644
> >>--- a/virt/kvm/iommu.c
> >>+++ b/virt/kvm/iommu.c
> >>@@ -32,12 +32,27 @@ static int kvm_iommu_unmap_memslots(struct kvm *kvm);
> >>  static void kvm_iommu_put_pages(struct kvm *kvm,
> >>gfn_t base_gfn, unsigned long npages);
> >>
> >>+static pfn_t kvm_pin_pages(struct kvm *kvm, struct kvm_memory_slot *slot,
> >>+  gfn_t gfn, unsigned long size)
> >>+{
> >>+   gfn_t end_gfn;
> >>+   pfn_t pfn;
> >>+
> >>+   pfn = gfn_to_pfn_memslot(kvm, slot, gfn);
> >If gfn_to_pfn_memslot returns pfn of bad_page, you might create a
> >large iommu translation for it?
> 
> How could it return a bad_page?  The whole thing is called for a valid slot.

Userspace can pass a valid slot with a read-only vma, for example.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] emulate accessed bit for EPT

2010-02-07 Thread Marcelo Tosatti
On Fri, Feb 05, 2010 at 07:14:13PM +0100, Andrea Arcangeli wrote:
> On Fri, Feb 05, 2010 at 03:34:23PM -0200, Marcelo Tosatti wrote:
> > But perhaps a module parameter to turn accessed bit emulation off might
> > be handy in the future?
> 
> Maybe, but somebody should show that this can overall become a
> downside, which I doubt... I think if it does, the VM is to blame for
> calling page_referenced when there is no point to do so just yet.

Agreed. ACK.

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 03/18] KVM: PPC: Teach MMIO Signedness

2010-02-07 Thread Alexander Graf


Am 07.02.2010 um 17:27 schrieb Anthony Liguori :


On 02/07/2010 06:32 AM, Avi Kivity wrote:

On 02/04/2010 05:55 PM, Alexander Graf wrote:
The guest I was trying to get to run uses the LHA and LHAU  
instructions.
Those instructions basically do a load, but also sign extend the  
result.


Since we need to fill our registers by hand when doing MMIO, we  
also need

to sign extend manually.

This patch implements sign extended MMIO and the LHA(U)  
instructions.


@@ -300,6 +300,25 @@ static void kvmppc_complete_mmio_load(struct  
kvm_vcpu *vcpu,

 }
 }

+if (vcpu->arch.mmio_sign_extend) {
+switch (run->mmio.len) {
+#ifdef CONFIG_PPC64
+case 4:
+if (gpr&  0x8000)
+gpr |= 0xULL;
+break;


Wouldn't

 gpr = (s64)(gpr << 32) >> 32;

work?  Not sure if >> is guaranteed to sign extend.


It's technically implementation dependent but I don't know of an  
implementation that doesn't sign extend.


Hrm, would

gpr = (s64)(s32)gpr;

work? :)


Alex
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 00/18] KVM: PPC: Virtualize Gekko guests

2010-02-07 Thread Alexander Graf
Avi Kivity wrote:
> On 02/07/2010 05:49 PM, Alexander Graf wrote:
>> Am 07.02.2010 um 13:54 schrieb Avi Kivity :
>>
>>> On 02/04/2010 05:55 PM, Alexander Graf wrote:
 In an effort to get KVM on PPC more useful for other userspace
 users than
 Qemu, I figured it'd be a nice idea to implement virtualization of the
 Gekko CPU.

 The Gekko is the CPU used in the GameCube. In a slightly more modern
 fashion it lives on in the Wii today.

 Using this patch set and a modified version of Dolphin, I was able to
 virtualize simple GameCube demos on a 970MP system.

 As always, while getting this to run I stumbled across several broken
 parts and fixed them as they came up. So expect some bug fixes in this
 patch set too.

>>>
>>> This is halfway into emulation rather than virtualization.  What
>>> does performance look like when running fpu intensive applications?
>>
>> It is for the FPU. It is not for whatever runs on the CPU.
>>
>> I haven't benchmarked things so far,
>>
>> The only two choices I have to get this running is in-kernel
>> emulation or userspace emulation. According to how x86 deals with
>> things I suppose full state transition to userspace and continuing
>> emulation there isn't considered a good idea. So I went with in-kernel.
>
> It's not a good idea for the kernel either, if it happens all the
> time.  If a typical Gekko application uses the fpu and the emulated
> instructions intensively, performance will suck badly (as in: qemu/tcg
> will be faster).
>

Yeah, I haven't really gotten far enough to run full-blown guests yet.
So far I'm on demos and they look pretty good.

But as far as intercept speed goes - I just tried running this little
piece of code in kvmctl:

.global _start
_start:
lir3, 42
mtsprg0, r3
mfsprgr4, 0
b_start

and measured the amount of exits I get on my test machine:

processor: 0
cpu: PPC970MP, altivec supported
clock: 2500.00MHz
revision: 1.1 (pvr 0044 0101)

--->

exits  1811108

I have no idea how we manage to get that many exits, but apparently we
are. So I'm less concerned about the speed of the FPU rerouting at the
moment.

If it really gets unusably slow, I'd rather binary patch the guest on
the fly in KVM according to rules set by the userspace client. But we'll
get there when it turns out to be too slow. For now I'd rather like to
have something working at all and then improve speed :-).

Alex
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 03/18] KVM: PPC: Teach MMIO Signedness

2010-02-07 Thread Anthony Liguori

On 02/07/2010 03:35 PM, Alexander Graf wrote:
It's technically implementation dependent but I don't know of an 
implementation that doesn't sign extend.



Hrm, would

gpr = (s64)(s32)gpr;

work? :)


Yes.  Integer promotion does guarantee sign extension.

Regards,

Anthony Liguori



Alex


--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] KVM: tests_base.cfg: major guest OS cleanup

2010-02-07 Thread Lucas Meneghel Rodrigues
On Sun, 2010-02-07 at 13:24 -0500, Michael Goldish wrote:

Let me explain the reasoning behind this:

> Looks like you dropped all step file tests, and I don't think
> this is a good idea.  Keeping them is harmless, so I don't think
> they should be dropped unless they're completely useless.
> If you don't want them to run, keep them out of the sample test
> sets, but by removing them entirely from the sample config file,
> you're making it much harder for those who use them to keep
> using them.

Good point, I didn't mean to do it just for the sake of not having them
anymore:
 * In the vast majority of the cases, I have downloaded and tested the
latest ISOS from the OS vendors, and tested it with the unattended
files. I do admit that I didn't try the step files, but I assumed they
wouldn't work so that's why I dropped them.

* Removing older versions of distributions like Fedora have good reason:
The support cycle of them is very short (18 months), by their very
nature (Fedora is a technology preview, we are allways moving on). So
it's not reasonable to keep testing guest OS that might have bugs that
no one will care to fix.

* As for the longer living versions of OS, like windows and RHEL,
there's a clear version for using the latest versions as well: The OS
vendors usually provide the updates (I mean inside a same family of that
OS, like RHEL 3.X, or windows2008) and they strongly advise System
Adminsitrators to allways use the latest version when installing new
systems.

In other words, let's say we find a bug on RHEL 5.3. The very first
thing we'll be asked to is to try reproducing the problem on 5.4, and if
it doesn't, the bug will be closed (same if you are using windows
[version] SP X, they will allways ask you to try the latest SP). So IMHO
it's important to focus our testing resources on the versions of the OS
that will be used on new installs by the general IT public. So I looked
after the latest ISOS that the vendors had to offer and tested only with
step install, since I didn't have to change the unattended files for
doing so.

> Of course, if we become quite confident that no one will ever want
> to use step files again, getting rid of them isn't bad.  I do
> however think that some people will keep using them, and I even
> think it's a good idea to run a quick step file test as part of
> our daily routine on autotest.virt.bos, just to make sure KVM
> displays stuff properly and responds to keyboard input.

Yes, I do think step installs are important:

 * Good method to verify input and image display, like you just
mentioned;
 * They are truly a universal way to get OSs installed. So if the given
OS has no reasonable unattended install system, we have step file based
install ready to rescue us.

So in terms of maintance, we are better of keeping unattended installs
for the OSs we know how to do it, but keep up to date step files for the
guest OSs that we can't use with unattended. Like, bump the versions for
all the OSs that we can't do unattended version, and have up to date
step files for those.

> Regarding reorganizing duplicate information:
> You don't need to get rid of step file tests in order to avoid
> config code duplication.  You can reuse information like this:
> 
> - 11.32:
> image_name = fc11-32
> ...
> (common stuff)
> ...
> install:
> steps = Fedora-11-32.steps
> (install specific stuff)
> unattended_install:
> unattended_file = unattended/something
> (unattended_install specific stuff)

Sure, most of step file removals were due to the fact that I introduced
new ISOs, like I explained before.

> Regarding version bumping:
> Since bumping guest OS versions breaks step file installs, I think
> we should either keep the old versions next to the new ones in the
> same config file, and exclude them from sample test sets, or keep
> them in a separate config file, e.g. tests_base_old.cfg.sample
> (or tests_base.cfg.sample.old?).
> In any case, I don't think we should remove the old versions.
> This also applies to the old Fedoras.

We could keep them, but like I've said, we should focus our testing
resources using the latest versions provided by the OS vendors.


--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] KVM: tests_base.cfg: major guest OS cleanup

2010-02-07 Thread Marcelo Tosatti
On Sun, Feb 07, 2010 at 09:26:21PM -0200, Lucas Meneghel Rodrigues wrote:
> On Sun, 2010-02-07 at 13:24 -0500, Michael Goldish wrote:
> 
> Let me explain the reasoning behind this:
> 
> > Looks like you dropped all step file tests, and I don't think
> > this is a good idea.  Keeping them is harmless, so I don't think
> > they should be dropped unless they're completely useless.
> > If you don't want them to run, keep them out of the sample test
> > sets, but by removing them entirely from the sample config file,
> > you're making it much harder for those who use them to keep
> > using them.
> 
> Good point, I didn't mean to do it just for the sake of not having them
> anymore:
>  * In the vast majority of the cases, I have downloaded and tested the
> latest ISOS from the OS vendors, and tested it with the unattended
> files. I do admit that I didn't try the step files, but I assumed they
> wouldn't work so that's why I dropped them.
> 
> * Removing older versions of distributions like Fedora have good reason:
> The support cycle of them is very short (18 months), by their very
> nature (Fedora is a technology preview, we are allways moving on). 
> So it's not reasonable to keep testing guest OS that might have bugs
> that no one will care to fix.

Disagree. Testing of older guests can trigger KVM bugs which are not
seen during testing of newer ones. 

> * As for the longer living versions of OS, like windows and RHEL,
> there's a clear version for using the latest versions as well: The OS
> vendors usually provide the updates (I mean inside a same family of that
> OS, like RHEL 3.X, or windows2008) and they strongly advise System
> Adminsitrators to allways use the latest version when installing new
> systems.
> 
> In other words, let's say we find a bug on RHEL 5.3. The very first
> thing we'll be asked to is to try reproducing the problem on 5.4, and if
> it doesn't, the bug will be closed (same if you are using windows
> [version] SP X, they will allways ask you to try the latest SP).

Not really. If the test succeeded before, and it regresses after qemu or 
KVM changes, its a bug we should care about.

> So IMHO
> it's important to focus our testing resources on the versions of the OS
> that will be used on new installs by the general IT public. So I looked
> after the latest ISOS that the vendors had to offer and tested only with
> step install, since I didn't have to change the unattended files for
> doing so.
> 
> > Of course, if we become quite confident that no one will ever want
> > to use step files again, getting rid of them isn't bad.  I do
> > however think that some people will keep using them, and I even
> > think it's a good idea to run a quick step file test as part of
> > our daily routine on autotest.virt.bos, just to make sure KVM
> > displays stuff properly and responds to keyboard input.
> 
> Yes, I do think step installs are important:
> 
>  * Good method to verify input and image display, like you just
> mentioned;
>  * They are truly a universal way to get OSs installed. So if the given
> OS has no reasonable unattended install system, we have step file based
> install ready to rescue us.
> 
> So in terms of maintance, we are better of keeping unattended installs
> for the OSs we know how to do it, but keep up to date step files for the
> guest OSs that we can't use with unattended. Like, bump the versions for
> all the OSs that we can't do unattended version, and have up to date
> step files for those.

Why don't keep both? 

> > Regarding reorganizing duplicate information:
> > You don't need to get rid of step file tests in order to avoid
> > config code duplication.  You can reuse information like this:
> > 
> > - 11.32:
> > image_name = fc11-32
> > ...
> > (common stuff)
> > ...
> > install:
> > steps = Fedora-11-32.steps
> > (install specific stuff)
> > unattended_install:
> > unattended_file = unattended/something
> > (unattended_install specific stuff)
> 
> Sure, most of step file removals were due to the fact that I introduced
> new ISOs, like I explained before.
> 
> > Regarding version bumping:
> > Since bumping guest OS versions breaks step file installs, I think
> > we should either keep the old versions next to the new ones in the
> > same config file, and exclude them from sample test sets, or keep
> > them in a separate config file, e.g. tests_base_old.cfg.sample
> > (or tests_base.cfg.sample.old?).
> > In any case, I don't think we should remove the old versions.
> > This also applies to the old Fedoras.
> 
> We could keep them, but like I've said, we should focus our testing
> resources using the latest versions provided by the OS vendors.

Agree, but please keep the old step file method around in the reposity
for all guests. Not only they can be useful if problems arise with
unattended install, but the screen comparison method that is used can
find bugs that unattended install cannot (issues that involves d

Re: [PATCH] KVM test: Ensure multiple pre/post commands can run

2010-02-07 Thread Lucas Meneghel Rodrigues
On Sun, 2010-02-07 at 12:59 -0500, Michael Goldish wrote:
> - "Lucas Meneghel Rodrigues"  wrote:
> 
> > The way tests are currently defined, running unattended
> > install + hugepages will allways skip unattended install
> > setup step (coincidentally the tests on our test farm
> > were working because previous executions of the unattended
> > install script ran, leaving the environment prepared for
> > unattended install).
> > 
> > So, make sure pre_commands on default tests.cfg file are
> > additive, and the preprocessor splits the pre_command
> > string, and executes pre/post commands in sequence.
> > 
> > Signed-off-by: Lucas Meneghel Rodrigues 
> 
> Why not just append ';' to each command?
> 
> For example:
> 
> pre_command = "scripts/unattended.py;"
> ...
> pre_command += " scripts/hugepage.py;"
> 
> (the quotes can be omitted)
> 
> IMO this is simpler.

Yes, agreed. Will revert the patch and fix this.

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 1/1 net-next] virtio_net: remove send queue

2010-02-07 Thread Rusty Russell
On Wed, 3 Feb 2010 08:18:51 am Shirley Ma wrote:
> Use detach buffers API in virtio to free unused buffers in send queue when 
> shutting down 
> virtio_net to avoid maintaining skb link list for each transmit packet.
> 
> Signed-off-by: Shirley Ma 

Hi Shirley,

  Nice cleanup.  Please cc: net...@vger.kernel.org for net patches...

Your comment could be slightly enhanced to indicate that the API is new, eg;

Now we have a virtio detach API (in commit XX), we don't need to
track xmit skbs in the  driver.

Acked-by: Rusty Russell 

Thanks!
Rusty.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] KVM: tests_base.cfg: major guest OS cleanup

2010-02-07 Thread Lucas Meneghel Rodrigues
On Sun, 2010-02-07 at 22:44 -0200, Marcelo Tosatti wrote:
> On Sun, Feb 07, 2010 at 09:26:21PM -0200, Lucas Meneghel Rodrigues wrote:

> > * Removing older versions of distributions like Fedora have good reason:
> > The support cycle of them is very short (18 months), by their very
> > nature (Fedora is a technology preview, we are allways moving on). 
> > So it's not reasonable to keep testing guest OS that might have bugs
> > that no one will care to fix.
> 
> Disagree. Testing of older guests can trigger KVM bugs which are not
> seen during testing of newer ones. 

Ok, that's a fair concern, we need to make sure that bugs found on older
guests will get enough attention though.
 
> > So in terms of maintance, we are better of keeping unattended installs
> > for the OSs we know how to do it, but keep up to date step files for the
> > guest OSs that we can't use with unattended. Like, bump the versions for
> > all the OSs that we can't do unattended version, and have up to date
> > step files for those.
> 
> Why don't keep both? 

Mostly because it was my understanding that we only cared about bugs on
the latest versions of each supported OS, given that premise it made
sense to me to keep only unattended install, which depends less on OS
details (for example, the same kickstart and answer files could be used
verbatim to both old and new versions of the OSes).

But now I see your point and Michael's, sorry, will bring back all the
step file install parameters back.

> > We could keep them, but like I've said, we should focus our testing
> > resources using the latest versions provided by the OS vendors.
> 
> Agree, but please keep the old step file method around in the reposity
> for all guests. Not only they can be useful if problems arise with
> unattended install, but the screen comparison method that is used can
> find bugs that unattended install cannot (issues that involves drawing
> the screen).

All righty, will get them all back during the morning!

Lucas

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] qemu-kvm: Speed up of the dirty-bitmap-traveling

2010-02-07 Thread OHMURA Kei
> Sounds logical - do you have numbers on the improvement?

Sure.  The patch showed approximately 3-7 times speed up when measured with 
rdtsc.  The test environment and detailed results are described below.

---
tmp = rdtsc();
/* function of original code*/
t1 += rdtsc() - tmp;

tmp = rdtsc();
/* function of this patch */
t2 += rdtsc() - tmp;
---

Test Envirionment:
CPU: 4x Intel Xeon Quad Core 2.66GHz
Mem size: 6GB
kvm version: 2.6.31-17-server
qemu version: commit ed880109f74f0a4dd5b7ec09e6a2d9ba4903d9a5

Host OS: Ubuntu 9.10 (kernel 2.6.31)
Guest OS: Debian/GNU Linux lenny (kernel 2.6.26)
Guest Mem size: 512MB

We executed live migration three times.  This data shows, how many times the 
function is called (#called), runtime of original (orig.), runtime of this 
patch (patch), speedup ratio (ratio), when live migration run.

Experimental results:
Test1: Guest OS read 3GB file, which is bigger than memory.
#called orig.(msec) patch(msec) ratio
114 1.000.156.76
132 1.570.256.26
96  1.000.166.27
 
Test2: Guest OS read/write 3GB file, which is bigger than memory.
#called orig.(msec) patch(msec) ratio
219638.110.63.59
225639.610.83.68
211236.310.33.53



> Would be great if you could provide a version for upstream as well
> because it will likely replace this qemu-kvm code on day.

O.K.  We'll prepare it.

We'll also post a patch set to quicken dirty pages checking in ram_save_block
and ram_save_live soon.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


vnc mouse trouble; -usbdevice tablet no help

2010-02-07 Thread Ross Boylan
I have been unable to get the real mouse and the virtualized mouse to
stay together when using kvm with linux host, guest and client.
Previous advice (to me and others) was to use -usbdevice tablet.  I've
tried that, and a variety of kvm/qemu versions, but no luck.

Can anyone suggest what to do, or how to diagnose it?

vnc and the mouse work OK for Windows XP guests.

Host system:
Intel(R) Xeon(R) CPU   E5420  @ 2.50GHz, 8 cores
qemu-kvm 0.12.2, built from source
linux 2.6.32-trunk-amd64 (stock Debian kernel)
guest linux 2.6.26-2-amd64

Invoked with
sudo vdeq bin/qemu-system-x86_64 -net
nic,vlan=1,macaddr=52:54:a0:14:01:01 \
-name "Lenny BCU Server" \
-vga std -vnc localhost:5 -usb -usbdevice tablet \
-net vde,vlan=1,sock=/var/run/vde2/tap0.ctl \
-boot c \
-hda /dev/turtle/Lenny06BCU \
-soundhw es1370 -m 1G -smp 2
I've tried without the -usb; I added it hoping it would help.  It
didn't.

My client has been either the host system or 32 bit linux system.
Mostly I've used krdc.

All systems basically Debian Lenny, with bits from later releases (e.g.,
the kernel on the host).

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html