Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Thu, Jun 27, 2013 at 04:09:50PM +0200, Stefan Pietsch wrote:
> On 26.06.2013 12:47, Gleb Natapov wrote:
> > On Mon, Jun 24, 2013 at 10:42:57PM +0200, Stefan Pietsch wrote:
> >> On 24.06.2013 14:30, Gleb Natapov wrote:
> >>> On Mon, Jun 24, 2013 at 01:59:34PM +0200, Stefan Pietsch wrote:
>  As soon as I remove "kvmvapic.bin" the virtual machine boots with
>  qemu-kvm 1.5.0. I just verified this with Linux kernel 3.10.0-rc5.
>  "emulate_invalid_guest_state=0" or "emulate_invalid_guest_state=1" make
>  no difference.
> 
>  Please send your patches.
> >>> Here it is, run with it and kvmvapic.bin present. See what is printed in
> >>> dmesg after the failure.
> >>>
> >>>
> >>> diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
> >>> index f4a5b3f..65488a4 100644
> >>> --- a/arch/x86/kvm/vmx.c
> >>> +++ b/arch/x86/kvm/vmx.c
> >>> @@ -3385,6 +3385,7 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
> >>>  {
> >>>   struct vcpu_vmx *vmx = to_vmx(vcpu);
> >>>   u32 ar;
> >>> + unsigned long rip;
> >>>  
> >>>   if (vmx->rmode.vm86_active && seg != VCPU_SREG_LDTR) {
> >>>   *var = vmx->rmode.segs[seg];
> >>> @@ -3408,6 +3409,9 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
> >>>   var->db = (ar >> 14) & 1;
> >>>   var->g = (ar >> 15) & 1;
> >>>   var->unusable = (ar >> 16) & 1;
> >>> + rip = kvm_rip_read(vcpu);
> >>> + if ((rip == 0xc101611c || rip == 0xc101611a) && seg == VCPU_SREG_FS)
> >>> + printk("base=%p limit=%p selector=%x ar=%x\n", var->base, 
> >>> var->limit, var->selector, ar);
> >>>  }
> >>>  
> >>>  static u64 vmx_get_segment_base(struct kvm_vcpu *vcpu, int seg)
> >>
> >>
> >> Booting kernel Linux 3.10-rc5 with your patch applied produces these
> >> messages in dmesg when starting a virtual machine:
> >>
> >> emulate_invalid_guest_state=0
> >> [  118.732151] base= limit=  (null) selector=ffff ar=0
> >> [  118.732341] base= limit=  (null) selector=ffff ar=0
> >>
> > I've butchered printk format, but it gives me the idea of what is going
> > on anyway. Can you try the patch below with
> > emulate_invalid_guest_state=0|1?
> > 
> > 
> > diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
> > index f4a5b3f..eb062ce 100644
> > --- a/arch/x86/kvm/vmx.c
> > +++ b/arch/x86/kvm/vmx.c
> > @@ -3395,19 +3395,20 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
> > var->selector = vmx_read_guest_seg_selector(vmx, seg);
> > return;
> > }
> > +
> > var->base = vmx_read_guest_seg_base(vmx, seg);
> > var->limit = vmx_read_guest_seg_limit(vmx, seg);
> > var->selector = vmx_read_guest_seg_selector(vmx, seg);
> > ar = vmx_read_guest_seg_ar(vmx, seg);
> > +   var->unusable = (ar >> 16) & 1;
> > var->type = ar & 15;
> > var->s = (ar >> 4) & 1;
> > var->dpl = (ar >> 5) & 3;
> > -   var->present = (ar >> 7) & 1;
> > +   var->present = !var->unusable;
> > var->avl = (ar >> 12) & 1;
> > var->l = (ar >> 13) & 1;
> > var->db = (ar >> 14) & 1;
> > var->g = (ar >> 15) & 1;
> > -   var->unusable = (ar >> 16) & 1;
> >  }
> >  
> >  static u64 vmx_get_segment_base(struct kvm_vcpu *vcpu, int seg)
> 
> 
> Kernel 3.10-rc5 with your latest patch applied can successfully boot the
> virtual machine with emulate_invalid_guest_state 0 or 1.
Excellent. I will send the patch to the list. Thank you for your help
in tracking it.

--
Gleb.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 26.06.2013 12:47, Gleb Natapov wrote:
> On Mon, Jun 24, 2013 at 10:42:57PM +0200, Stefan Pietsch wrote:
>> On 24.06.2013 14:30, Gleb Natapov wrote:
>>> On Mon, Jun 24, 2013 at 01:59:34PM +0200, Stefan Pietsch wrote:
 As soon as I remove "kvmvapic.bin" the virtual machine boots with
 qemu-kvm 1.5.0. I just verified this with Linux kernel 3.10.0-rc5.
 "emulate_invalid_guest_state=0" or "emulate_invalid_guest_state=1" make
 no difference.

 Please send your patches.
>>> Here it is, run with it and kvmvapic.bin present. See what is printed in
>>> dmesg after the failure.
>>>
>>>
>>> diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
>>> index f4a5b3f..65488a4 100644
>>> --- a/arch/x86/kvm/vmx.c
>>> +++ b/arch/x86/kvm/vmx.c
>>> @@ -3385,6 +3385,7 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
>>>  {
>>> struct vcpu_vmx *vmx = to_vmx(vcpu);
>>> u32 ar;
>>> +   unsigned long rip;
>>>  
>>> if (vmx->rmode.vm86_active && seg != VCPU_SREG_LDTR) {
>>> *var = vmx->rmode.segs[seg];
>>> @@ -3408,6 +3409,9 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
>>> var->db = (ar >> 14) & 1;
>>> var->g = (ar >> 15) & 1;
>>> var->unusable = (ar >> 16) & 1;
>>> +   rip = kvm_rip_read(vcpu);
>>> +   if ((rip == 0xc101611c || rip == 0xc101611a) && seg == VCPU_SREG_FS)
>>> +   printk("base=%p limit=%p selector=%x ar=%x\n", var->base, 
>>> var->limit, var->selector, ar);
>>>  }
>>>  
>>>  static u64 vmx_get_segment_base(struct kvm_vcpu *vcpu, int seg)
>>
>>
>> Booting kernel Linux 3.10-rc5 with your patch applied produces these
>> messages in dmesg when starting a virtual machine:
>>
>> emulate_invalid_guest_state=0
>> [  118.732151] base= limit=  (null) selector=ffff ar=0
>> [  118.732341] base= limit=  (null) selector=ffff ar=0
>>
> I've butchered printk format, but it gives me the idea of what is going
> on anyway. Can you try the patch below with
> emulate_invalid_guest_state=0|1?
> 
> 
> diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
> index f4a5b3f..eb062ce 100644
> --- a/arch/x86/kvm/vmx.c
> +++ b/arch/x86/kvm/vmx.c
> @@ -3395,19 +3395,20 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
>   var->selector = vmx_read_guest_seg_selector(vmx, seg);
>   return;
>   }
> +
>   var->base = vmx_read_guest_seg_base(vmx, seg);
>   var->limit = vmx_read_guest_seg_limit(vmx, seg);
>   var->selector = vmx_read_guest_seg_selector(vmx, seg);
>   ar = vmx_read_guest_seg_ar(vmx, seg);
> + var->unusable = (ar >> 16) & 1;
>   var->type = ar & 15;
>   var->s = (ar >> 4) & 1;
>   var->dpl = (ar >> 5) & 3;
> - var->present = (ar >> 7) & 1;
> + var->present = !var->unusable;
>   var->avl = (ar >> 12) & 1;
>   var->l = (ar >> 13) & 1;
>   var->db = (ar >> 14) & 1;
>   var->g = (ar >> 15) & 1;
> - var->unusable = (ar >> 16) & 1;
>  }
>  
>  static u64 vmx_get_segment_base(struct kvm_vcpu *vcpu, int seg)


Kernel 3.10-rc5 with your latest patch applied can successfully boot the
virtual machine with emulate_invalid_guest_state 0 or 1.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Mon, Jun 24, 2013 at 10:42:57PM +0200, Stefan Pietsch wrote:
> On 24.06.2013 14:30, Gleb Natapov wrote:
> > On Mon, Jun 24, 2013 at 01:59:34PM +0200, Stefan Pietsch wrote:
> >> As soon as I remove "kvmvapic.bin" the virtual machine boots with
> >> qemu-kvm 1.5.0. I just verified this with Linux kernel 3.10.0-rc5.
> >> "emulate_invalid_guest_state=0" or "emulate_invalid_guest_state=1" make
> >> no difference.
> >>
> >> Please send your patches.
> > Here it is, run with it and kvmvapic.bin present. See what is printed in
> > dmesg after the failure.
> > 
> > 
> > diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
> > index f4a5b3f..65488a4 100644
> > --- a/arch/x86/kvm/vmx.c
> > +++ b/arch/x86/kvm/vmx.c
> > @@ -3385,6 +3385,7 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
> >  {
> > struct vcpu_vmx *vmx = to_vmx(vcpu);
> > u32 ar;
> > +   unsigned long rip;
> >  
> > if (vmx->rmode.vm86_active && seg != VCPU_SREG_LDTR) {
> > *var = vmx->rmode.segs[seg];
> > @@ -3408,6 +3409,9 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
> > var->db = (ar >> 14) & 1;
> > var->g = (ar >> 15) & 1;
> > var->unusable = (ar >> 16) & 1;
> > +   rip = kvm_rip_read(vcpu);
> > +   if ((rip == 0xc101611c || rip == 0xc101611a) && seg == VCPU_SREG_FS)
> > +   printk("base=%p limit=%p selector=%x ar=%x\n", var->base, 
> > var->limit, var->selector, ar);
> >  }
> >  
> >  static u64 vmx_get_segment_base(struct kvm_vcpu *vcpu, int seg)
> 
> 
> Booting kernel Linux 3.10-rc5 with your patch applied produces these
> messages in dmesg when starting a virtual machine:
> 
> emulate_invalid_guest_state=0
> [  118.732151] base= limit=  (null) selector=ffff ar=0
> [  118.732341] base= limit=  (null) selector=ffff ar=0
> 
I've butchered printk format, but it gives me the idea of what is going
on anyway. Can you try the patch below with
emulate_invalid_guest_state=0|1?


diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index f4a5b3f..eb062ce 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -3395,19 +3395,20 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
var->selector = vmx_read_guest_seg_selector(vmx, seg);
return;
}
+
var->base = vmx_read_guest_seg_base(vmx, seg);
var->limit = vmx_read_guest_seg_limit(vmx, seg);
var->selector = vmx_read_guest_seg_selector(vmx, seg);
ar = vmx_read_guest_seg_ar(vmx, seg);
+   var->unusable = (ar >> 16) & 1;
var->type = ar & 15;
var->s = (ar >> 4) & 1;
var->dpl = (ar >> 5) & 3;
-   var->present = (ar >> 7) & 1;
+   var->present = !var->unusable;
var->avl = (ar >> 12) & 1;
var->l = (ar >> 13) & 1;
var->db = (ar >> 14) & 1;
var->g = (ar >> 15) & 1;
-   var->unusable = (ar >> 16) & 1;
 }
 
 static u64 vmx_get_segment_base(struct kvm_vcpu *vcpu, int seg)

--
Gleb.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 24.06.2013 14:30, Gleb Natapov wrote:
> On Mon, Jun 24, 2013 at 01:59:34PM +0200, Stefan Pietsch wrote:
>> As soon as I remove "kvmvapic.bin" the virtual machine boots with
>> qemu-kvm 1.5.0. I just verified this with Linux kernel 3.10.0-rc5.
>> "emulate_invalid_guest_state=0" or "emulate_invalid_guest_state=1" make
>> no difference.
>>
>> Please send your patches.
> Here it is, run with it and kvmvapic.bin present. See what is printed in
> dmesg after the failure.
> 
> 
> diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
> index f4a5b3f..65488a4 100644
> --- a/arch/x86/kvm/vmx.c
> +++ b/arch/x86/kvm/vmx.c
> @@ -3385,6 +3385,7 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
>  {
>   struct vcpu_vmx *vmx = to_vmx(vcpu);
>   u32 ar;
> + unsigned long rip;
>  
>   if (vmx->rmode.vm86_active && seg != VCPU_SREG_LDTR) {
>   *var = vmx->rmode.segs[seg];
> @@ -3408,6 +3409,9 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
>   var->db = (ar >> 14) & 1;
>   var->g = (ar >> 15) & 1;
>   var->unusable = (ar >> 16) & 1;
> + rip = kvm_rip_read(vcpu);
> + if ((rip == 0xc101611c || rip == 0xc101611a) && seg == VCPU_SREG_FS)
> + printk("base=%p limit=%p selector=%x ar=%x\n", var->base, 
> var->limit, var->selector, ar);
>  }
>  
>  static u64 vmx_get_segment_base(struct kvm_vcpu *vcpu, int seg)


Booting kernel Linux 3.10-rc5 with your patch applied produces these
messages in dmesg when starting a virtual machine:

emulate_invalid_guest_state=0
[  118.732151] base= limit=  (null) selector=ffff ar=0
[  118.732341] base= limit=  (null) selector=ffff ar=0

emulate_invalid_guest_state=1
[  196.481653] base= limit=  (null) selector=ffff ar=0
[  196.481700] base= limit=  (null) selector=ffff ar=0
[  196.481706] base= limit=  (null) selector=ffff ar=0
[  196.481711] base= limit=  (null) selector=ffff ar=0
[  196.481716] base= limit=  (null) selector=ffff ar=0
[  196.481720] base= limit=  (null) selector=ffff ar=0
[  196.481725] base= limit=  (null) selector=ffff ar=0
[  196.481730] base= limit=  (null) selector=ffff ar=0
[  196.481735] base= limit=  (null) selector=ffff ar=0
[  196.481739] base= limit=  (null) selector=ffff ar=0
[  196.481777] base= limit=  (null) selector=ffff ar=0
[  196.482068] base= limit=  (null) selector=ffff ar=0
[  196.482073] base= limit=  (null) selector=ffff ar=0
[  196.482079] base= limit=  (null) selector=ffff ar=0
[  196.482084] base= limit=  (null) selector=ffff ar=0
[  196.482131] base= limit=  (null) selector=ffff ar=0
[  196.482136] base= limit=  (null) selector=ffff ar=0
[  196.482142] base= limit=  (null) selector=ffff ar=0
[  196.482146] base= limit=  (null) selector=ffff ar=0
[  196.482193] base= limit=  (null) selector=ffff ar=0
[  196.482198] base= limit=  (null) selector=ffff ar=0
[  196.482203] base= limit=  (null) selector=ffff ar=0
[  196.482208] base= limit=  (null) selector=ffff ar=0
[  196.482255] base= limit=  (null) selector=ffff ar=0
[  196.482259] base= limit=  (null) selector=ffff ar=0
[  196.482265] base= limit=  (null) selector=ffff ar=0
[  196.482269] base= limit=  (null) selector=ffff ar=0
[  196.482316] base= limit=  (null) selector=ffff ar=0
[  196.482321] base= limit=  (null) selector=ffff ar=0
[  196.482326] base= limit=  (null) selector=ffff ar=0
[  196.482331] base= limit=  (null) selector=ffff ar=0
[  196.482378] base= limit=  (null) selector=ffff ar=0
[  196.482382] base= limit=  (null) selector=ffff ar=0
[  196.482388] base= limit=  (null) selector=ffff ar=0
[  196.482392] base= limit=  (null) selector=ffff ar=0
[  196.482439] base= limit=  (null) selector=ffff ar=0
[  196.482444] base= limit=  (null) selector=ffff ar=0
[  196.482449] base= limit=  (null) selector=ffff ar=0
[  196.482454] base= limit=  (null) selector=ffff ar=0
[  196.482501] base= limit=  (null) selector=ffff ar=0
[  196.482505] base= limit=  (null) selector=ffff ar=0
[  196.482511] base= limit=  (null) selector=ffff ar=0
[  196.482516] base= limit=  (null) selector=ffff ar=0
[  196.482562] base= limit=  (null) selector=ffff ar=0
[  196.482567] base= limit=  (null) selector=ffff ar=0
[  196.482573] base= limit=  (null) selector=ffff ar=0
[  196.482577] base= limit=  (null) selector=ffff ar=0
[  196.483137] base= limit=  (null) selector=ffff ar=0
[  196.483142] base= limit=  (null) selector=ffff ar=0
[  196.483147] base=

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Mon, Jun 24, 2013 at 01:59:34PM +0200, Stefan Pietsch wrote:
> On 24.06.2013 13:47, Gleb Natapov wrote:
> > On Mon, Jun 24, 2013 at 01:43:26PM +0200, Stefan Pietsch wrote:
> >> On 23.06.2013 19:36, Gleb Natapov wrote:
> >>> On Sun, Jun 23, 2013 at 06:51:30PM +0200, Stefan Pietsch wrote:
>  On 23.06.2013 09:51, Gleb Natapov wrote:
> > On Thu, Jun 20, 2013 at 07:01:49PM +0200, Stefan Pietsch wrote:
> >>> Can you provide the output of 25391454e73e3156202264eb3c473825afe4bc94
> >>> and emulate_invalid_guest_state=0. Also run "x/20i $pc-20" in qemu
> >>> monitor after the hang.
> >>
> >>
> >> 25391454e73e3156202264eb3c473825afe4bc94
> >>  emulate_invalid_guest_state=0
> >>
> > Very interesting. Looks like somewhere during TPR access FS
> > register gets corrupted. Can you remove /usr/share/kvm/kvmvapic.bin
> > and try again? This will disable some code paths during TPR access and
> > will narrow down the issue.
> 
> 
>  Doing this, qemu complains
>  "Could not open option rom 'kvmvapic.bin': No such file or directory",
>  but the virtual machine boots successful with
>  emulate_invalid_guest_state=0 and emulate_invalid_guest_state=1.
> 
> >>> Hmm, I think we ate close. Can you try with upstream qemu?
> >>>
>  kvmvapic.bin comes with Debian package "seabios 1.7.2-3".
> >>
> >> I already tried this with the Debian package qemu-kvm 1.5.0+dfsg-4.
> > And it didn't work? Mind trying some debug kernel patches? I suspect
> > your CPU does something no CPU I have do, so I want to verify it.
> 
> 
> As soon as I remove "kvmvapic.bin" the virtual machine boots with
> qemu-kvm 1.5.0. I just verified this with Linux kernel 3.10.0-rc5.
> "emulate_invalid_guest_state=0" or "emulate_invalid_guest_state=1" make
> no difference.
> 
> Please send your patches.
Here it is, run with it and kvmvapic.bin present. See what is printed in
dmesg after the failure.


diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index f4a5b3f..65488a4 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -3385,6 +3385,7 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
 {
struct vcpu_vmx *vmx = to_vmx(vcpu);
u32 ar;
+   unsigned long rip;
 
if (vmx->rmode.vm86_active && seg != VCPU_SREG_LDTR) {
*var = vmx->rmode.segs[seg];
@@ -3408,6 +3409,9 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,
var->db = (ar >> 14) & 1;
var->g = (ar >> 15) & 1;
var->unusable = (ar >> 16) & 1;
+   rip = kvm_rip_read(vcpu);
+   if ((rip == 0xc101611c || rip == 0xc101611a) && seg == VCPU_SREG_FS)
+   printk("base=%p limit=%p selector=%x ar=%x\n", var->base, 
var->limit, var->selector, ar);
 }
 
 static u64 vmx_get_segment_base(struct kvm_vcpu *vcpu, int seg)
--
Gleb.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 24.06.2013 13:47, Gleb Natapov wrote:
> On Mon, Jun 24, 2013 at 01:43:26PM +0200, Stefan Pietsch wrote:
>> On 23.06.2013 19:36, Gleb Natapov wrote:
>>> On Sun, Jun 23, 2013 at 06:51:30PM +0200, Stefan Pietsch wrote:
 On 23.06.2013 09:51, Gleb Natapov wrote:
> On Thu, Jun 20, 2013 at 07:01:49PM +0200, Stefan Pietsch wrote:
>>> Can you provide the output of 25391454e73e3156202264eb3c473825afe4bc94
>>> and emulate_invalid_guest_state=0. Also run "x/20i $pc-20" in qemu
>>> monitor after the hang.
>>
>>
>> 25391454e73e3156202264eb3c473825afe4bc94
>>  emulate_invalid_guest_state=0
>>
> Very interesting. Looks like somewhere during TPR access FS
> register gets corrupted. Can you remove /usr/share/kvm/kvmvapic.bin
> and try again? This will disable some code paths during TPR access and
> will narrow down the issue.


 Doing this, qemu complains
 "Could not open option rom 'kvmvapic.bin': No such file or directory",
 but the virtual machine boots successful with
 emulate_invalid_guest_state=0 and emulate_invalid_guest_state=1.

>>> Hmm, I think we ate close. Can you try with upstream qemu?
>>>
 kvmvapic.bin comes with Debian package "seabios 1.7.2-3".
>>
>> I already tried this with the Debian package qemu-kvm 1.5.0+dfsg-4.
> And it didn't work? Mind trying some debug kernel patches? I suspect
> your CPU does something no CPU I have do, so I want to verify it.


As soon as I remove "kvmvapic.bin" the virtual machine boots with
qemu-kvm 1.5.0. I just verified this with Linux kernel 3.10.0-rc5.
"emulate_invalid_guest_state=0" or "emulate_invalid_guest_state=1" make
no difference.

Please send your patches.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Mon, Jun 24, 2013 at 01:43:26PM +0200, Stefan Pietsch wrote:
> On 23.06.2013 19:36, Gleb Natapov wrote:
> > On Sun, Jun 23, 2013 at 06:51:30PM +0200, Stefan Pietsch wrote:
> >> On 23.06.2013 09:51, Gleb Natapov wrote:
> >>> On Thu, Jun 20, 2013 at 07:01:49PM +0200, Stefan Pietsch wrote:
> > Can you provide the output of 25391454e73e3156202264eb3c473825afe4bc94
> > and emulate_invalid_guest_state=0. Also run "x/20i $pc-20" in qemu
> > monitor after the hang.
> 
> 
>  25391454e73e3156202264eb3c473825afe4bc94
>   emulate_invalid_guest_state=0
> 
> >>> Very interesting. Looks like somewhere during TPR access FS
> >>> register gets corrupted. Can you remove /usr/share/kvm/kvmvapic.bin
> >>> and try again? This will disable some code paths during TPR access and
> >>> will narrow down the issue.
> >>
> >>
> >> Doing this, qemu complains
> >> "Could not open option rom 'kvmvapic.bin': No such file or directory",
> >> but the virtual machine boots successful with
> >> emulate_invalid_guest_state=0 and emulate_invalid_guest_state=1.
> >>
> > Hmm, I think we ate close. Can you try with upstream qemu?
> > 
> >> kvmvapic.bin comes with Debian package "seabios 1.7.2-3".
> 
> I already tried this with the Debian package qemu-kvm 1.5.0+dfsg-4.
And it didn't work? Mind trying some debug kernel patches? I suspect
your CPU does something no CPU I have do, so I want to verify it.

--
Gleb.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 23.06.2013 19:36, Gleb Natapov wrote:
> On Sun, Jun 23, 2013 at 06:51:30PM +0200, Stefan Pietsch wrote:
>> On 23.06.2013 09:51, Gleb Natapov wrote:
>>> On Thu, Jun 20, 2013 at 07:01:49PM +0200, Stefan Pietsch wrote:
> Can you provide the output of 25391454e73e3156202264eb3c473825afe4bc94
> and emulate_invalid_guest_state=0. Also run "x/20i $pc-20" in qemu
> monitor after the hang.


 25391454e73e3156202264eb3c473825afe4bc94
  emulate_invalid_guest_state=0

>>> Very interesting. Looks like somewhere during TPR access FS
>>> register gets corrupted. Can you remove /usr/share/kvm/kvmvapic.bin
>>> and try again? This will disable some code paths during TPR access and
>>> will narrow down the issue.
>>
>>
>> Doing this, qemu complains
>> "Could not open option rom 'kvmvapic.bin': No such file or directory",
>> but the virtual machine boots successful with
>> emulate_invalid_guest_state=0 and emulate_invalid_guest_state=1.
>>
> Hmm, I think we ate close. Can you try with upstream qemu?
> 
>> kvmvapic.bin comes with Debian package "seabios 1.7.2-3".

I already tried this with the Debian package qemu-kvm 1.5.0+dfsg-4.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Sun, Jun 23, 2013 at 06:51:30PM +0200, Stefan Pietsch wrote:
> On 23.06.2013 09:51, Gleb Natapov wrote:
> > On Thu, Jun 20, 2013 at 07:01:49PM +0200, Stefan Pietsch wrote:
> >>> Can you provide the output of 25391454e73e3156202264eb3c473825afe4bc94
> >>> and emulate_invalid_guest_state=0. Also run "x/20i $pc-20" in qemu
> >>> monitor after the hang.
> >>
> >>
> >> 25391454e73e3156202264eb3c473825afe4bc94
> >>  emulate_invalid_guest_state=0
> >>
> > Very interesting. Looks like somewhere during TPR access FS
> > register gets corrupted. Can you remove /usr/share/kvm/kvmvapic.bin
> > and try again? This will disable some code paths during TPR access and
> > will narrow down the issue.
> 
> 
> Doing this, qemu complains
> "Could not open option rom 'kvmvapic.bin': No such file or directory",
> but the virtual machine boots successful with
> emulate_invalid_guest_state=0 and emulate_invalid_guest_state=1.
> 
Hmm, I think we ate close. Can you try with upstream qemu?

> kvmvapic.bin comes with Debian package "seabios 1.7.2-3".

--
Gleb.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 23.06.2013 09:51, Gleb Natapov wrote:
> On Thu, Jun 20, 2013 at 07:01:49PM +0200, Stefan Pietsch wrote:
>>> Can you provide the output of 25391454e73e3156202264eb3c473825afe4bc94
>>> and emulate_invalid_guest_state=0. Also run "x/20i $pc-20" in qemu
>>> monitor after the hang.
>>
>>
>> 25391454e73e3156202264eb3c473825afe4bc94
>>  emulate_invalid_guest_state=0
>>
> Very interesting. Looks like somewhere during TPR access FS
> register gets corrupted. Can you remove /usr/share/kvm/kvmvapic.bin
> and try again? This will disable some code paths during TPR access and
> will narrow down the issue.


Doing this, qemu complains
"Could not open option rom 'kvmvapic.bin': No such file or directory",
but the virtual machine boots successful with
emulate_invalid_guest_state=0 and emulate_invalid_guest_state=1.

kvmvapic.bin comes with Debian package "seabios 1.7.2-3".

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Thu, Jun 20, 2013 at 07:01:49PM +0200, Stefan Pietsch wrote:
> > Can you provide the output of 25391454e73e3156202264eb3c473825afe4bc94
> > and emulate_invalid_guest_state=0. Also run "x/20i $pc-20" in qemu
> > monitor after the hang.
> 
> 
> 25391454e73e3156202264eb3c473825afe4bc94
>  emulate_invalid_guest_state=0
> 
Very interesting. Looks like somewhere during TPR access FS
register gets corrupted. Can you remove /usr/share/kvm/kvmvapic.bin
and try again? This will disable some code paths during TPR access and
will narrow down the issue.

--
Gleb.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 19.06.2013 16:38, Gleb Natapov wrote:
> On Wed, Jun 19, 2013 at 04:12:04PM +0200, Stefan Pietsch wrote:
>> On 19.06.2013 15:41, Gleb Natapov wrote:
>>> On Sun, Jun 16, 2013 at 02:25:04AM +0200, Stefan Pietsch wrote:

 first bad commit: [25391454e73e3156202264eb3c473825afe4bc94]
 KVM: VMX: don't clobber segment AR of unusable segments.


 25391454e73e3156202264eb3c473825afe4bc94
 emulate_invalid_guest_state=0 -> hangs and shows "KVM: entry failed"
 emulate_invalid_guest_state=1 -> hangs

>>> With emulate_invalid_guest_state=1 the commit does nothing. Can you
>>> double check that 218e763f458c44f30041c1b48b4371e130fd4317 works for you
>>> with emulate_invalid_guest_state=1?
>>
>>
>> 218e763f458c44f30041c1b48b4371e130fd4317
>> emulate_invalid_guest_state=0 -> works
>> emulate_invalid_guest_state=1 -> hangs
>>
>> 25391454e73e3156202264eb3c473825afe4bc94 broke
>> emulate_invalid_guest_state=0.
> Can you provide the output of 25391454e73e3156202264eb3c473825afe4bc94
> and emulate_invalid_guest_state=0. Also run "x/20i $pc-20" in qemu
> monitor after the hang.


25391454e73e3156202264eb3c473825afe4bc94
 emulate_invalid_guest_state=0

(qemu) info registers
EAX= EBX=0001 ECX=f000 EDX=f000
ESI=00195e93 EDI= EBP=de84c000 ESP=de84df64
EIP=c101611c EFL=00010246 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =007b   00c0f300 DPL=3 DS   [-WA]
CS =0060   00c09b00 DPL=0 CS32 [-RA]
SS =0068   00c09300 DPL=0 DS   [-WA]
DS =007b   00c0f300 DPL=3 DS   [-WA]
FS =  ffff 00f0ff00 DPL=3 CS64 [CRA]
GS =00e0 c1438b40 0018 00409100 DPL=0 DS   [--A]
LDT=  ffff 00f0ff00 DPL=3 CS64 [CRA]
TR =0080 c1400f00 206b 8b00 DPL=0 TSS32-busy
GDT= c13f6000 00ff
IDT= c13f5000 07ff
CR0=8005003b CR2= CR3=014bc000 CR4=0690
DR0= DR1= DR2=0007
DR3=
DR6=0ff0 DR7=0400
EFER=
FCW=037f FSW= [ST=0] FTW=00 MXCSR=1f80
FPR0=f44d002c6000 400d FPR1=80847fe7 400e
FPR2=fa007fa24000 400e FPR3=80e88055f000 400e
FPR4=ea61009c4000 400d FPR5=ea62009c4000 400c
FPR6=800bf600 4015 FPR7= 
XMM00=
XMM01=
XMM02=
XMM03=
XMM04=
XMM05=
XMM06=
XMM07=

(qemu) x/20i $pc-20
0xc1016108:  (bad)
0xc1016109:  decl   0x158bc310(%ecx)
0xc101610f:  les-0x7b723ec0(%ebx),%eax
0xc1016115:  adc%al,(%eax)
0xc1016117:  sar$0xff,%bh
0xc101611a:  mov(%eax),%eax
0xc101611c:  ret
0xc101611d:  jmp0xc1016121
0xc101611f:  pause
0xc1016121:  mov(%eax),%edx
0xc1016123:  test   %edx,%edx
0xc1016125:  je 0xc101611f
0xc1016127:  ret
0xc1016128:  cmp$0xfe,%eax
0xc101612d:  setle  %al
0xc1016130:  movzbl %al,%eax
0xc1016133:  ret
0xc1016134:  mov0xc1407b64,%edx
0xc101613a:  mov$0x20,%eax
0xc101613f:  call   *0x9c(%edx)


last 20 lines of the trace:
 qemu-system-x86-4042  [000]   295.592694: vcpu_match_mmio:  gva
0xb0d0 gpa 0xfee000d0 Read GVA
 qemu-system-x86-4042  [000]   295.592694: kvm_apic:
apic_read APIC_LDR = 0x100
 qemu-system-x86-4042  [000]   295.592695: kvm_mmio: mmio
read len 4 gpa 0xfee000d0 val 0x100
 qemu-system-x86-4042  [000]   295.592696: kvm_entry:vcpu 0
 qemu-system-x86-4042  [000]   295.592699: kvm_exit: reason
EXCEPTION_NMI rip 0xc101611a info b080 8b0e
 qemu-system-x86-4042  [000]   295.592700: kvm_page_fault:   address
b080 error_code 9
 qemu-system-x86-4042  [000]   295.592701: kvm_emulate_insn:
0:c101611a:8b 00 (prot32)
 qemu-system-x86-4042  [000]   295.592702: vcpu_match_mmio:  gva
0xb080 gpa 0xfee00080 Read GVA
 qemu-system-x86-4042  [000]   295.592703: kvm_apic:
apic_read APIC_TASKPRI = 0x0
 qemu-system-x86-4042  [000]   295.592703: kvm_mmio: mmio
read len 4 gpa 0xfee00080 val 0x0
 qemu-system-x86-4042  [000]   295.592704: kvm_userspace_exit:   reason
KVM_EXIT_TPR_ACCESS (12)
 qemu-system-x86-4042  [000]   295.592805: kvm_entry:vcpu 0
 qemu-system-x86-4042  [000]   295.592808: kvm_exit: reason
 rip 0xc101611c info 0 8b0e
 qemu-system-x86-4042  [000]   295.592809: kvm_userspace_exit:   reason
KVM_EXIT_FAIL_ENTRY (9)
 qemu-system-x86-4040  [001]   295.594993: kvm_set_irq:  gsi 0
level 1 source 0
 qemu-system-x86-4040  [001]   295.594995: kvm_pic_set_irq:  chip 0
pin 0 (edge)
 qemu-system-x86-4040  [001]   295.594997: kvm_ioapic_set_irq:   pin 2
dst 0 vec=0 

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Wed, Jun 19, 2013 at 04:12:04PM +0200, Stefan Pietsch wrote:
> On 19.06.2013 15:41, Gleb Natapov wrote:
> > On Sun, Jun 16, 2013 at 02:25:04AM +0200, Stefan Pietsch wrote:
> >>
> >> first bad commit: [25391454e73e3156202264eb3c473825afe4bc94]
> >> KVM: VMX: don't clobber segment AR of unusable segments.
> >>
> >>
> >> 25391454e73e3156202264eb3c473825afe4bc94
> >> emulate_invalid_guest_state=0 -> hangs and shows "KVM: entry failed"
> >> emulate_invalid_guest_state=1 -> hangs
> >>
> > With emulate_invalid_guest_state=1 the commit does nothing. Can you
> > double check that 218e763f458c44f30041c1b48b4371e130fd4317 works for you
> > with emulate_invalid_guest_state=1?
> 
> 
> 218e763f458c44f30041c1b48b4371e130fd4317
> emulate_invalid_guest_state=0 -> works
> emulate_invalid_guest_state=1 -> hangs
> 
> 25391454e73e3156202264eb3c473825afe4bc94 broke
> emulate_invalid_guest_state=0.
Can you provide the output of 25391454e73e3156202264eb3c473825afe4bc94
and emulate_invalid_guest_state=0. Also run "x/20i $pc-20" in qemu
monitor after the hang.

--
Gleb.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 19.06.2013 15:41, Gleb Natapov wrote:
> On Sun, Jun 16, 2013 at 02:25:04AM +0200, Stefan Pietsch wrote:
>>
>> first bad commit: [25391454e73e3156202264eb3c473825afe4bc94]
>> KVM: VMX: don't clobber segment AR of unusable segments.
>>
>>
>> 25391454e73e3156202264eb3c473825afe4bc94
>> emulate_invalid_guest_state=0 -> hangs and shows "KVM: entry failed"
>> emulate_invalid_guest_state=1 -> hangs
>>
> With emulate_invalid_guest_state=1 the commit does nothing. Can you
> double check that 218e763f458c44f30041c1b48b4371e130fd4317 works for you
> with emulate_invalid_guest_state=1?


218e763f458c44f30041c1b48b4371e130fd4317
emulate_invalid_guest_state=0 -> works
emulate_invalid_guest_state=1 -> hangs

25391454e73e3156202264eb3c473825afe4bc94 broke
emulate_invalid_guest_state=0.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Sun, Jun 16, 2013 at 02:25:04AM +0200, Stefan Pietsch wrote:
> On 13.06.2013 16:59, Stefan Pietsch wrote:
> > On 13.06.2013 15:42, Paolo Bonzini wrote:
> >> Il 13/06/2013 07:57, Stefan Pietsch ha scritto:
> >>> git bisect tells me:
> >>> 79fd50c67f91136add9726fb7719b57a66c6f763 is the first bad commit
> >>
> >> This is an s390 commit, so the bisect somehow went wrong.  Can you
> >> confirm that 3.7 works and 3.8 doesn't?
> > 
> > Confirmed. Something went wrong.
> > I replayed the bisect log and now I have
> > 
> > git bisect bad 9626357371b519f2b955fef399647181034a77fe
> > git bisect good ef4e359d9b9e2dc022f79840fd207796b524a893
> > git bisect good b5c78e04dd061b776978dad61dd85357081147b0
> > git bisect good 9e2d59ad580d590134285f361a0e80f0e98c0207
> > git bisect bad 69086a78bdc973ec0b722be790b146e84ba8a8c4
> > git bisect good 9ecf9b085a0926e07c78c08a07296bbfd1c37d07
> > git bisect bad 21fbd5809ad126b949206d78e0a0e07ec872ea11
> > git bisect bad 79fd50c67f91136add9726fb7719b57a66c6f763
> > git bisect bad aa11e3a8a6d9f92c3fe4b91a9aca5d8c23d55d4d
> > git bisect good 66cdd0ceaf65a18996f561b770eedde1d123b019
> > git bisect bad d99e415275dd3f757b75981adad8645cdc26da45
> > 
> > So please wait for my results.
> 
> 
> Bisecting leads to
> 
> git bisect bad 378a8b099fc207ddcb91b19a8c1457667e0af398
> git bisect good 007a3b547512d69f67ceb9641796d64552bd337e
> git bisect good 1f3141e80b149e7215313dff29e9a0c47811b1d1
> git bisect good 286da4156dc65c8a054580fdd96b7709132dce8d
> git bisect bad 25391454e73e3156202264eb3c473825afe4bc94
> git bisect good 218e763f458c44f30041c1b48b4371e130fd4317
> 
> 
> first bad commit: [25391454e73e3156202264eb3c473825afe4bc94]
> KVM: VMX: don't clobber segment AR of unusable segments.
> 
> 
> 25391454e73e3156202264eb3c473825afe4bc94
> emulate_invalid_guest_state=0 -> hangs and shows "KVM: entry failed"
> emulate_invalid_guest_state=1 -> hangs
>
With emulate_invalid_guest_state=1 the commit does nothing. Can you
double check that 218e763f458c44f30041c1b48b4371e130fd4317 works for you
with emulate_invalid_guest_state=1?
 
--
Gleb.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 17.06.2013 18:07, Paolo Bonzini wrote:
> Il 16/06/2013 02:25, Stefan Pietsch ha scritto:
>> Bisecting leads to
>>
>> git bisect bad 378a8b099fc207ddcb91b19a8c1457667e0af398
>> git bisect good 007a3b547512d69f67ceb9641796d64552bd337e
>> git bisect good 1f3141e80b149e7215313dff29e9a0c47811b1d1
>> git bisect good 286da4156dc65c8a054580fdd96b7709132dce8d
>> git bisect bad 25391454e73e3156202264eb3c473825afe4bc94
>> git bisect good 218e763f458c44f30041c1b48b4371e130fd4317
>>
>>
>> first bad commit: [25391454e73e3156202264eb3c473825afe4bc94]
>> KVM: VMX: don't clobber segment AR of unusable segments.
>>
>> 25391454e73e3156202264eb3c473825afe4bc94
>> emulate_invalid_guest_state=0 -> hangs and shows "KVM: entry failed"
>> emulate_invalid_guest_state=1 -> hangs
>>
>> Please note, I had to compile some revisions with
>> 3f0c3d0bb2bcc4b88b22452a7cf0073ee9a0f1e6 applied, caused by
>> 9ae9febae9500a0a6f5ce29ee4b8d942b5332529.
> 
> Can you please execute "info registers" and "x/10i $pc" from the QEMU
> monitor at the time of the hang, and include the output?  Using
> "-monitor stdio" or the new GTK+ interface can help.
> 
> Also, can you run under tracing (for information on how to do this, see
> http://www.linux-kvm.org/page/Tracing) and include the bottom of the log?

Tested with 25391454e73e3156202264eb3c473825afe4bc94
 emulate_invalid_guest_state=1


(qemu) info registers
EAX=00010286 EBX= ECX=c12c527c EDX=
ESI=00010286 EDI=c14c4744 EBP=c10161f5 ESP=de84df10
EIP=c1014a8d EFL=00010286 [--S--P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =007b   00c0f300 DPL=3 DS   [-WA]
CS =0060   00c09b00 DPL=0 CS32 [-RA]
SS =0068   00c09300 DPL=0 DS   [-WA]
DS =007b   00c0f300 DPL=3 DS   [-WA]
FS =  ffff 00f0ff00 DPL=3 CS64 [CRA]
GS =00e0 c1438b40 0018 00409100 DPL=0 DS   [--A]
LDT=  ffff 00f0ff00 DPL=3 CS64 [CRA]
TR =0080 c1400f00 206b 8b00 DPL=0 TSS32-busy
GDT= c13f6000 00ff
IDT= c13f5000 07ff
CR0=8005003b CR2= CR3=014bc000 CR4=0690
DR0= DR1= DR2=0007
DR3=
DR6=0ff0 DR7=0400
EFER=
FCW=037f FSW= [ST=0] FTW=00 MXCSR=1f80
FPR0=f44d002c6000 400d FPR1=80847fe7 400e
FPR2=fa007fa24000 400e FPR3=80e88055f000 400e
FPR4=ea61009c4000 400d FPR5=ea62009c4000 400c
FPR6=800bf600 4015 FPR7= 
XMM00=
XMM01=
XMM02=
XMM03=
XMM04=
XMM05=
XMM06=
XMM07=

(qemu) x/10i $pc
0xc1014a8d:  lea0x0(%esi),%esi
0xc1014a91:  ret
0xc1014a92:  cli
0xc1014a93:  nop
0xc1014a94:  lea0x0(%esi),%esi
0xc1014a98:  ret
0xc1014a99:  push   %eax
0xc1014a9a:  call   0xc1014a84
0xc1014a9f:  mov%eax,(%esp)
0xc1014aa2:  call   0xc1014a92


last 20 lines of the trace:
 qemu-system-x86-3575  [000]   542.279800: kvm_entry:vcpu 0
 qemu-system-x86-3575  [000]   542.279802: kvm_inj_virq: irq 48
 qemu-system-x86-3575  [000]   542.279802: kvm_entry:vcpu 0
 qemu-system-x86-3575  [000]   542.279803: kvm_inj_virq: irq 48
 qemu-system-x86-3575  [000]   542.279804: kvm_entry:vcpu 0
 qemu-system-x86-3575  [000]   542.279805: kvm_inj_virq: irq 48
 qemu-system-x86-3575  [000]   542.279806: kvm_entry:vcpu 0
 qemu-system-x86-3575  [000]   542.279807: kvm_inj_virq: irq 48
 qemu-system-x86-3575  [000]   542.279808: kvm_entry:vcpu 0
 qemu-system-x86-3575  [000]   542.279809: kvm_inj_virq: irq 48
 qemu-system-x86-3575  [000]   542.279810: kvm_entry:vcpu 0
 qemu-system-x86-3575  [000]   542.279811: kvm_inj_virq: irq 48
 qemu-system-x86-3575  [000]   542.279812: kvm_entry:vcpu 0
 qemu-system-x86-3573  [001]   542.280010: kvm_set_irq:  gsi 0
level 1 source 0
 qemu-system-x86-3573  [001]   542.280013: kvm_pic_set_irq:  chip 0
pin 0 (edge|masked)
 qemu-system-x86-3573  [001]   542.280015: kvm_apic_accept_irq:  apicid
0 vec 48 (LowPrio|edge) (coalesced)
 qemu-system-x86-3573  [001]   542.280015: kvm_ioapic_set_irq:   pin 2
dst 1 vec=48 (LowPrio|logical|edge) (coalesced)
 qemu-system-x86-3573  [001]   542.280016: kvm_set_irq:  gsi 0
level 0 source 0
 qemu-system-x86-3573  [001]   542.280017: kvm_pic_set_irq:  chip 0
pin 0 (edge|masked)
 qemu-system-x86-3573  [001]   542.280017: kvm_ioapic_set_irq:   pin 2
dst 1 vec=48 (LowPrio|logical|edge)

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majord

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Paolo Bonzini]

Il 16/06/2013 02:25, Stefan Pietsch ha scritto:
> Bisecting leads to
> 
> git bisect bad 378a8b099fc207ddcb91b19a8c1457667e0af398
> git bisect good 007a3b547512d69f67ceb9641796d64552bd337e
> git bisect good 1f3141e80b149e7215313dff29e9a0c47811b1d1
> git bisect good 286da4156dc65c8a054580fdd96b7709132dce8d
> git bisect bad 25391454e73e3156202264eb3c473825afe4bc94
> git bisect good 218e763f458c44f30041c1b48b4371e130fd4317
> 
> 
> first bad commit: [25391454e73e3156202264eb3c473825afe4bc94]
> KVM: VMX: don't clobber segment AR of unusable segments.
> 
> 25391454e73e3156202264eb3c473825afe4bc94
> emulate_invalid_guest_state=0 -> hangs and shows "KVM: entry failed"
> emulate_invalid_guest_state=1 -> hangs
> 
> Please note, I had to compile some revisions with
> 3f0c3d0bb2bcc4b88b22452a7cf0073ee9a0f1e6 applied, caused by
> 9ae9febae9500a0a6f5ce29ee4b8d942b5332529.

Can you please execute "info registers" and "x/10i $pc" from the QEMU
monitor at the time of the hang, and include the output?  Using
"-monitor stdio" or the new GTK+ interface can help.

Also, can you run under tracing (for information on how to do this, see
http://www.linux-kvm.org/page/Tracing) and include the bottom of the log?

Thanks,

Paolo

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 13.06.2013 16:59, Stefan Pietsch wrote:
> On 13.06.2013 15:42, Paolo Bonzini wrote:
>> Il 13/06/2013 07:57, Stefan Pietsch ha scritto:
>>> git bisect tells me:
>>> 79fd50c67f91136add9726fb7719b57a66c6f763 is the first bad commit
>>
>> This is an s390 commit, so the bisect somehow went wrong.  Can you
>> confirm that 3.7 works and 3.8 doesn't?
> 
> Confirmed. Something went wrong.
> I replayed the bisect log and now I have
> 
> git bisect bad 9626357371b519f2b955fef399647181034a77fe
> git bisect good ef4e359d9b9e2dc022f79840fd207796b524a893
> git bisect good b5c78e04dd061b776978dad61dd85357081147b0
> git bisect good 9e2d59ad580d590134285f361a0e80f0e98c0207
> git bisect bad 69086a78bdc973ec0b722be790b146e84ba8a8c4
> git bisect good 9ecf9b085a0926e07c78c08a07296bbfd1c37d07
> git bisect bad 21fbd5809ad126b949206d78e0a0e07ec872ea11
> git bisect bad 79fd50c67f91136add9726fb7719b57a66c6f763
> git bisect bad aa11e3a8a6d9f92c3fe4b91a9aca5d8c23d55d4d
> git bisect good 66cdd0ceaf65a18996f561b770eedde1d123b019
> git bisect bad d99e415275dd3f757b75981adad8645cdc26da45
> 
> So please wait for my results.


Bisecting leads to

git bisect bad 378a8b099fc207ddcb91b19a8c1457667e0af398
git bisect good 007a3b547512d69f67ceb9641796d64552bd337e
git bisect good 1f3141e80b149e7215313dff29e9a0c47811b1d1
git bisect good 286da4156dc65c8a054580fdd96b7709132dce8d
git bisect bad 25391454e73e3156202264eb3c473825afe4bc94
git bisect good 218e763f458c44f30041c1b48b4371e130fd4317


first bad commit: [25391454e73e3156202264eb3c473825afe4bc94]
KVM: VMX: don't clobber segment AR of unusable segments.


25391454e73e3156202264eb3c473825afe4bc94
emulate_invalid_guest_state=0 -> hangs and shows "KVM: entry failed"
emulate_invalid_guest_state=1 -> hangs

Please note, I had to compile some revisions with
3f0c3d0bb2bcc4b88b22452a7cf0073ee9a0f1e6 applied, caused by
9ae9febae9500a0a6f5ce29ee4b8d942b5332529.

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 13.06.2013 15:42, Paolo Bonzini wrote:
> Il 13/06/2013 07:57, Stefan Pietsch ha scritto:
>> git bisect tells me:
>> 79fd50c67f91136add9726fb7719b57a66c6f763 is the first bad commit
> 
> This is an s390 commit, so the bisect somehow went wrong.  Can you
> confirm that 3.7 works and 3.8 doesn't?

Confirmed. Something went wrong.
I replayed the bisect log and now I have

git bisect bad 9626357371b519f2b955fef399647181034a77fe
git bisect good ef4e359d9b9e2dc022f79840fd207796b524a893
git bisect good b5c78e04dd061b776978dad61dd85357081147b0
git bisect good 9e2d59ad580d590134285f361a0e80f0e98c0207
git bisect bad 69086a78bdc973ec0b722be790b146e84ba8a8c4
git bisect good 9ecf9b085a0926e07c78c08a07296bbfd1c37d07
git bisect bad 21fbd5809ad126b949206d78e0a0e07ec872ea11
git bisect bad 79fd50c67f91136add9726fb7719b57a66c6f763
git bisect bad aa11e3a8a6d9f92c3fe4b91a9aca5d8c23d55d4d
git bisect good 66cdd0ceaf65a18996f561b770eedde1d123b019
git bisect bad d99e415275dd3f757b75981adad8645cdc26da45

So please wait for my results.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Paolo Bonzini]

Il 13/06/2013 09:42, Paolo Bonzini ha scritto:
> Il 13/06/2013 07:57, Stefan Pietsch ha scritto:
>> git bisect tells me:
>> 79fd50c67f91136add9726fb7719b57a66c6f763 is the first bad commit
> 
> This is an s390 commit, so the bisect somehow went wrong.  Can you
> confirm that 3.7 works and 3.8 doesn't?

Sorry, 3.8 works and 3.9 doesn't
(66cdd0ceaf65a18996f561b770eedde1d123b019 was the 3.8 merge window
update, and your bisect shows it as good).

Can you double-check this with both normal "modprobe kvm_intel" and
"modprobe kvm_intel emulate_invalid_guest_state=0"?

Paolo

> Please check these pairs:
> 
> 9e2d59a and 89f883372fa60f604d136924baf3e89ff1870e9e
> 39ab967 and 875b7679abbb232b584f2eec59fa6e45690dd6c4
> 10b3866 and ea4a0ce11160200410abbabd44ec9e75e93a95be
> 4ffd4eb and ccae663cd4f62890d862c660e5ed762eb9821c14
> 896ea17 and 66cdd0ceaf65a18996f561b770eedde1d123b019
> 
> Please tell us which pair introduced the failure.  Then:
> 
> - if you get a "bad and bad" pair, tell us and we'll figure out what's
> next :)
> 
> - if you get a "good and bad" pair, do a "git bisect" between the two
> commits in that pair.
> 
> Thanks!
> 
> Paolo
> 
>> This is my bisect log:
>>
>> git bisect start
>> git bisect bad 9626357371b519f2b955fef399647181034a77fe
>> git bisect good ef4e359d9b9e2dc022f79840fd207796b524a893
>> git bisect good b5c78e04dd061b776978dad61dd85357081147b0
>> git bisect good 9e2d59ad580d590134285f361a0e80f0e98c0207
>> git bisect bad 69086a78bdc973ec0b722be790b146e84ba8a8c4
>> git bisect good 9ecf9b085a0926e07c78c08a07296bbfd1c37d07
>> git bisect bad 21fbd5809ad126b949206d78e0a0e07ec872ea11
>> git bisect bad 79fd50c67f91136add9726fb7719b57a66c6f763
>> git bisect good 66cdd0ceaf65a18996f561b770eedde1d123b019
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>> the body of a message to majord...@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Paolo Bonzini]

Il 13/06/2013 07:57, Stefan Pietsch ha scritto:
> git bisect tells me:
> 79fd50c67f91136add9726fb7719b57a66c6f763 is the first bad commit

This is an s390 commit, so the bisect somehow went wrong.  Can you
confirm that 3.7 works and 3.8 doesn't?

Please check these pairs:

9e2d59a and 89f883372fa60f604d136924baf3e89ff1870e9e
39ab967 and 875b7679abbb232b584f2eec59fa6e45690dd6c4
10b3866 and ea4a0ce11160200410abbabd44ec9e75e93a95be
4ffd4eb and ccae663cd4f62890d862c660e5ed762eb9821c14
896ea17 and 66cdd0ceaf65a18996f561b770eedde1d123b019

Please tell us which pair introduced the failure.  Then:

- if you get a "bad and bad" pair, tell us and we'll figure out what's
next :)

- if you get a "good and bad" pair, do a "git bisect" between the two
commits in that pair.

Thanks!

Paolo

> This is my bisect log:
> 
> git bisect start
> git bisect bad 9626357371b519f2b955fef399647181034a77fe
> git bisect good ef4e359d9b9e2dc022f79840fd207796b524a893
> git bisect good b5c78e04dd061b776978dad61dd85357081147b0
> git bisect good 9e2d59ad580d590134285f361a0e80f0e98c0207
> git bisect bad 69086a78bdc973ec0b722be790b146e84ba8a8c4
> git bisect good 9ecf9b085a0926e07c78c08a07296bbfd1c37d07
> git bisect bad 21fbd5809ad126b949206d78e0a0e07ec872ea11
> git bisect bad 79fd50c67f91136add9726fb7719b57a66c6f763
> git bisect good 66cdd0ceaf65a18996f561b770eedde1d123b019
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 09.06.2013 11:43, Gleb Natapov wrote:
> On Thu, Jun 06, 2013 at 02:10:39PM +0200, Stefan Pietsch wrote:
>> On 06.06.2013 13:40, Gleb Natapov wrote:
>>> On Thu, Jun 06, 2013 at 01:35:13PM +0200, Stefan Pietsch wrote:
>>
 I had no success with the Debian kernel 3.10~rc4-1~exp1 (3.10-rc4-686-pae).

 The machine hangs after "Enabling APIC mode:  Flat.  Using 1 I/O APICs".
>>> OK, since it looks like it hangs during timer initialization can you try
>>> to disable kvmclock? Add -cpu qemu64,-kvmclock to your command line.
>>> Also can you provide the output of "cat /proc/cpuinfo" on your host? And
>>> complete serial output before hang.
>>
>>
>> command line:
>> qemu-system-i386 -machine accel=kvm -m 512 -cpu qemu64,-kvmclock -cdrom
>> grml32-full_2013.02.iso -serial file:ttyS0.log
>>
>>
>> ttyS0.log:
>> ##
>>
> 
> Nothing out of ordinary here. Since you can reproduce the hang and I
> cannot, can you try and bisect it? Also can trace kvm during the hang
> http://www.linux-kvm.org/page/Tracing? Start the trace as close to hang
> as possible and stop it as quick after it as possible too to make trace
> file smaller.


git bisect tells me:
79fd50c67f91136add9726fb7719b57a66c6f763 is the first bad commit


This is my bisect log:

git bisect start
git bisect bad 9626357371b519f2b955fef399647181034a77fe
git bisect good ef4e359d9b9e2dc022f79840fd207796b524a893
git bisect good b5c78e04dd061b776978dad61dd85357081147b0
git bisect good 9e2d59ad580d590134285f361a0e80f0e98c0207
git bisect bad 69086a78bdc973ec0b722be790b146e84ba8a8c4
git bisect good 9ecf9b085a0926e07c78c08a07296bbfd1c37d07
git bisect bad 21fbd5809ad126b949206d78e0a0e07ec872ea11
git bisect bad 79fd50c67f91136add9726fb7719b57a66c6f763
git bisect good 66cdd0ceaf65a18996f561b770eedde1d123b019

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Thu, Jun 06, 2013 at 02:10:39PM +0200, Stefan Pietsch wrote:
> On 06.06.2013 13:40, Gleb Natapov wrote:
> > On Thu, Jun 06, 2013 at 01:35:13PM +0200, Stefan Pietsch wrote:
> 
> >> I had no success with the Debian kernel 3.10~rc4-1~exp1 (3.10-rc4-686-pae).
> >>
> >> The machine hangs after "Enabling APIC mode:  Flat.  Using 1 I/O APICs".
> > OK, since it looks like it hangs during timer initialization can you try
> > to disable kvmclock? Add -cpu qemu64,-kvmclock to your command line.
> > Also can you provide the output of "cat /proc/cpuinfo" on your host? And
> > complete serial output before hang.
> 
> 
> command line:
> qemu-system-i386 -machine accel=kvm -m 512 -cpu qemu64,-kvmclock -cdrom
> grml32-full_2013.02.iso -serial file:ttyS0.log
> 
> 
> ttyS0.log:
> ##
> 

Nothing out of ordinary here. Since you can reproduce the hang and I
cannot, can you try and bisect it? Also can trace kvm during the hang
http://www.linux-kvm.org/page/Tracing? Start the trace as close to hang
as possible and stop it as quick after it as possible too to make trace
file smaller.

--
Gleb.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 06.06.2013 13:40, Gleb Natapov wrote:
> On Thu, Jun 06, 2013 at 01:35:13PM +0200, Stefan Pietsch wrote:

>> I had no success with the Debian kernel 3.10~rc4-1~exp1 (3.10-rc4-686-pae).
>>
>> The machine hangs after "Enabling APIC mode:  Flat.  Using 1 I/O APICs".
> OK, since it looks like it hangs during timer initialization can you try
> to disable kvmclock? Add -cpu qemu64,-kvmclock to your command line.
> Also can you provide the output of "cat /proc/cpuinfo" on your host? And
> complete serial output before hang.


command line:
qemu-system-i386 -machine accel=kvm -m 512 -cpu qemu64,-kvmclock -cdrom
grml32-full_2013.02.iso -serial file:ttyS0.log



/proc/cpuinfo:
##

processor   : 0
vendor_id   : GenuineIntel
cpu family  : 6
model   : 14
model name  : Intel(R) Core(TM) Duo CPU  L2400  @ 1.66GHz
stepping: 12
microcode   : 0x54
cpu MHz : 1000.000
cache size  : 2048 KB
physical id : 0
siblings: 2
core id : 0
cpu cores   : 2
apicid  : 0
initial apicid  : 0
fdiv_bug: no
f00f_bug: no
coma_bug: no
fpu : yes
fpu_exception   : yes
cpuid level : 10
wp  : yes
flags   : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx constant_tsc
arch_perfmon bts aperfmperf pni monitor vmx est tm2 xtpr pdcm dtherm
bogomips: 3325.02
clflush size: 64
cache_alignment : 64
address sizes   : 32 bits physical, 32 bits virtual
power management:

processor   : 1
vendor_id   : GenuineIntel
cpu family  : 6
model   : 14
model name  : Intel(R) Core(TM) Duo CPU  L2400  @ 1.66GHz
stepping: 12
microcode   : 0x54
cpu MHz : 1000.000
cache size  : 2048 KB
physical id : 0
siblings: 2
core id : 1
cpu cores   : 2
apicid  : 1
initial apicid  : 1
fdiv_bug: no
f00f_bug: no
coma_bug: no
fpu : yes
fpu_exception   : yes
cpuid level : 10
wp  : yes
flags   : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx constant_tsc
arch_perfmon bts aperfmperf pni monitor vmx est tm2 xtpr pdcm dtherm
bogomips: 3325.02
clflush size: 64
cache_alignment : 64
address sizes   : 32 bits physical, 32 bits virtual
power management:



ttyS0.log:
##

[0.00] Initializing cgroup subsys cpuset
[0.00] Initializing cgroup subsys cpu
[0.00] Linux version 3.7-1-grml-486 (t...@grml.org) (gcc version
4.7.2 (Debian 4.7.2-5) ) #1 Debian 3.7.9-1+grml.1
[0.00] e820: BIOS-provided physical RAM map:
[0.00] BIOS-e820: [mem 0x-0x0009fbff]
usable
[0.00] BIOS-e820: [mem 0x0009fc00-0x0009]
reserved
[0.00] BIOS-e820: [mem 0x000f-0x000f]
reserved
[0.00] BIOS-e820: [mem 0x0010-0x1fffdfff]
usable
[0.00] BIOS-e820: [mem 0x1fffe000-0x1fff]
reserved
[0.00] BIOS-e820: [mem 0xfeffc000-0xfeff]
reserved
[0.00] BIOS-e820: [mem 0xfffc-0x]
reserved
[0.00] Notice: NX (Execute Disable) protection cannot be
enabled: non-PAE kernel!
[0.00] SMBIOS 2.4 present.
[0.00] Hypervisor detected: KVM
[0.00] e820: last_pfn = 0x1fffe max_arch_pfn = 0x10
[0.00] PAT not supported by CPU.
[0.00] found SMP MP-table at [mem 0x000fdb00-0x000fdb0f] mapped
at [c00fdb00]
[0.00] init_memory_mapping: [mem 0x-0x1fffdfff]
[0.00] RAMDISK: [mem 0x1f33-0x1ffdbfff]
[0.00] ACPI: RSDP 000fd9a0 00014 (v00 BOCHS )
[0.00] ACPI: RSDT 1fffe4b0 00034 (v01 BOCHS  BXPCRSDT 0001
BXPC 0001)
[0.00] ACPI: FACP 1f80 00074 (v01 BOCHS  BXPCFACP 0001
BXPC 0001)
[0.00] ACPI: DSDT 1fffe4f0 011A9 (v01   BXPC   BXDSDT 0001
INTL 20100528)
[0.00] ACPI: FACS 1f40 00040
[0.00] ACPI: SSDT 1800 00735 (v01 BOCHS  BXPCSSDT 0001
BXPC 0001)
[0.00] ACPI: APIC 16e0 00078 (v01 BOCHS  BXPCAPIC 0001
BXPC 0001)
[0.00] ACPI: HPET 16a0 00038 (v01 BOCHS  BXPCHPET 0001
BXPC 0001)
[0.00] 0MB HIGHMEM available.
[0.00] 511MB LOWMEM available.
[0.00]   mapped low ram: 0 - 1fffe000
[0.00]   low ram: 0 - 1fffe000
[0.00] Zone ranges:
[0.00]   DMA  [mem 0x0001-0x00ff]
[0.00]   Normal   [mem 0x0100-0x1fffdfff]
[0.00]   HighMem  empty
[0.00] Movable zone start for each node
[0.00] Early memory node ranges
[0.00]   node   0: [mem 0x0001-0x0009efff]
[0.00]   node   0: [mem 0x0010-0x1fffdfff]
[0.00] Using APIC driver default
[0.00] ACPI: PM-Timer IO Port: 0xb008
[0.0

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Thu, Jun 06, 2013 at 01:35:13PM +0200, Stefan Pietsch wrote:
> On 06.06.2013 08:42, Gleb Natapov wrote:
> > On Wed, Jun 05, 2013 at 02:51:19PM +0200, Stefan Pietsch wrote:
> >> On 05.06.2013 14:10, Gleb Natapov wrote:
> >>> On Wed, Jun 05, 2013 at 01:57:25PM +0200, Stefan Pietsch wrote:
>  On 19.05.2013 14:32, Gleb Natapov wrote:
> > On Sun, May 19, 2013 at 02:00:31AM +0100, Ben Hutchings wrote:
> >> Dear KVM maintainers, it appears that there is a gap in x86 emulation,
> >> at least on a 32-bit host.  Stefan found this when running GRML, a live
> >> distribution which can be downloaded from:
> >> .  His original
> >> reported is at .
> >>
> > Can you verify with latest linux.git HEAD? It works for me there on
> > 64bit. There were a lot of problems fixed in this area in 3.9/3.10 time 
> > frame,
> > so it would be helpful if you'll test 32bit before I install one myself.
> 
> 
>  Kernel version 3.9.4-1 (linux-image-3.9-1-686-pae) made things worse.
> 
>  The virtual machine tries to boot the kernel, but stops after a few
>  seconds and the kern.log shows:
> >>> At what point does it stop?
> >>
> >>
> >> The machine stops at:
> >>
> >> Performance Events: Broken PMU hardware detected, using software events
> >> only.
> >> Failed to access perfctr msr (MSR c1 is 0)
> >> Enabling APIC mode:  Flat.  Using 1 I/O APICs
> > Timer initialization is what comes next.
> > 
> > I tried 32bit kernel compiled from kvm.git "next" (3.10.0-rc2+) branch and 
> > upstream
> > qemu and I cannot reproduce the problem. The guest boots fine.
> 
> 
> I had no success with the Debian kernel 3.10~rc4-1~exp1 (3.10-rc4-686-pae).
> 
> The machine hangs after "Enabling APIC mode:  Flat.  Using 1 I/O APICs".
OK, since it looks like it hangs during timer initialization can you try
to disable kvmclock? Add -cpu qemu64,-kvmclock to your command line.
Also can you provide the output of "cat /proc/cpuinfo" on your host? And
complete serial output before hang.

--
Gleb.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 06.06.2013 08:42, Gleb Natapov wrote:
> On Wed, Jun 05, 2013 at 02:51:19PM +0200, Stefan Pietsch wrote:
>> On 05.06.2013 14:10, Gleb Natapov wrote:
>>> On Wed, Jun 05, 2013 at 01:57:25PM +0200, Stefan Pietsch wrote:
 On 19.05.2013 14:32, Gleb Natapov wrote:
> On Sun, May 19, 2013 at 02:00:31AM +0100, Ben Hutchings wrote:
>> Dear KVM maintainers, it appears that there is a gap in x86 emulation,
>> at least on a 32-bit host.  Stefan found this when running GRML, a live
>> distribution which can be downloaded from:
>> .  His original
>> reported is at .
>>
> Can you verify with latest linux.git HEAD? It works for me there on
> 64bit. There were a lot of problems fixed in this area in 3.9/3.10 time 
> frame,
> so it would be helpful if you'll test 32bit before I install one myself.


 Kernel version 3.9.4-1 (linux-image-3.9-1-686-pae) made things worse.

 The virtual machine tries to boot the kernel, but stops after a few
 seconds and the kern.log shows:
>>> At what point does it stop?
>>
>>
>> The machine stops at:
>>
>> Performance Events: Broken PMU hardware detected, using software events
>> only.
>> Failed to access perfctr msr (MSR c1 is 0)
>> Enabling APIC mode:  Flat.  Using 1 I/O APICs
> Timer initialization is what comes next.
> 
> I tried 32bit kernel compiled from kvm.git "next" (3.10.0-rc2+) branch and 
> upstream
> qemu and I cannot reproduce the problem. The guest boots fine.


I had no success with the Debian kernel 3.10~rc4-1~exp1 (3.10-rc4-686-pae).

The machine hangs after "Enabling APIC mode:  Flat.  Using 1 I/O APICs".

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Thu, Jun 06, 2013 at 09:42:40AM +0300, Gleb Natapov wrote:
> On Wed, Jun 05, 2013 at 02:51:19PM +0200, Stefan Pietsch wrote:
> > On 05.06.2013 14:10, Gleb Natapov wrote:
> > > On Wed, Jun 05, 2013 at 01:57:25PM +0200, Stefan Pietsch wrote:
> > >> On 19.05.2013 14:32, Gleb Natapov wrote:
> > >>> On Sun, May 19, 2013 at 02:00:31AM +0100, Ben Hutchings wrote:
> >  Dear KVM maintainers, it appears that there is a gap in x86 emulation,
> >  at least on a 32-bit host.  Stefan found this when running GRML, a live
> >  distribution which can be downloaded from:
> >  .  His original
> >  reported is at .
> > 
> > >>> Can you verify with latest linux.git HEAD? It works for me there on
> > >>> 64bit. There were a lot of problems fixed in this area in 3.9/3.10 time 
> > >>> frame,
> > >>> so it would be helpful if you'll test 32bit before I install one myself.
> > >>
> > >>
> > >> Kernel version 3.9.4-1 (linux-image-3.9-1-686-pae) made things worse.
> > >>
> > >> The virtual machine tries to boot the kernel, but stops after a few
> > >> seconds and the kern.log shows:
> > > At what point does it stop?
> > 
> > 
> > The machine stops at:
> > 
> > Performance Events: Broken PMU hardware detected, using software events
> > only.
> > Failed to access perfctr msr (MSR c1 is 0)
> > Enabling APIC mode:  Flat.  Using 1 I/O APICs
> Timer initialization is what comes next.
> 
> I tried 32bit kernel compiled from kvm.git "next" (3.10.0-rc2+) branch and 
> upstream
> qemu and I cannot reproduce the problem. The guest boots fine.
> 
Actually the branch I tested is "master" not "next", but this should not
make a difference.

--
Gleb.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Wed, Jun 05, 2013 at 02:51:19PM +0200, Stefan Pietsch wrote:
> On 05.06.2013 14:10, Gleb Natapov wrote:
> > On Wed, Jun 05, 2013 at 01:57:25PM +0200, Stefan Pietsch wrote:
> >> On 19.05.2013 14:32, Gleb Natapov wrote:
> >>> On Sun, May 19, 2013 at 02:00:31AM +0100, Ben Hutchings wrote:
>  Dear KVM maintainers, it appears that there is a gap in x86 emulation,
>  at least on a 32-bit host.  Stefan found this when running GRML, a live
>  distribution which can be downloaded from:
>  .  His original
>  reported is at .
> 
> >>> Can you verify with latest linux.git HEAD? It works for me there on
> >>> 64bit. There were a lot of problems fixed in this area in 3.9/3.10 time 
> >>> frame,
> >>> so it would be helpful if you'll test 32bit before I install one myself.
> >>
> >>
> >> Kernel version 3.9.4-1 (linux-image-3.9-1-686-pae) made things worse.
> >>
> >> The virtual machine tries to boot the kernel, but stops after a few
> >> seconds and the kern.log shows:
> > At what point does it stop?
> 
> 
> The machine stops at:
> 
> Performance Events: Broken PMU hardware detected, using software events
> only.
> Failed to access perfctr msr (MSR c1 is 0)
> Enabling APIC mode:  Flat.  Using 1 I/O APICs
Timer initialization is what comes next.

I tried 32bit kernel compiled from kvm.git "next" (3.10.0-rc2+) branch and 
upstream
qemu and I cannot reproduce the problem. The guest boots fine.

--
Gleb.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 05.06.2013 14:10, Gleb Natapov wrote:
> On Wed, Jun 05, 2013 at 01:57:25PM +0200, Stefan Pietsch wrote:
>> On 19.05.2013 14:32, Gleb Natapov wrote:
>>> On Sun, May 19, 2013 at 02:00:31AM +0100, Ben Hutchings wrote:
 Dear KVM maintainers, it appears that there is a gap in x86 emulation,
 at least on a 32-bit host.  Stefan found this when running GRML, a live
 distribution which can be downloaded from:
 .  His original
 reported is at .

>>> Can you verify with latest linux.git HEAD? It works for me there on
>>> 64bit. There were a lot of problems fixed in this area in 3.9/3.10 time 
>>> frame,
>>> so it would be helpful if you'll test 32bit before I install one myself.
>>
>>
>> Kernel version 3.9.4-1 (linux-image-3.9-1-686-pae) made things worse.
>>
>> The virtual machine tries to boot the kernel, but stops after a few
>> seconds and the kern.log shows:
> At what point does it stop?


The machine stops at:

Performance Events: Broken PMU hardware detected, using software events
only.
Failed to access perfctr msr (MSR c1 is 0)
Enabling APIC mode:  Flat.  Using 1 I/O APICs

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Wed, Jun 05, 2013 at 01:57:25PM +0200, Stefan Pietsch wrote:
> On 19.05.2013 14:32, Gleb Natapov wrote:
> > On Sun, May 19, 2013 at 02:00:31AM +0100, Ben Hutchings wrote:
> >> Dear KVM maintainers, it appears that there is a gap in x86 emulation,
> >> at least on a 32-bit host.  Stefan found this when running GRML, a live
> >> distribution which can be downloaded from:
> >> .  His original
> >> reported is at .
> >>
> > Can you verify with latest linux.git HEAD? It works for me there on
> > 64bit. There were a lot of problems fixed in this area in 3.9/3.10 time 
> > frame,
> > so it would be helpful if you'll test 32bit before I install one myself.
> 
> 
> Kernel version 3.9.4-1 (linux-image-3.9-1-686-pae) made things worse.
> 
> The virtual machine tries to boot the kernel, but stops after a few
> seconds and the kern.log shows:
At what point does it stop?

> 
> kernel: [13851.000412] kvm [7482]: vcpu0 disabled perfctr wrmsr: 0xc1
> data 0x
> 
That's harmless.

> 
> virtual machine was started with:
> qemu-system-i386 -machine accel=kvm -m 512 -cdrom grml32-full_2013.02.iso
> 
> qemu-system-x86: 1.5.0+dfsg-3

--
Gleb.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 19.05.2013 14:32, Gleb Natapov wrote:
> On Sun, May 19, 2013 at 02:00:31AM +0100, Ben Hutchings wrote:
>> Dear KVM maintainers, it appears that there is a gap in x86 emulation,
>> at least on a 32-bit host.  Stefan found this when running GRML, a live
>> distribution which can be downloaded from:
>> .  His original
>> reported is at .
>>
> Can you verify with latest linux.git HEAD? It works for me there on
> 64bit. There were a lot of problems fixed in this area in 3.9/3.10 time frame,
> so it would be helpful if you'll test 32bit before I install one myself.


Kernel version 3.9.4-1 (linux-image-3.9-1-686-pae) made things worse.

The virtual machine tries to boot the kernel, but stops after a few
seconds and the kern.log shows:

kernel: [13851.000412] kvm [7482]: vcpu0 disabled perfctr wrmsr: 0xc1
data 0x


virtual machine was started with:
qemu-system-i386 -machine accel=kvm -m 512 -cdrom grml32-full_2013.02.iso

qemu-system-x86: 1.5.0+dfsg-3

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Ben Hutchings]

On Wed, May 29, 2013 at 05:05:55PM +0200, Stefan Pietsch wrote:
> On 19.05.2013 14:32, Gleb Natapov wrote:
> > On Sun, May 19, 2013 at 02:00:31AM +0100, Ben Hutchings wrote:
> >> Dear KVM maintainers, it appears that there is a gap in x86 emulation,
> >> at least on a 32-bit host.  Stefan found this when running GRML, a live
> >> distribution which can be downloaded from:
> >> .  His original
> >> reported is at .
> >>
> > Can you verify with latest linux.git HEAD? It works for me there on
> > 64bit. There were a lot of problems fixed in this area in 3.9/3.10 time 
> > frame,
> > so it would be helpful if you'll test 32bit before I install one myself.
> 
> 
> Ben,
> 
> can you provide a 3.9 series kernel package?

I will do soon, but you should be able to build your own:

# ...unpack upstream source...
$ cp /boot/config-$(uname -r) .config
$ yes "" | make oldconfig
$ make deb-pkg -j$(nproc)

Ben.

-- 
Ben Hutchings
We get into the habit of living before acquiring the habit of thinking.
  - Albert Camus
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Stefan Pietsch]

On 19.05.2013 14:32, Gleb Natapov wrote:
> On Sun, May 19, 2013 at 02:00:31AM +0100, Ben Hutchings wrote:
>> Dear KVM maintainers, it appears that there is a gap in x86 emulation,
>> at least on a 32-bit host.  Stefan found this when running GRML, a live
>> distribution which can be downloaded from:
>> .  His original
>> reported is at .
>>
> Can you verify with latest linux.git HEAD? It works for me there on
> 64bit. There were a lot of problems fixed in this area in 3.9/3.10 time frame,
> so it would be helpful if you'll test 32bit before I install one myself.


Ben,

can you provide a 3.9 series kernel package?

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Gleb Natapov]

On Sun, May 19, 2013 at 02:00:31AM +0100, Ben Hutchings wrote:
> Dear KVM maintainers, it appears that there is a gap in x86 emulation,
> at least on a 32-bit host.  Stefan found this when running GRML, a live
> distribution which can be downloaded from:
> .  His original
> reported is at .
> 
Can you verify with latest linux.git HEAD? It works for me there on
64bit. There were a lot of problems fixed in this area in 3.9/3.10 time frame,
so it would be helpful if you'll test 32bit before I install one myself.

> On Thu, 2013-05-16 at 13:26 +0200, Stefan Pietsch wrote:
> > On 09.05.2013 20:56, Stefan Pietsch wrote:
> > > On 09.05.2013 03:08, Ben Hutchings wrote:
> > > 
> > >> Please could you test some of the intermediate versions at
> > >>  to find the first upstream
> > >> version where this was broken.
> > > 
> > > The first version which does not work is 3.6.4-1~experimental.1.
> > > 3.5.5-1~experimental.1 works.
> > 
> > 
> > I was able to start KVM under kernel version 3.8.12-1 after loading the
> > "kvm_intel" module with the option "emulate_invalid_guest_state=0".
> 
> And one of the many changes between 3.5 and 3.6 was to change the
> default value of that parameter from 0 to 1.  So we don't know when the
> the bug in emulation was introduced (or if it was always there).
> 
> Ben.
> 
> -- 
> Ben Hutchings
> The generation of random numbers is too important to be left to chance.
> - Robert Coveyou



--
Gleb.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#707257: linux-image-3.8-1-686-pae: KVM crashes with "entry failed, hardware error 0x80000021"

[Ben Hutchings]

Dear KVM maintainers, it appears that there is a gap in x86 emulation,
at least on a 32-bit host.  Stefan found this when running GRML, a live
distribution which can be downloaded from:
.  His original
reported is at .

On Thu, 2013-05-16 at 13:26 +0200, Stefan Pietsch wrote:
> On 09.05.2013 20:56, Stefan Pietsch wrote:
> > On 09.05.2013 03:08, Ben Hutchings wrote:
> > 
> >> Please could you test some of the intermediate versions at
> >>  to find the first upstream
> >> version where this was broken.
> > 
> > The first version which does not work is 3.6.4-1~experimental.1.
> > 3.5.5-1~experimental.1 works.
> 
> 
> I was able to start KVM under kernel version 3.8.12-1 after loading the
> "kvm_intel" module with the option "emulate_invalid_guest_state=0".

And one of the many changes between 3.5 and 3.6 was to change the
default value of that parameter from 0 to 1.  So we don't know when the
the bug in emulation was introduced (or if it was always there).

Ben.

-- 
Ben Hutchings
The generation of random numbers is too important to be left to chance.
- Robert Coveyou


signature.asc
Description: This is a digitally signed message part