Re: Problem with KVM guest switching to x86 long mode
Avi Kivity wrote:
The instruction at 0x28 is enabling paging, next insn fetch faults, so
the paging structures must be incorrect.
Questions:
- what is the u64 at cr3? (call it pte4)
- what is the u64 at (pte4 ~0xfff)? (call it pte3)
- what is the u64 at (pte3 ~0xfff)? (pte2)
- what is the u64 at ((pte2 ~0xfff) + 2048)? (pte1)
Note if bit 7 of pte2 is set, then pte1 is unneeded.
Sorry for the delay. Here you go:
penb...@tiger:~/vm$ ./kvm bzImage
KVM exit reason: 8 (KVM_EXIT_SHUTDOWN)
Registers:
rip: 001000ed rsp: 005d54b8 flags: 00010046
rax: 8001 rbx: 01f2c000 rcx: c080
rdx: rsi: 00013670 rdi: 02408000
rbp: 0010 r8: r9:
r10: r11: r12:
r13: r14: r15:
cr0: 8011 cr2: 001000ed cr3: 02402000
cr4: 0020 cr8:
Segment registers:
register selector base limit type p dpl db s l g avl
cs0010 0b1 0 1 1 0 1 0
ss0018 031 0 1 1 0 1 0
ds0018 031 0 1 1 0 1 0
es0018 031 0 1 1 0 1 0
fs0018 031 0 1 1 0 1 0
gs0018 031 0 1 1 0 1 0
tr0020 1000 0067 0b1 0 0 0 0 0 0
ldt 000 0 0 0 0 0 0
gdt 005ca458 0030
idt
[ efer: 0500 apic base: nmi: disabled ]
Interrupt bitmap:
Code: 08 49 75 f3 8d 83 00 60 4d 00 0f 22 d8 b9 80 00 00 c0 0f 32 0f ba
e8 08 0f 30 6a 10 8d 85 00 02 00 00 50 b8 01 00 00 80 0f 22 c0 cb f4
eb fd 9c 6a 00 9d 9c 58 89 c3 35 00 00 20 00 50 9d 9c 58
Stack:
0x005d54b8: 00 02 10 00 10 00 00 00
0x005d54c0: 00 00 00 00 00 00 00 00
0x005d54c8: 00 00 00 00 00 00 00 00
0x005d54d0: 00 00 00 00 00 00 00 00
Page Tables:
pte4: 02403007 pte3: 02404007 pte2: 0183
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with KVM guest switching to x86 long mode
On 04/11/2010 09:30 AM, Pekka Enberg wrote: Avi Kivity wrote: The instruction at 0x28 is enabling paging, next insn fetch faults, so the paging structures must be incorrect. Questions: - what is the u64 at cr3? (call it pte4) - what is the u64 at (pte4 ~0xfff)? (call it pte3) - what is the u64 at (pte3 ~0xfff)? (pte2) - what is the u64 at ((pte2 ~0xfff) + 2048)? (pte1) Note if bit 7 of pte2 is set, then pte1 is unneeded. Sorry for the delay. Here you go: Page Tables: pte4: 02403007 pte3: 02404007 pte2: 0183 These are all correct. The only thing I can think of, is that MAXPHYADDR is small value. And indeed, if I run it on an ept capable machine (which does the check in hardware, not software), I get IO error: OUT port=cf8, size=4, count=1 KVM exit reason: 2 (KVM_EXIT_IO) Registers: rip: 813a0d8f rsp: 8167bdf8 flags: 0086 rax: 8000c300 rbx: c000 rcx: rdx: 0cf8 rsi: 0018 rdi: rbp: 8167be08 r8: r9: r10: 0006 r11: bbceeb20 r12: r13: r14: r15: cr0: 80050033 cr2: cr3: 01001000 cr4: 00a0 cr8: So the guest is in long mode, happily trying to access pci config space. MAXPHYADDR comes from cpuid 8008.eax[0:7]. Typical values are 36-40 (number of physical address bits supported by the processor). What value does your guest see? -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with KVM guest switching to x86 long mode
Avi Kivity wrote: Sorry for the delay. Here you go: Page Tables: pte4: 02403007 pte3: 02404007 pte2: 0183 These are all correct. The only thing I can think of, is that MAXPHYADDR is small value. And indeed, if I run it on an ept capable machine (which does the check in hardware, not software), I get IO error: OUT port=cf8, size=4, count=1 KVM exit reason: 2 (KVM_EXIT_IO) Registers: rip: 813a0d8f rsp: 8167bdf8 flags: 0086 rax: 8000c300 rbx: c000 rcx: rdx: 0cf8 rsi: 0018 rdi: rbp: 8167be08 r8: r9: r10: 0006 r11: bbceeb20 r12: r13: r14: r15: cr0: 80050033 cr2: cr3: 01001000 cr4: 00a0 cr8: So the guest is in long mode, happily trying to access pci config space. MAXPHYADDR comes from cpuid 8008.eax[0:7]. Typical values are 36-40 (number of physical address bits supported by the processor). What value does your guest see? Ah, nice catch! The host cpuid doesn't seem to support 8008 at so I didn't set it up in the guest either. I now added dummy emulation for it and the triple fault is fixed. Thanks Avi! Pekka -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with KVM guest switching to x86 long mode
On 04/11/2010 12:48 PM, Pekka Enberg wrote:
So the guest is in long mode, happily trying to access pci config space.
MAXPHYADDR comes from cpuid 8008.eax[0:7]. Typical values are
36-40 (number of physical address bits supported by the processor).
What value does your guest see?
Ah, nice catch! The host cpuid doesn't seem to support 8008 at so
I didn't set it up in the guest either. I now added dummy emulation
for it and the triple fault is fixed.
It should work without 8008 set up - failure should happen only if
it is setup incorrectly:
int cpuid_maxphyaddr(struct kvm_vcpu *vcpu)
{
struct kvm_cpuid_entry2 *best;
best = kvm_find_cpuid_entry(vcpu, 0x8008, 0);
if (best)
return best-eax 0xff;
return 36;
}
--
error compiling committee.c: too many arguments to function
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with KVM guest switching to x86 long mode
Avi Kivity wrote:
On 04/11/2010 12:48 PM, Pekka Enberg wrote:
So the guest is in long mode, happily trying to access pci config space.
MAXPHYADDR comes from cpuid 8008.eax[0:7]. Typical values are
36-40 (number of physical address bits supported by the processor).
What value does your guest see?
Ah, nice catch! The host cpuid doesn't seem to support 8008 at so
I didn't set it up in the guest either. I now added dummy emulation
for it and the triple fault is fixed.
It should work without 8008 set up - failure should happen only if
it is setup incorrectly:
int cpuid_maxphyaddr(struct kvm_vcpu *vcpu)
{
struct kvm_cpuid_entry2 *best;
best = kvm_find_cpuid_entry(vcpu, 0x8008, 0);
if (best)
return best-eax 0xff;
return 36;
}
Well, like I said, if I don't add an entry for it in struct kvm_cpuid2
that's passed to KVM_SET_CPUID2 ioctl, I see a triple fault on Core 2
Duo CPU that doesn't seem to have native 0x8008.
Pekka
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with KVM guest switching to x86 long mode
On 04/11/2010 01:02 PM, Pekka Enberg wrote:
It should work without 8008 set up - failure should happen only
if it is setup incorrectly:
int cpuid_maxphyaddr(struct kvm_vcpu *vcpu)
{
struct kvm_cpuid_entry2 *best;
best = kvm_find_cpuid_entry(vcpu, 0x8008, 0);
if (best)
return best-eax 0xff;
return 36;
}
Well, like I said, if I don't add an entry for it in struct kvm_cpuid2
that's passed to KVM_SET_CPUID2 ioctl, I see a triple fault on Core 2
Duo CPU that doesn't seem to have native 0x8008.
Do you have a function 8, though? Looks like a bug in kvm may confuse
the two.
--
error compiling committee.c: too many arguments to function
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with KVM guest switching to x86 long mode
Avi Kivity wrote:
On 04/11/2010 01:02 PM, Pekka Enberg wrote:
It should work without 8008 set up - failure should happen only
if it is setup incorrectly:
int cpuid_maxphyaddr(struct kvm_vcpu *vcpu)
{
struct kvm_cpuid_entry2 *best;
best = kvm_find_cpuid_entry(vcpu, 0x8008, 0);
if (best)
return best-eax 0xff;
return 36;
}
Well, like I said, if I don't add an entry for it in struct kvm_cpuid2
that's passed to KVM_SET_CPUID2 ioctl, I see a triple fault on Core 2
Duo CPU that doesn't seem to have native 0x8008.
Do you have a function 8, though? Looks like a bug in kvm may confuse
the two.
Yeah, the host has function 8. I'm more than happy to test patches to
fix the problem.
Pekka
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with KVM guest switching to x86 long mode
On 04/11/2010 02:52 PM, Pekka Enberg wrote: Do you have a function 8, though? Looks like a bug in kvm may confuse the two. Yeah, the host has function 8. I'm more than happy to test patches to fix the problem. Coming up after a quick git blame to see if I can see how the bug was introduced. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with KVM guest switching to x86 long mode
On 04/11/2010 03:02 PM, Avi Kivity wrote: On 04/11/2010 02:52 PM, Pekka Enberg wrote: Do you have a function 8, though? Looks like a bug in kvm may confuse the two. Yeah, the host has function 8. I'm more than happy to test patches to fix the problem. Coming up after a quick git blame to see if I can see how the bug was introduced. Ugh, I shouldn't have. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with KVM guest switching to x86 long mode
On 04/08/2010 09:26 PM, Pekka Enberg wrote: Hi! I am working on a light-weight KVM userspace launcher for Linux and am bit stuck with a guest Linux kernel restarting when it tries to enter long mode. The register dump looks like this: penb...@tiger:~/vm$ ./kvm bzImage KVM exit reason: 8 (KVM_EXIT_SHUTDOWN) Registers: rip: 001000ed rsp: 005d54b8 flags: 00010046 rax: 8001 rbx: 01f2c000 rcx: c080 rdx: rsi: 00013670 rdi: 02408000 rbp: 0010 r8: r9: r10: r11: r12: r13: r14: r15: cr0: 8011 cr2: 001000ed cr3: 02402000 cr4: 0020 cr8: Segment registers: register selector base limit type p dpl db s l g avl cs0010 0b1 0 1 1 0 1 0 ss0018 031 0 1 1 0 1 0 ds0018 031 0 1 1 0 1 0 es0018 031 0 1 1 0 1 0 fs0018 031 0 1 1 0 1 0 gs0018 031 0 1 1 0 1 0 tr0020 1000 0067 0b1 0 0 0 0 0 0 ldt 000 0 0 0 0 0 0 These all look reasonable. Please add a gdtr dump and an idtr dump. 2b:* cb lret-- trapping instruction Post the two u32s at ss:rsp - ss:rsp+8. That will tell us where the guest is trying to return. Actually, from the dump: 1a:6a 10pushq $0x10 1c:8d 85 00 02 00 00lea0x200(%rbp),%eax 22:50 push %rax it looks like you're returning to segment 0x10, this should be the word at ss:rsp+4. So if you dump the 2 u32s at gdtr.base+0x10..gdtr.base+0x18 we'll see if there's anything wrong with the segment descriptor. -- I have a truly marvellous patch that fixes the bug which this signature is too narrow to contain. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with KVM guest switching to x86 long mode
Avi Kivity wrote:
These all look reasonable. Please add a gdtr dump and an idtr dump.
Done.
2b:*cb lret-- trapping instruction
Post the two u32s at ss:rsp - ss:rsp+8. That will tell us where the
guest is trying to return. Actually, from the dump:
1a:6a 10pushq $0x10
1c:8d 85 00 02 00 00lea0x200(%rbp),%eax
22:50 push %rax
it looks like you're returning to segment 0x10, this should be the word
at ss:rsp+4. So if you dump the 2 u32s at
gdtr.base+0x10..gdtr.base+0x18 we'll see if there's anything wrong with
the segment descriptor.
Here you go:
penb...@tiger:~/vm$ ./kvm bzImage
KVM exit reason: 8 (KVM_EXIT_SHUTDOWN)
Registers:
rip: 001000ed rsp: 005d54b8 flags: 00010046
rax: 8001 rbx: 01f2c000 rcx: c080
rdx: rsi: 00013670 rdi: 02408000
rbp: 0010 r8: r9:
r10: r11: r12:
r13: r14: r15:
cr0: 8011 cr2: 001000ed cr3: 02402000
cr4: 0020 cr8:
Segment registers:
register selector base limit type p dpl db s l g avl
cs0010 0b1 0 1 1 0 1 0
ss0018 031 0 1 1 0 1 0
ds0018 031 0 1 1 0 1 0
es0018 031 0 1 1 0 1 0
fs0018 031 0 1 1 0 1 0
gs0018 031 0 1 1 0 1 0
tr0020 1000 0067 0b1 0 0 0 0 0 0
ldt 000 0 0 0 0 0 0
gdt 005ca458 0030
idt
[ efer: 0500 apic base: nmi: disabled ]
Interrupt bitmap:
Code: 08 49 75 f3 8d 83 00 60 4d 00 0f 22 d8 b9 80 00 00 c0 0f 32 0f ba
e8 08 0f 30 6a 10 8d 85 00 02 00 00 50 b8 01 00 00 80 0f 22 c0 cb f4
eb fd 9c 6a 00 9d 9c 58 89 c3 35 00 00 20 00 50 9d 9c 58
Stack:
0x005d54b8: 00 02 10 00 10 00 00 00 -- return value
0x005d54c0: 00 00 00 00 00 00 00 00
0x005d54c8: 00 00 00 00 00 00 00 00
0x005d54d0: 00 00 00 00 00 00 00 00
GDT:
0x005ca458: 30 00 58 a4 5c 00 00 00
0x005ca460: 00 00 00 00 00 00 00 00
0x005ca468: ff ff 00 00 00 9a af 00 -- gtr.base + 0x10
0x005ca470: ff ff 00 00 00 92 cf 00
0x005ca478: 00 00 00 00 00 89 80 00
0x005ca480: 00 00 00 00 00 00 00 00
Pekka
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem with KVM guest switching to x86 long mode
On 04/08/2010 09:59 PM, Pekka Enberg wrote: 2b:*cb lret-- trapping instruction Post the two u32s at ss:rsp - ss:rsp+8. That will tell us where the guest is trying to return. Actually, from the dump: 1a:6a 10pushq $0x10 1c:8d 85 00 02 00 00lea0x200(%rbp),%eax 22:50 push %rax it looks like you're returning to segment 0x10, this should be the word at ss:rsp+4. So if you dump the 2 u32s at gdtr.base+0x10..gdtr.base+0x18 we'll see if there's anything wrong with the segment descriptor. Here you go: I was asking for the wrong things. penb...@tiger:~/vm$ ./kvm bzImage KVM exit reason: 8 (KVM_EXIT_SHUTDOWN) Registers: rip: 001000ed rsp: 005d54b8 flags: 00010046 rax: 8001 rbx: 01f2c000 rcx: c080 rdx: rsi: 00013670 rdi: 02408000 rbp: 0010 r8: r9: r10: r11: r12: r13: r14: r15: cr0: 8011 cr2: 001000ed cr3: 02402000 cr2 points at rip. So it isn't lret not executing correctly, it's the cpu not able to fetch lret at all. The code again: 23: b8 01 00 00 80 mov$0x8001,%eax 28: 0f 22 c0mov%rax,%cr0 2b:* cb lret-- trapping instruction The instruction at 0x28 is enabling paging, next insn fetch faults, so the paging structures must be incorrect. Questions: - what is the u64 at cr3? (call it pte4) - what is the u64 at (pte4 ~0xfff)? (call it pte3) - what is the u64 at (pte3 ~0xfff)? (pte2) - what is the u64 at ((pte2 ~0xfff) + 2048)? (pte1) Note if bit 7 of pte2 is set, then pte1 is unneeded. -- I have a truly marvellous patch that fixes the bug which this signature is too narrow to contain. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
