Re: kernel 3.9.x kvm hangs after seabios
On Wed, May 08, 2013 at 04:52:52PM +0200, Tomas Papan wrote: > patch is working :) > > Thank you very much Gleb. > Thank you for your patience. Curious but it was. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel 3.9.x kvm hangs after seabios
patch is working :) Thank you very much Gleb. Regards Tomas -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel 3.9.x kvm hangs after seabios
-- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel 3.9.x kvm hangs after seabios
On Wed, May 08, 2013 at 03:50:47PM +0200, Tomas Papan wrote:
> Ok, the cpu stays at 0% when it hangs, there is only one 100% cpu peak
> which happens when the vm starts ( I think this is quite normal).
>
> However I run following command, and I stop it right when it hangs:
> anakin trace2 # virsh start vm-jack; pid=`virsh qemu-monitor-command
> vm-jack --hmp info cpus | grep '\*' | awk '{print $5}' | cut -d\=
> -f2`; trace-cmd record -P $pid -p function
>
> if anyone is interested it produces a 1.6 GB file (the compressed
> version can be found here: http://papan.sk/share/trace2.dat.tar.gz
> (150 MB))
>
Thanks! Can you test the patch below:
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 6667042..0af1807 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -5197,6 +5197,12 @@ static int handle_invalid_guest_state(struct kvm_vcpu
*vcpu)
return 0;
}
+ if (vcpu->arch.halt_request) {
+ vcpu->arch.halt_request = 0;
+ ret = kvm_emulate_halt(vcpu);
+ goto out;
+ }
+
if (signal_pending(current))
goto out;
if (need_resched())
--
Gleb.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel 3.9.x kvm hangs after seabios
Ok, the cpu stays at 0% when it hangs, there is only one 100% cpu peak
which happens when the vm starts ( I think this is quite normal).
However I run following command, and I stop it right when it hangs:
anakin trace2 # virsh start vm-jack; pid=`virsh qemu-monitor-command
vm-jack --hmp info cpus | grep '\*' | awk '{print $5}' | cut -d\=
-f2`; trace-cmd record -P $pid -p function
if anyone is interested it produces a 1.6 GB file (the compressed
version can be found here: http://papan.sk/share/trace2.dat.tar.gz
(150 MB))
Tomas
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel 3.9.x kvm hangs after seabios
On Wed, May 08, 2013 at 02:51:48PM +0200, Tomas Papan wrote: > Hi, > > No nothing, I check all logs (even syslog) > Yeah, since status of the vm is "running" you are not suppose to see there anything. > 1) virsh # qemu-monitor-command vm-jack --hmp info status > VM status: running > > 2) morpheus@anakin ~ $ ps aux | grep vm-jack > qemu 3822 0.5 0.1 8952256 23600 ? Sl 13:59 0:08 > /usr/bin/qemu-system-x86_64 -machine accel=kvm -name vm-jack -S > -machine pc-0.14,accel=kvm,usb=off -cpu > Nehalem,+rdtscp,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme > -m 8192 -smp 4,sockets=4,cores=1,threads=1 -uuid > 03196c23-24ba-d398-a000-582b0e88b0e7 -no-user-config -nodefaults > -chardev > socket,id=charmonitor,path=/var/lib/libvirt/qemu/vm-jack.monitor,server,nowait > -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc > -no-shutdown -boot order=c,menu=on -device > piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive > file=/var/lib/libvirt/images/jack.img,if=none,id=drive-virtio-disk0,format=raw > -device > virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0 > -drive > file=/var/lib/libvirt/images/kernel.img,if=none,id=drive-virtio-disk1,format=raw > -device > virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk1,id=virtio-disk1 > -drive if=none,id=drive-ide0-1-0,readonly=on,format=raw -device > ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev > tap,fd=19,id=hostnet0 -device > e1000,netdev=hostnet0,id=net0,mac=52:54:00:21:1c:e0,bus=pci.0,addr=0x3 > -chardev pty,id=charserial0 -device > isa-serial,chardev=charserial0,id=serial0 -vnc 127.0.0.1:0 -k en-us > -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 > > 3) it took some time, I didn't have debug_fs, then tracing... but the > file is stored here (15 MB) http://papan.sk/share/trace.dat.tar.gz > Very interesting. In the middle of the run vcpu decides that it does not want to run any more. How much cpu time qemu takes when it happens? If it is 100% can you do the following: 1. run qemu-monitor-command vm-jack --hmp info cpus 2. note thread id for cpu #0 3. run "trace-cmd record -P $pid -p function" where $pid is the pid thread id that you've found in 2. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel 3.9.x kvm hangs after seabios
Sorry, I didn't write that well, I checked that log too... nothing is there... anakin qemu # cat vm-jack.log 2013-05-08 13:02:52.358+: starting up LC_ALL=C PATH=/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/opt/bin HOME=/root USER=root QEMU_AUDIO_DRV=none /usr/bin/qemu-kvm -name vm-jack -S -machine pc-0.14,accel=kvm,usb=off -cpu Nehalem,+rdtscp,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -m 8192 -smp 4,sockets=4,cores=1,threads=1 -uuid 03196c23-24ba-d398-a000-582b0e88b0e7 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/vm-jack.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot order=c,menu=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/jack.img,if=none,id=drive-virtio-disk0,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0 -drive file=/var/lib/libvirt/images/kernel.img,if=none,id=drive-virtio-disk1,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk1,id=virtio-disk1 -drive if=none,id=drive-ide0-1-0,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=19,id=hostnet0 -device e1000,netdev=hostnet0,id=net0,mac=52:54:00:21:1c:e0,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -vnc 127.0.0.1:0 -k en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 char device redirected to /dev/pts/3 (label charserial0) -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel 3.9.x kvm hangs after seabios
On Wed, May 08, 2013 at 02:08:55PM +0200, Tomas Papan wrote: > Hi, > > I found this in the libvirt (but those messages are same in 3.8.x) > anakin libvirt # cat libvirtd.log > 2013-05-08 11:59:29.645+: 3750: info : libvirt version: 1.0.5 > 2013-05-08 11:59:29.645+: 3750: error : udevGetDMIData:1548 : > Failed to get udev device for syspath '/sys/devices/virtual/dmi/id' or > '/sys/class/dmi/id' > 2013-05-08 11:59:29.680+: 3750: warning : > ebiptablesDriverInitCLITools:4225 : Could not find 'ebtables' > executable You need to look at /var/log/libvirt/qemu/$GUESTNAME.log for QEMU related messages. The libvirtd.log file only has the libvirt related messages. Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel 3.9.x kvm hangs after seabios
Hi, No nothing, I check all logs (even syslog) 1) virsh # qemu-monitor-command vm-jack --hmp info status VM status: running 2) morpheus@anakin ~ $ ps aux | grep vm-jack qemu 3822 0.5 0.1 8952256 23600 ? Sl 13:59 0:08 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name vm-jack -S -machine pc-0.14,accel=kvm,usb=off -cpu Nehalem,+rdtscp,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -m 8192 -smp 4,sockets=4,cores=1,threads=1 -uuid 03196c23-24ba-d398-a000-582b0e88b0e7 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/vm-jack.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot order=c,menu=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/jack.img,if=none,id=drive-virtio-disk0,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0 -drive file=/var/lib/libvirt/images/kernel.img,if=none,id=drive-virtio-disk1,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk1,id=virtio-disk1 -drive if=none,id=drive-ide0-1-0,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=19,id=hostnet0 -device e1000,netdev=hostnet0,id=net0,mac=52:54:00:21:1c:e0,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -vnc 127.0.0.1:0 -k en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 3) it took some time, I didn't have debug_fs, then tracing... but the file is stored here (15 MB) http://papan.sk/share/trace.dat.tar.gz Regards Tomas -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel 3.9.x kvm hangs after seabios
On Wed, May 08, 2013 at 02:08:55PM +0200, Tomas Papan wrote: > Hi, > > I found this in the libvirt (but those messages are same in 3.8.x) > anakin libvirt # cat libvirtd.log > 2013-05-08 11:59:29.645+: 3750: info : libvirt version: 1.0.5 > 2013-05-08 11:59:29.645+: 3750: error : udevGetDMIData:1548 : > Failed to get udev device for syspath '/sys/devices/virtual/dmi/id' or > '/sys/class/dmi/id' > 2013-05-08 11:59:29.680+: 3750: warning : > ebiptablesDriverInitCLITools:4225 : Could not find 'ebtables' > executable > Nothing about KVM internal error? Couple of more things please: 1. Output of qemu-monitor-command vm-jack --hmp info status. 2. command line. 3. trace http://www.linux-kvm.org/page/Tracing -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel 3.9.x kvm hangs after seabios
Hi, I found this in the libvirt (but those messages are same in 3.8.x) anakin libvirt # cat libvirtd.log 2013-05-08 11:59:29.645+: 3750: info : libvirt version: 1.0.5 2013-05-08 11:59:29.645+: 3750: error : udevGetDMIData:1548 : Failed to get udev device for syspath '/sys/devices/virtual/dmi/id' or '/sys/class/dmi/id' 2013-05-08 11:59:29.680+: 3750: warning : ebiptablesDriverInitCLITools:4225 : Could not find 'ebtables' executable virsh # qemu-monitor-command vm-jack --hmp info registers EAX=0002 EBX=64a1 ECX=6e08 EDX=000fc5ab ESI=c5b8 EDI=6eec EBP=dffd83e0 ESP=6df8 EIP=c46b EFL=00010002 [---] CPL=0 II=0 A20=1 SMM=0 HLT=1 ES = 9300 CS =f000 000f 9b00 SS = 9300 DS = 9300 FS = 9300 GS = 9300 LDT= 8200 TR = 8b00 GDT= 000fd3a8 0037 IDT= 000fd3e6 CR0=0010 CR2= CR3= CR4= DR0= DR1= DR2= DR3= DR6=0ff0 DR7=0400 EFER= FCW=037f FSW= [ST=0] FTW=00 MXCSR=1f80 FPR0= FPR1= FPR2= FPR3= FPR4= FPR5= FPR6= FPR7= XMM00= XMM01= XMM02= XMM03= XMM04= XMM05= XMM06= XMM07= bios.bin can be found here http://papan.sk/share/bios.bin I should mentioned that I'm using gentoo and libvirt 1.0.5. I'm sorry if gmail interface breaks output. Regrads Tomas On Wed, May 8, 2013 at 1:55 PM, Gleb Natapov wrote: > On Wed, May 08, 2013 at 11:22:01AM +, Tomas Papan wrote: >> I have the same issue, with 3.9.1 (3.9.0 too) it hangs right after seabios... >> (no problem in 3.8.11) >> >> qemu-1.4.1 >> seabios-1.7.2.1 >> > Is there anything interesting in libvirt logfile? > > Also please send the output of "qemu-monitor-command vm-jack --hmp info > registers" > > And, just in case, can you send me your bios.bin image. Mine work. > >> after setting emulate_invalid_guest_state=0 everything works just fine. >> >> virsh # qemu-monitor-command vm-jack --hmp x/8i \$pc >> 0x000fc46b: lgdtw %cs:-0x2c60 >> 0x000fc471: mov%cr0,%eax >> 0x000fc474: or $0x1,%eax >> 0x000fc478: mov%eax,%cr0 >> 0x000fc47b: ljmpl $0x8,$0xfc483 >> 0x000fc483: mov$0x10,%ax >> 0x000fc486: add%al,(%bx,%si) >> 0x000fc488: mov%ax,%ds >> >> >> virsh # qemu-monitor-command vm-jack --hmp x/64b \$pc >> 0x000fc46b: lgdtw %cs:-0x2c60 >> 0x000fc471: mov%cr0,%eax >> 0x000fc474: or $0x1,%eax >> 0x000fc478: mov%eax,%cr0 >> 0x000fc47b: ljmpl $0x8,$0xfc483 >> 0x000fc483: mov$0x10,%ax >> 0x000fc486: add%al,(%bx,%si) >> 0x000fc488: mov%ax,%ds >> 0x000fc48a: mov%ax,%es >> 0x000fc48c: mov%ax,%ss >> 0x000fc48e: mov%ax,%fs >> 0x000fc490: mov%ax,%gs >> 0x000fc492: mov%cx,%ax >> 0x000fc494: jmp*%dx >> 0x000fc496: mov%ax,%cx >> 0x000fc498: mov$0x20,%ax >> 0x000fc49b: add%al,(%bx,%si) >> 0x000fc49d: mov%ax,%ds >> 0x000fc49f: mov%ax,%es >> 0x000fc4a1: mov%ax,%ss >> 0x000fc4a3: mov%ax,%fs >> 0x000fc4a5: mov%ax,%gs >> 0x000fc4a7: ljmpl $0xc189,$0x18c4c4 >> 0x000fc4af: mov$0x30,%ax >> 0x000fc4b2: add%al,(%bx,%si) >> 0x000fc4b4: mov%ax,%ds >> 0x000fc4b6: mov%ax,%es >> 0x000fc4b8: mov%ax,%ss >> 0x000fc4ba: mov%ax,%fs >> 0x000fc4bc: mov%ax,%gs >> 0x000fc4be: ljmpl $0x200f,$0x28c4c4 >> 0x000fc4c6: shlb $0xe0,-0x7d(%bp) >> 0x000fc4ca: decb (%bx) >> 0x000fc4cc: and%al,%al >> 0x000fc4ce: ljmp $0xf000,$0xc4d3 >> 0x000fc4d3: lidtw %cs:-0x2c18 >> 0x000fc4d9: xor%ax,%ax >> 0x000fc4db: mov%ax,%fs >> 0x000fc4dd: mov%ax,%gs >> 0x000fc4df: mov%ax,%es >> 0x000fc4e1: mov%ax,%ds >> 0x000fc4e3: mov%ax,%ss >> 0x000fc4e5: mov%ecx,%eax >> 0x000fc4e8: jmpl *%edx >> 0x000fc4eb: push %ebp >> 0x000fc4ed: push %eax >> 0x000fc4ef: pushl %es >> 0x000fc4f1: push %cs >> 0x000fc4f2: push $0xc536 >> 0x000fc4f5: addr32 pushw %es:0
Re: kernel 3.9.x kvm hangs after seabios
On Wed, May 08, 2013 at 11:22:01AM +, Tomas Papan wrote: > I have the same issue, with 3.9.1 (3.9.0 too) it hangs right after seabios... > (no problem in 3.8.11) > > qemu-1.4.1 > seabios-1.7.2.1 > Is there anything interesting in libvirt logfile? Also please send the output of "qemu-monitor-command vm-jack --hmp info registers" And, just in case, can you send me your bios.bin image. Mine work. > after setting emulate_invalid_guest_state=0 everything works just fine. > > virsh # qemu-monitor-command vm-jack --hmp x/8i \$pc > 0x000fc46b: lgdtw %cs:-0x2c60 > 0x000fc471: mov%cr0,%eax > 0x000fc474: or $0x1,%eax > 0x000fc478: mov%eax,%cr0 > 0x000fc47b: ljmpl $0x8,$0xfc483 > 0x000fc483: mov$0x10,%ax > 0x000fc486: add%al,(%bx,%si) > 0x000fc488: mov%ax,%ds > > > virsh # qemu-monitor-command vm-jack --hmp x/64b \$pc > 0x000fc46b: lgdtw %cs:-0x2c60 > 0x000fc471: mov%cr0,%eax > 0x000fc474: or $0x1,%eax > 0x000fc478: mov%eax,%cr0 > 0x000fc47b: ljmpl $0x8,$0xfc483 > 0x000fc483: mov$0x10,%ax > 0x000fc486: add%al,(%bx,%si) > 0x000fc488: mov%ax,%ds > 0x000fc48a: mov%ax,%es > 0x000fc48c: mov%ax,%ss > 0x000fc48e: mov%ax,%fs > 0x000fc490: mov%ax,%gs > 0x000fc492: mov%cx,%ax > 0x000fc494: jmp*%dx > 0x000fc496: mov%ax,%cx > 0x000fc498: mov$0x20,%ax > 0x000fc49b: add%al,(%bx,%si) > 0x000fc49d: mov%ax,%ds > 0x000fc49f: mov%ax,%es > 0x000fc4a1: mov%ax,%ss > 0x000fc4a3: mov%ax,%fs > 0x000fc4a5: mov%ax,%gs > 0x000fc4a7: ljmpl $0xc189,$0x18c4c4 > 0x000fc4af: mov$0x30,%ax > 0x000fc4b2: add%al,(%bx,%si) > 0x000fc4b4: mov%ax,%ds > 0x000fc4b6: mov%ax,%es > 0x000fc4b8: mov%ax,%ss > 0x000fc4ba: mov%ax,%fs > 0x000fc4bc: mov%ax,%gs > 0x000fc4be: ljmpl $0x200f,$0x28c4c4 > 0x000fc4c6: shlb $0xe0,-0x7d(%bp) > 0x000fc4ca: decb (%bx) > 0x000fc4cc: and%al,%al > 0x000fc4ce: ljmp $0xf000,$0xc4d3 > 0x000fc4d3: lidtw %cs:-0x2c18 > 0x000fc4d9: xor%ax,%ax > 0x000fc4db: mov%ax,%fs > 0x000fc4dd: mov%ax,%gs > 0x000fc4df: mov%ax,%es > 0x000fc4e1: mov%ax,%ds > 0x000fc4e3: mov%ax,%ss > 0x000fc4e5: mov%ecx,%eax > 0x000fc4e8: jmpl *%edx > 0x000fc4eb: push %ebp > 0x000fc4ed: push %eax > 0x000fc4ef: pushl %es > 0x000fc4f1: push %cs > 0x000fc4f2: push $0xc536 > 0x000fc4f5: addr32 pushw %es:0x24(%eax) > 0x000fc4fa: addr32 pushl %es:0x20(%eax) > 0x000fc500: addr32 mov %es:0x4(%eax),%edi > 0x000fc506: addr32 mov %es:0x8(%eax),%esi > 0x000fc50c: addr32 mov %es:0xc(%eax),%ebp > 0x000fc512: addr32 mov %es:0x10(%eax),%ebx > 0x000fc518: addr32 mov %es:0x14(%eax),%edx > 0x000fc51e: addr32 mov %es:0x18(%eax),%ecx > 0x000fc524: addr32 mov %es:(%eax),%ds > 0x000fc528: addr32 pushl %es:0x1c(%eax) > 0x000fc52e: addr32 mov %es:0x2(%eax),%es > 0x000fc533: pop%eax > 0x000fc535: iret > 0x000fc536: pushf > 0x000fc537: cli > > -- > To unsubscribe from this list: send the line "unsubscribe kvm" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
