Re: Network hangs when communicating with host
[apologies for the delay -- I've been off for a week and am catching up on email] On Tue, Oct 20, 2015 at 09:58:51AM -0400, Sasha Levin wrote: > On 10/20/2015 09:42 AM, Dmitry Vyukov wrote: > > I now have another issue. My binary fails to mmap a file within lkvm > > sandbox. The same binary works fine on host and in qemu. I've added > > strace into sandbox script, and here is the output: > > > > [pid 837] openat(AT_FDCWD, "syzkaller-shm048878722", O_RDWR|O_CLOEXEC) = 5 > > [pid 837] mmap(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_SHARED, 5, > > 0) = -1 EINVAL (Invalid argument) > > > > I don't see anything that can potentially cause EINVAL here. Is it > > possible that lkvm somehow affects kernel behavior here? > > > > I run lkvm as: > > > > $ taskset 1 /kvmtool/lkvm sandbox --disk syz-0 --mem=2048 --cpus=2 > > --kernel /arch/x86/boot/bzImage --network mode=user --sandbox > > /workdir/kvm/syz-0.sh > > It's possible that something in the virtio-9p layer is broken. I'll give > it a look in the evening. I ended up with the patch below, but it's really ugly and I didn't get round to posting it. Will --->8 >From 7cbcdfef1b9f094db4bf75676f22339f3164e103 Mon Sep 17 00:00:00 2001 From: Will DeaconDate: Fri, 17 Apr 2015 17:31:36 +0100 Subject: [PATCH] kvmtool: virtio-net: fix VIRTIO_NET_F_MRG_RXBUF usage in rx thread When merging virtio-net buffers using the VIRTIO_NET_F_MRG_RXBUF feature, the first buffer added to the used ring should indicate the total number of buffers used to hold the packet. Unfortunately, kvmtool has a number of issues when constructing these merged buffers: - Commit 5131332e3f1a ("kvmtool: convert net backend to support bi-endianness") introduced a strange loop counter, which resulted in hdr->num_buffers being set redundantly the first time round - When adding the buffers to the ring, we actually add them one-by-one, allowing the guest to see the header before we've inserted the rest of the data buffers... - ... which is made worse because we non-atomically increment the num_buffers count in the header each time we insert a new data buffer Consequently, the guest quickly becomes confused in its net rx code and the whole thing grinds to a halt. This is easily exemplified by trying to boot a root filesystem over NFS, which seldom succeeds. This patch resolves the issues by allowing us to insert items into the used ring without updating the index. Once the full payload has been added and num_buffers corresponds to the total size, we *then* publish the buffers to the guest. Cc: Marc Zyngier Cc: Sasha Levin Signed-off-by: Will Deacon --- include/kvm/virtio.h | 2 ++ virtio/core.c| 32 +--- virtio/net.c | 31 +++ 3 files changed, 42 insertions(+), 23 deletions(-) diff --git a/include/kvm/virtio.h b/include/kvm/virtio.h index 768ee9668d44..8324ba7d38be 100644 --- a/include/kvm/virtio.h +++ b/include/kvm/virtio.h @@ -112,6 +112,8 @@ static inline bool virt_queue__available(struct virt_queue *vq) return virtio_guest_to_host_u16(vq, vq->vring.avail->idx) != vq->last_avail_idx; } +void virt_queue__used_idx_advance(struct virt_queue *queue, u16 jump); +struct vring_used_elem * virt_queue__set_used_elem_no_update(struct virt_queue *queue, u32 head, u32 len, u16 offset); struct vring_used_elem *virt_queue__set_used_elem(struct virt_queue *queue, u32 head, u32 len); bool virtio_queue__should_signal(struct virt_queue *vq); diff --git a/virtio/core.c b/virtio/core.c index 3b6e4d7cd045..d6ac289d450e 100644 --- a/virtio/core.c +++ b/virtio/core.c @@ -21,22 +21,17 @@ const char* virtio_trans_name(enum virtio_trans trans) return "unknown"; } -struct vring_used_elem *virt_queue__set_used_elem(struct virt_queue *queue, u32 head, u32 len) +void virt_queue__used_idx_advance(struct virt_queue *queue, u16 jump) { - struct vring_used_elem *used_elem; u16 idx = virtio_guest_to_host_u16(queue, queue->vring.used->idx); - used_elem = >vring.used->ring[idx % queue->vring.num]; - used_elem->id = virtio_host_to_guest_u32(queue, head); - used_elem->len = virtio_host_to_guest_u32(queue, len); - /* * Use wmb to assure that used elem was updated with head and len. * We need a wmb here since we can't advance idx unless we're ready * to pass the used element to the guest. */ wmb(); - idx++; + idx += jump; queue->vring.used->idx = virtio_host_to_guest_u16(queue, idx); /* @@ -45,6 +40,29 @@ struct vring_used_elem *virt_queue__set_used_elem(struct virt_queue *queue, u32 * an updated idx. */ wmb(); +} + +struct vring_used_elem * +virt_queue__set_used_elem_no_update(struct virt_queue *queue, u32 head, + u32
Re: Bug#802926: linux-image-4.2.0-1-amd64: KVM hangs with 100% cpu on 4.2
It seems that real mode virtualisation on Nehalem has regressed in 4.2: On Sun, 2015-10-25 at 10:08 +0100, Stefan Fritsch wrote: [...] > I cannot use KVM with 4.2, qemu loops with 100% CPU during seabios > initialization. Booting with the latest linux-image-4.1.0-2-amd64 fixes > the issue. [...] > kvm tracing shows: > > ... > qemu-system-x86-3219 [007] 1090.728418: kvm_set_irq: gsi 12 level 0 > source 0 > qemu-system-x86-3219 [007] 1090.728418: kvm_pic_set_irq: chip 1 pin 4 > (edge|masked) > qemu-system-x86-3219 [007] 1090.728419: kvm_ioapic_set_irq: pin 12 > dst 0 vec=0 (Fixed|physical|edge|masked) > qemu-system-x86-3219 [007] 1090.728419: kvm_set_irq: gsi 1 level 0 > source 0 > qemu-system-x86-3219 [007] 1090.728420: kvm_pic_set_irq: chip 0 pin 1 > (edge|masked) > qemu-system-x86-3219 [007] 1090.728420: kvm_ioapic_set_irq: pin 1 dst > 0 vec=0 (Fixed|physical|edge|masked) > qemu-system-x86-3219 [007] 1090.728420: kvm_set_irq: gsi 12 level 0 > source 0 > qemu-system-x86-3219 [007] 1090.728421: kvm_pic_set_irq: chip 1 pin 4 > (edge|masked) > qemu-system-x86-3219 [007] 1090.728421: kvm_ioapic_set_irq: pin 12 > dst 0 vec=0 (Fixed|physical|edge|masked) > qemu-system-x86-3219 [007] 1090.728723: kvm_set_irq: gsi 0 level 0 > source 0 > qemu-system-x86-3219 [007] 1090.728724: kvm_pic_set_irq: chip 0 pin 0 > (edge) > qemu-system-x86-3219 [007] 1090.728725: kvm_ioapic_set_irq: pin 2 dst > 0 vec=0 (Fixed|physical|edge|masked) > qemu-system-x86-3219 [007] 1090.728725: kvm_set_irq: gsi 0 level 0 > source 0 > qemu-system-x86-3219 [007] 1090.728725: kvm_pic_set_irq: chip 0 pin 0 > (edge) > qemu-system-x86-3219 [007] 1090.728725: kvm_ioapic_set_irq: pin 2 dst > 0 vec=0 (Fixed|physical|edge|masked) > qemu-system-x86-3219 [007] 1090.728726: kvm_set_irq: gsi 0 level 0 > source 0 > qemu-system-x86-3219 [007] 1090.728726: kvm_pic_set_irq: chip 0 pin 0 > (edge) > qemu-system-x86-3219 [007] 1090.728726: kvm_ioapic_set_irq: pin 2 dst > 0 vec=0 (Fixed|physical|edge|masked) > qemu-system-x86-3221 [000] d... 1090.729926: kvm_write_tsc_offset: vcpu=0 > prev=18446740943986499809 next=18446740943986499809 > qemu-system-x86-3221 [000] 1090.729927: kvm_track_tsc: vcpu_id 0 > masterclock 1 offsetmatched 0 nr_online 1 hostclock tsc > qemu-system-x86-3221 [000] 1090.730004: kvm_update_master_clock: > masterclock 1 hostclock tsc offsetmatched 1 > qemu-system-x86-3221 [000] d... 1090.730010: kvm_entry: vcpu 0 > qemu-system-x86-3221 [000] 1090.730013: kvm_emulate_insn: > :fff0:ea 5b e0 00 f0 (real) > qemu-system-x86-3221 [000] d... 1090.730016: kvm_entry: vcpu 0 > qemu-system-x86-3221 [000] d... 1090.730017: kvm_exit: reason > EPT_VIOLATION rip 0xe05b info 81 0 > qemu-system-x86-3221 [000] 1090.730018: kvm_page_fault: address > feffc000 error_code 81 > qemu-system-x86-3221 [000] d... 1090.730032: kvm_entry: vcpu 0 > qemu-system-x86-3221 [000] d... 1090.730034: kvm_exit: reason > EXCEPTION_NMI rip 0xe05b info 0 8b08 > qemu-system-x86-3221 [000] 1090.730035: kvm_inj_exception: #DF (0x0) > qemu-system-x86-3221 [000] d... 1090.730040: kvm_entry: vcpu 0 > qemu-system-x86-3221 [000] d... 1090.730041: kvm_exit: reason > EXCEPTION_NMI rip 0xfea5 info 0 8b08 > qemu-system-x86-3221 [000] 1090.730041: kvm_inj_exception: #DF (0x0) > qemu-system-x86-3221 [000] d... 1090.730043: kvm_entry: vcpu 0 > ... [...] > I am rather surprised that there is no bug report about this yet, so I > guess it only happens on some hardware. Mine is a Core i7-860. Ben. -- Ben Hutchings Beware of bugs in the above code; I have only proved it correct, not tried it. - Donald Knuth signature.asc Description: This is a digitally signed message part
Re: Network hangs when communicating with host
On 10/20/2015 09:42 AM, Dmitry Vyukov wrote: > I now have another issue. My binary fails to mmap a file within lkvm > sandbox. The same binary works fine on host and in qemu. I've added > strace into sandbox script, and here is the output: > > [pid 837] openat(AT_FDCWD, "syzkaller-shm048878722", O_RDWR|O_CLOEXEC) = 5 > [pid 837] mmap(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_SHARED, 5, > 0) = -1 EINVAL (Invalid argument) > > I don't see anything that can potentially cause EINVAL here. Is it > possible that lkvm somehow affects kernel behavior here? > > I run lkvm as: > > $ taskset 1 /kvmtool/lkvm sandbox --disk syz-0 --mem=2048 --cpus=2 > --kernel /arch/x86/boot/bzImage --network mode=user --sandbox > /workdir/kvm/syz-0.sh It's possible that something in the virtio-9p layer is broken. I'll give it a look in the evening. Thanks, Sasha -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Network hangs when communicating with host
I now have another issue. My binary fails to mmap a file within lkvm sandbox. The same binary works fine on host and in qemu. I've added strace into sandbox script, and here is the output: [pid 837] openat(AT_FDCWD, "syzkaller-shm048878722", O_RDWR|O_CLOEXEC) = 5 [pid 837] mmap(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_SHARED, 5, 0) = -1 EINVAL (Invalid argument) I don't see anything that can potentially cause EINVAL here. Is it possible that lkvm somehow affects kernel behavior here? I run lkvm as: $ taskset 1 /kvmtool/lkvm sandbox --disk syz-0 --mem=2048 --cpus=2 --kernel /arch/x86/boot/bzImage --network mode=user --sandbox /workdir/kvm/syz-0.sh On Mon, Oct 19, 2015 at 4:20 PM, Sasha Levin <sasha.le...@oracle.com> wrote: > On 10/19/2015 05:28 AM, Dmitry Vyukov wrote: >> On Mon, Oct 19, 2015 at 11:22 AM, Andre Przywara <andre.przyw...@arm.com> >> wrote: >>> Hi Dmitry, >>> >>> On 19/10/15 10:05, Dmitry Vyukov wrote: >>>> On Fri, Oct 16, 2015 at 7:25 PM, Sasha Levin <sasha.le...@oracle.com> >>>> wrote: >>>>> On 10/15/2015 04:20 PM, Dmitry Vyukov wrote: >>>>>> Hello, >>>>>> >>>>>> I am trying to run a program in lkvm sandbox so that it communicates >>>>>> with a program on host. I run lkvm as: >>>>>> >>>>>> ./lkvm sandbox --disk sandbox-test --mem=2048 --cpus=4 --kernel >>>>>> /arch/x86/boot/bzImage --network mode=user -- /my_prog >>>>>> >>>>>> /my_prog then connects to a program on host over a tcp socket. >>>>>> I see that host receives some data, sends some data back, but then >>>>>> my_prog hangs on network read. >>>>>> >>>>>> To localize this I wrote 2 programs (attached). ping is run on host >>>>>> and pong is run from lkvm sandbox. They successfully establish tcp >>>>>> connection, but after some iterations both hang on read. >>>>>> >>>>>> Networking code in Go runtime is there for more than 3 years, widely >>>>>> used in production and does not have any known bugs. However, it uses >>>>>> epoll edge-triggered readiness notifications that known to be tricky. >>>>>> Is it possible that lkvm contains some networking bug? Can it be >>>>>> related to the data races in lkvm I reported earlier today? >>> >>> Just to let you know: >>> I think we have seen networking issues in the past - root over NFS had >>> issues IIRC. Will spent some time on debugging this and it looked like a >>> race condition in kvmtool's virtio implementation. I think pinning >>> kvmtool's virtio threads to one host core made this go away. However >>> although he tried hard (even by Will's standards!) he couldn't find a >>> the real root cause or a fix at the time he looked at it and we found >>> other ways to work around the issues (using virtio-blk or initrd's). >>> >>> So it's quite possible that there are issues. I haven't had time yet to >>> look at your sanitizer reports, but it looks like a promising approach >>> to find the root cause. >> >> >> Thanks, Andre! >> >> ping/pong does not hang within at least 5 minutes when I run lkvm >> under taskset 1. >> >> And, yeah, this pretty strongly suggests a data race. ThreadSanitizer >> can point you to the bug within a minute, so you just need to say >> "aha! it is here". Or maybe not. There are no guarantees. But if you >> already spent significant time on this, then checking the reports >> definitely looks like a good idea. > > Okay, that's good to know. > > I have a few busy days, but I'll definitely try to clear up these reports > as they seem to be pointing to real issues. > > > Thanks, > Sasha > -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Network hangs when communicating with host
Hi Dmitry, On 19/10/15 10:05, Dmitry Vyukov wrote: > On Fri, Oct 16, 2015 at 7:25 PM, Sasha Levin <sasha.le...@oracle.com> wrote: >> On 10/15/2015 04:20 PM, Dmitry Vyukov wrote: >>> Hello, >>> >>> I am trying to run a program in lkvm sandbox so that it communicates >>> with a program on host. I run lkvm as: >>> >>> ./lkvm sandbox --disk sandbox-test --mem=2048 --cpus=4 --kernel >>> /arch/x86/boot/bzImage --network mode=user -- /my_prog >>> >>> /my_prog then connects to a program on host over a tcp socket. >>> I see that host receives some data, sends some data back, but then >>> my_prog hangs on network read. >>> >>> To localize this I wrote 2 programs (attached). ping is run on host >>> and pong is run from lkvm sandbox. They successfully establish tcp >>> connection, but after some iterations both hang on read. >>> >>> Networking code in Go runtime is there for more than 3 years, widely >>> used in production and does not have any known bugs. However, it uses >>> epoll edge-triggered readiness notifications that known to be tricky. >>> Is it possible that lkvm contains some networking bug? Can it be >>> related to the data races in lkvm I reported earlier today? Just to let you know: I think we have seen networking issues in the past - root over NFS had issues IIRC. Will spent some time on debugging this and it looked like a race condition in kvmtool's virtio implementation. I think pinning kvmtool's virtio threads to one host core made this go away. However although he tried hard (even by Will's standards!) he couldn't find a the real root cause or a fix at the time he looked at it and we found other ways to work around the issues (using virtio-blk or initrd's). So it's quite possible that there are issues. I haven't had time yet to look at your sanitizer reports, but it looks like a promising approach to find the root cause. Cheers, Andre. >>> >>> I am on commit 3695adeb227813d96d9c41850703fb53a23845eb. >> >> Hey Dmitry, >> >> How long does it take to reproduce? I've been running ping/pong as you've >> described and it looks like it doesn't get stuck (read/writes keep going >> on both sides). >> >> Can you share your guest kernel config maybe? > > > Humm it my setup it happens within a minute or so. > > My kernel is not completely standard, but it works with qemu without > any problems. > It is not trivial to reproduce, but FWIW I on commit > f9fbf6b72ffaaca8612979116c872c9d5d9cc1f5 of > https://github.com/dvyukov/linux/commits/coverage branch. Config file > is attached. Then, I build it with custom gcc: revision 228818 + > https://codereview.appspot.com/267910043 patch. This is all per > https://github.com/google/syzkaller instructions. > > I run lkvm as: > ./lkvm sandbox --disk sandbox-test --mem=2048 --cpus=4 --kernel > /arch/x86/boot/bzImage --network mode=user -- /pong > > kvmtool is on 3695adeb227813d96d9c41850703fb53a23845eb. > > Just tried to do the same with qemu, it does not hang. > -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Network hangs when communicating with host
On Mon, Oct 19, 2015 at 11:22 AM, Andre Przywara <andre.przyw...@arm.com> wrote: > Hi Dmitry, > > On 19/10/15 10:05, Dmitry Vyukov wrote: >> On Fri, Oct 16, 2015 at 7:25 PM, Sasha Levin <sasha.le...@oracle.com> wrote: >>> On 10/15/2015 04:20 PM, Dmitry Vyukov wrote: >>>> Hello, >>>> >>>> I am trying to run a program in lkvm sandbox so that it communicates >>>> with a program on host. I run lkvm as: >>>> >>>> ./lkvm sandbox --disk sandbox-test --mem=2048 --cpus=4 --kernel >>>> /arch/x86/boot/bzImage --network mode=user -- /my_prog >>>> >>>> /my_prog then connects to a program on host over a tcp socket. >>>> I see that host receives some data, sends some data back, but then >>>> my_prog hangs on network read. >>>> >>>> To localize this I wrote 2 programs (attached). ping is run on host >>>> and pong is run from lkvm sandbox. They successfully establish tcp >>>> connection, but after some iterations both hang on read. >>>> >>>> Networking code in Go runtime is there for more than 3 years, widely >>>> used in production and does not have any known bugs. However, it uses >>>> epoll edge-triggered readiness notifications that known to be tricky. >>>> Is it possible that lkvm contains some networking bug? Can it be >>>> related to the data races in lkvm I reported earlier today? > > Just to let you know: > I think we have seen networking issues in the past - root over NFS had > issues IIRC. Will spent some time on debugging this and it looked like a > race condition in kvmtool's virtio implementation. I think pinning > kvmtool's virtio threads to one host core made this go away. However > although he tried hard (even by Will's standards!) he couldn't find a > the real root cause or a fix at the time he looked at it and we found > other ways to work around the issues (using virtio-blk or initrd's). > > So it's quite possible that there are issues. I haven't had time yet to > look at your sanitizer reports, but it looks like a promising approach > to find the root cause. Thanks, Andre! ping/pong does not hang within at least 5 minutes when I run lkvm under taskset 1. And, yeah, this pretty strongly suggests a data race. ThreadSanitizer can point you to the bug within a minute, so you just need to say "aha! it is here". Or maybe not. There are no guarantees. But if you already spent significant time on this, then checking the reports definitely looks like a good idea. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Network hangs when communicating with host
On 10/19/2015 05:28 AM, Dmitry Vyukov wrote: > On Mon, Oct 19, 2015 at 11:22 AM, Andre Przywara <andre.przyw...@arm.com> > wrote: >> Hi Dmitry, >> >> On 19/10/15 10:05, Dmitry Vyukov wrote: >>> On Fri, Oct 16, 2015 at 7:25 PM, Sasha Levin <sasha.le...@oracle.com> wrote: >>>> On 10/15/2015 04:20 PM, Dmitry Vyukov wrote: >>>>> Hello, >>>>> >>>>> I am trying to run a program in lkvm sandbox so that it communicates >>>>> with a program on host. I run lkvm as: >>>>> >>>>> ./lkvm sandbox --disk sandbox-test --mem=2048 --cpus=4 --kernel >>>>> /arch/x86/boot/bzImage --network mode=user -- /my_prog >>>>> >>>>> /my_prog then connects to a program on host over a tcp socket. >>>>> I see that host receives some data, sends some data back, but then >>>>> my_prog hangs on network read. >>>>> >>>>> To localize this I wrote 2 programs (attached). ping is run on host >>>>> and pong is run from lkvm sandbox. They successfully establish tcp >>>>> connection, but after some iterations both hang on read. >>>>> >>>>> Networking code in Go runtime is there for more than 3 years, widely >>>>> used in production and does not have any known bugs. However, it uses >>>>> epoll edge-triggered readiness notifications that known to be tricky. >>>>> Is it possible that lkvm contains some networking bug? Can it be >>>>> related to the data races in lkvm I reported earlier today? >> >> Just to let you know: >> I think we have seen networking issues in the past - root over NFS had >> issues IIRC. Will spent some time on debugging this and it looked like a >> race condition in kvmtool's virtio implementation. I think pinning >> kvmtool's virtio threads to one host core made this go away. However >> although he tried hard (even by Will's standards!) he couldn't find a >> the real root cause or a fix at the time he looked at it and we found >> other ways to work around the issues (using virtio-blk or initrd's). >> >> So it's quite possible that there are issues. I haven't had time yet to >> look at your sanitizer reports, but it looks like a promising approach >> to find the root cause. > > > Thanks, Andre! > > ping/pong does not hang within at least 5 minutes when I run lkvm > under taskset 1. > > And, yeah, this pretty strongly suggests a data race. ThreadSanitizer > can point you to the bug within a minute, so you just need to say > "aha! it is here". Or maybe not. There are no guarantees. But if you > already spent significant time on this, then checking the reports > definitely looks like a good idea. Okay, that's good to know. I have a few busy days, but I'll definitely try to clear up these reports as they seem to be pointing to real issues. Thanks, Sasha -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Network hangs when communicating with host
On 10/15/2015 04:20 PM, Dmitry Vyukov wrote: > Hello, > > I am trying to run a program in lkvm sandbox so that it communicates > with a program on host. I run lkvm as: > > ./lkvm sandbox --disk sandbox-test --mem=2048 --cpus=4 --kernel > /arch/x86/boot/bzImage --network mode=user -- /my_prog > > /my_prog then connects to a program on host over a tcp socket. > I see that host receives some data, sends some data back, but then > my_prog hangs on network read. > > To localize this I wrote 2 programs (attached). ping is run on host > and pong is run from lkvm sandbox. They successfully establish tcp > connection, but after some iterations both hang on read. > > Networking code in Go runtime is there for more than 3 years, widely > used in production and does not have any known bugs. However, it uses > epoll edge-triggered readiness notifications that known to be tricky. > Is it possible that lkvm contains some networking bug? Can it be > related to the data races in lkvm I reported earlier today? > > I am on commit 3695adeb227813d96d9c41850703fb53a23845eb. Hey Dmitry, How long does it take to reproduce? I've been running ping/pong as you've described and it looks like it doesn't get stuck (read/writes keep going on both sides). Can you share your guest kernel config maybe? Thanks, Sasha -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Network hangs when communicating with host
Hello, I am trying to run a program in lkvm sandbox so that it communicates with a program on host. I run lkvm as: ./lkvm sandbox --disk sandbox-test --mem=2048 --cpus=4 --kernel /arch/x86/boot/bzImage --network mode=user -- /my_prog /my_prog then connects to a program on host over a tcp socket. I see that host receives some data, sends some data back, but then my_prog hangs on network read. To localize this I wrote 2 programs (attached). ping is run on host and pong is run from lkvm sandbox. They successfully establish tcp connection, but after some iterations both hang on read. Networking code in Go runtime is there for more than 3 years, widely used in production and does not have any known bugs. However, it uses epoll edge-triggered readiness notifications that known to be tricky. Is it possible that lkvm contains some networking bug? Can it be related to the data races in lkvm I reported earlier today? I am on commit 3695adeb227813d96d9c41850703fb53a23845eb. Thank you package main import ( "log" "net" ) func main() { c, err := net.Dial("tcp", "192.168.33.1:39921") if err != nil { log.Fatalf("failed to dial: %v", err) } log.Printf("connected") for { var buf [1]byte n, err := c.Write(buf[:]) if err != nil || n != 1 { log.Fatalf("failed to write: %v", err) } log.Printf("write") n, err = c.Read(buf[:]) if err != nil || n != 1 { log.Fatalf("failed to read: %v", err) } log.Printf("read") } } package main import ( "log" "net" ) func main() { ln, err := net.Listen("tcp", "127.0.0.1:39921") if err != nil { log.Fatalf("failed to listen: %v", err) } log.Printf("listening") c, err := ln.Accept() if err != nil { log.Fatalf("failed to accept: %v", err) } log.Printf("connected") for { var buf [1]byte n, err := c.Read(buf[:]) if err != nil || n != 1 { log.Fatalf("failed to read: %v", err) } log.Printf("read") n, err = c.Write(buf[:]) if err != nil || n != 1 { log.Fatalf("failed to write: %v", err) } log.Printf("write") } }
[Bug 103851] New: qemu windows guest hangs on 100% cpu usage
https://bugzilla.kernel.org/show_bug.cgi?id=103851 Bug ID: 103851 Summary: qemu windows guest hangs on 100% cpu usage Product: Virtualization Version: unspecified Kernel Version: 3.13.6 Hardware: Intel OS: Linux Tree: Mainline Status: NEW Severity: blocking Priority: P1 Component: kvm Assignee: virtualization_...@kernel-bugs.osdl.org Reporter: biaoxian...@163.com Regression: No hi: I have two VM , one is winXP Prefessional SP3 32bit, another one is WindowsServer2008 Enterprise SP2 64bit. When I hot reboot winXP in guest OS, it'll hangs on progress bar, and all the vcpu thread in qemu is 100% usage. There are no message in kernel log when it happened, I try to rebuild kvm and add some debug info, I found the cpu exit reason is EXIT_REASON_PAUSE_INSTRUCTION. It seems like all the vcpu always in spinlock waiting. I'm not sure whether it's qemu's bug or kvm's. Any help would be appreciated. How reproducible: WinXP: seems always. WinServer2008: rare. Steps to Reproduce: winXP: hot reboot the xp guest os, hot reboot is necessary. WinServer2008: not sure, I didn't do anything, it just happened. The different between WinXP and WInServer2008: 1. When WinXP hangs, the boot progress bar is rolling, I think that vnc is work fine. 2. When WinServer2008 hangs, the vnc show the last screen and the screen won't change anything include system time. 3. When the VM hangs , if I execute "virsh suspend vm-name" and "virsh resume vm-name", the WinServer2008 will change to normal , and work fine not hangs anymore. But WinXP not change anything, still hangs. qemu version: QEMU emulator version 1.5.0, Copyright (c) 2003-2008 Fabrice Bellard host info: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz Ubuntu 12.04 LTS \n \l Linux cvknode2026 3.13.6 #1 SMP Fri Dec 12 09:17:35 CST 2014 x86_64 x86_64 x86_64 GNU/Linux qemu command line (guest OS XP): root 7124 1178 7.6 7750360 3761644 ? Sl 14:02 435:23 /usr/bin/kvm -name x -S -machine pc-i440fx-1.5,accel=kvm,usb=off,system=windows -cpu qemu64,hv_relaxed,hv_spinlocks=0x2000 -m 6144 -smp 12,maxcpus=72,sockets=12,cores=6,threads=1 -uuid d3832129-f77d-4b21-bbf7-fd337f53e572 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/x.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,clock=vm,driftfix=slew -no-hpet -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device usb-ehci,id=ehci,bus=pci.0,addr=0x4 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/vms/images/sn1-of-ff.qcow2,if=none,id=drive-ide0-0-0,format=qcow2,cache=directsync -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive if=none,id=drive-ide0-1-1,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=1,drive=drive-ide0-1-1,id=ide0-1-1,bootindex=2 -netdev tap,fd=24,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=0c:da:41:1d:f8:40,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/x.agent,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device usb-tablet,id=input0,bus=usb.0 -vnc 0.0.0.0:0 -device VGA,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 all qemu thread (guest OS XP): root@cvknode2026:/proc/7124/task# top -d 1 -H -p 7124 top - 14:37:05 up 7 days, 4:07, 1 user, load average: 10.71, 10.90, 10.19 Tasks: 14 total, 12 running, 2 sleeping, 0 stopped, 0 zombie Cpu(s): 38.8%us, 11.2%sy, 0.0%ni, 50.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 49159888k total, 35665128k used, 13494760k free, 436312k buffers Swap: 8803324k total, 0k used, 8803324k free, 28595100k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ P SWAP WCHAN COMMAND 7130 root 20 0 7568m 3.6g 6628 R 101 7.7 33:43.48 3 3.8g - kvm 7132 root 20 0 7568m 3.6g 6628 R 101 7.7 33:43.13 1 3.8g - kvm 7133 root 20 0 7568m 3.6g 6628 R 101 7.7 33:42.70 6 3.8g - kvm 7135 root 20 0 7568m 3.6g 6628 R 101 7.7 33:42.33 11 3.8g - kvm 7137 root 20 0 7568m 3.6g 6628 R 101 7.7 33:42.59 17 3.8g - kvm 7126 root 20 0 7568m 3.6g 6628 R 100 7.7 34:06.76 4 3.8g - kvm 7127 root 20 0 7568m 3.6g 6628 R 100 7.7 33:44.14 8 3.8g - kvm 7128 root 20 0 7568m 3.6g 6628 R 100 7.7 33:43.64 13 3.8g - kvm 7129 root 20 0 7568m 3.6g 6628 R 100 7.7 33:43.64 7 3.8g - kvm 7131 root 20 0 7568m 3.6g 6628 R 100 7.7 33:44.24 10 3.8g - kvm 7134 root 20 0 7568m 3.6g 6628 R 100 7.7 33:42.47 12 3.8g - kvm 7136 root 20 0 7568m 3.6g 6628 R 100 7.7 33:42.16 2 3.8g - kvm 7124 root 20 0 7568m 3.6g 6628 S 1 7.7 0:30.65 14 3.8g poll_sche kvm 7139 root 20 0 7568m 3.6g 6628 S 0 7.7 0:01.71 14 3.8g futex_wai kvm all thread's kernel stack (guest OS XP): root@cvknode
[Bug 103851] qemu windows guest hangs on 100% cpu usage
https://bugzilla.kernel.org/show_bug.cgi?id=103851 Wanpeng Lichanged: What|Removed |Added CC||wanpeng...@hotmail.com --- Comment #1 from Wanpeng Li --- If this can be reproduced against latest kvm tree? -- You are receiving this mail because: You are watching the assignee of the bug. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 93251] qemu-kvm guests randomly hangs after reboot command in guest
https://bugzilla.kernel.org/show_bug.cgi?id=93251 --- Comment #15 from Igor Mammedov imamm...@redhat.com --- Fixed in v4.0 commit 744961341d472db6272ed9b42319a90f5a2aa7c4 kvm: avoid page allocation failure in kvm_set_memory_region() -- You are receiving this mail because: You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 93251] qemu-kvm guests randomly hangs after reboot command in guest
https://bugzilla.kernel.org/show_bug.cgi?id=93251 --- Comment #12 from Thomas Stein himbe...@meine-oma.de --- Hello Igor. Is this bug in 3.18 also present? I'm asking because i consider a downgrade. thanks and cheers t. -- You are receiving this mail because: You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 93251] qemu-kvm guests randomly hangs after reboot command in guest
https://bugzilla.kernel.org/show_bug.cgi?id=93251 --- Comment #13 from Igor Mammedov imamm...@redhat.com --- Nope, it's only since 3.19. Could you test patch in comment 11? -- You are receiving this mail because: You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 93251] qemu-kvm guests randomly hangs after reboot command in guest
https://bugzilla.kernel.org/show_bug.cgi?id=93251 --- Comment #14 from Thomas Stein himbe...@meine-oma.de --- I have patch from comment 11 already running on two machines. No problems so far. -- You are receiving this mail because: You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 93251] qemu-kvm guests randomly hangs after reboot command in guest
https://bugzilla.kernel.org/show_bug.cgi?id=93251 --- Comment #11 from Igor Mammedov imamm...@redhat.com --- (In reply to Thomas Stein from comment #10) Hello. I suppose this patch is not included in 3.19.3? Nope, I've posted v2 of the patch fixing problems Marcelo pointed out: https://www.marc.info/?l=kvmm=142685412907951w=3 so it' needs to be reviewed/tested. -- You are receiving this mail because: You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 93251] qemu-kvm guests randomly hangs after reboot command in guest
https://bugzilla.kernel.org/show_bug.cgi?id=93251 --- Comment #10 from Thomas Stein himbe...@meine-oma.de --- Hello. I suppose this patch is not included in 3.19.3? thanks and cheers t. -- You are receiving this mail because: You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 93251] qemu-kvm guests randomly hangs after reboot command in guest
https://bugzilla.kernel.org/show_bug.cgi?id=93251 --- Comment #8 from Igor Mammedov imamm...@redhat.com --- (In reply to Thomas Stein from comment #7) Hello. After reverting commit 1d4e7e3c0bca747d0fc54069a6ab8393349431c0 i had no problem any more. But we have to keep in mind this error only happend now and then. Especially creating snapshots of running vm's triggered the error but this runs without crashing the vm now. I tested it a few times. cheers t. Could you try following patch without reverting 1d4e7e3c0bca ? http://article.gmane.org/gmane.comp.emulators.kvm.devel/134191 -- You are receiving this mail because: You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 93251] qemu-kvm guests randomly hangs after reboot command in guest
https://bugzilla.kernel.org/show_bug.cgi?id=93251 --- Comment #9 from Thomas Stein himbe...@meine-oma.de --- Hello. I applied the patch to vanilla 3.19.2. No problems so far. Did a few snapshots and vm restarts. cheers t. -- You are receiving this mail because: You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 93251] qemu-kvm guests randomly hangs after reboot command in guest
https://bugzilla.kernel.org/show_bug.cgi?id=93251 --- Comment #7 from Thomas Stein himbe...@meine-oma.de --- Hello. After reverting commit 1d4e7e3c0bca747d0fc54069a6ab8393349431c0 i had no problem any more. But we have to keep in mind this error only happend now and then. Especially creating snapshots of running vm's triggered the error but this runs without crashing the vm now. I tested it a few times. cheers t. -- You are receiving this mail because: You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 93251] qemu-kvm guests randomly hangs after reboot command in guest
https://bugzilla.kernel.org/show_bug.cgi?id=93251 Igor Mammedov imamm...@redhat.com changed: What|Removed |Added CC||imamm...@redhat.com --- Comment #6 from Igor Mammedov imamm...@redhat.com --- Since commit 1d4e7e3c increased struct kvm_memslots size from order 2 to order 4, like-hood of above failure grew in ~2x times (there where 90 order 2 free pages) Looking at log from comment 1 allocation failure shouldn't happened since there are still 40 Order 4 free pages: úno 13 18:10:27 argus kernel: Normal: 18555*4kB (UEMC) 145*8kB (UMC) 92*16kB (UM) 54*32kB (UM) 40*64kB (UM) 9*128kB (UM) 2*256kB (U) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 82804kB Anyway I'll post patch that changes allocation vmalloc-ed memory as quick fix and look for possibility of allocating actual amount of slots instead of possible maximum like it's done now. -- You are receiving this mail because: You are watching the assignee of the bug.-- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 93251] qemu-kvm guests randomly hangs after reboot command in guest
https://bugzilla.kernel.org/show_bug.cgi?id=93251 --- Comment #5 from Thomas Stein himbe...@meine-oma.de --- Hi. Reverted the commit right now. I have to boot with this kernel. More tomorrow. Thanks and cheers t. -- You are receiving this mail because: You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 93251] qemu-kvm guests randomly hangs after reboot command in guest
https://bugzilla.kernel.org/show_bug.cgi?id=93251 Thomas Stein himbe...@meine-oma.de changed: What|Removed |Added CC||himbe...@meine-oma.de --- Comment #3 from Thomas Stein himbe...@meine-oma.de --- Hello. I have the very same error on my kvm machines. Mar 19 20:35:07 hn kernel: qemu-system-x86: page allocation failure: order:4, mode:0x40d0 Mar 19 20:35:07 hn kernel: CPU: 0 PID: 7029 Comm: qemu-system-x86 Not tainted 3.19.2 #1 Mar 19 20:35:07 hn kernel: Hardware name: System manufacturer System Product Name/P8B WS, BIOS 0802 08/29/ 2011 Mar 19 20:35:07 hn kernel: 88038857f9d8 81711fce 0001 Mar 19 20:35:07 hn kernel: 40d0 88038857fa68 81108d4e 88042fded838 Mar 19 20:35:07 hn kernel: 0380 03c0 0286 ffff Mar 19 20:35:07 hn kernel: Call Trace: Mar 19 20:35:07 hn kernel: [81711fce] dump_stack+0x45/0x57 Mar 19 20:35:07 hn kernel: [81108d4e] warn_alloc_failed+0xce/0x120 Mar 19 20:35:07 hn kernel: [8110bc24] __alloc_pages_nodemask+0x614/0x850 Mar 19 20:35:07 hn kernel: [8140da5f] ? megasas_fire_cmd_gen2+0x3f/0x50 Mar 19 20:35:07 hn kernel: [8114783c] alloc_pages_current+0x8c/0x100 Mar 19 20:35:07 hn kernel: [81108eac] alloc_kmem_pages+0x2c/0xd0 Mar 19 20:35:07 hn kernel: [811227f3] kmalloc_order+0x13/0x50 Mar 19 20:35:07 hn kernel: [8112284f] kmalloc_order_trace+0x1f/0x90 Mar 19 20:35:07 hn kernel: [811511b9] __kmalloc_track_caller+0x159/0x250 Mar 19 20:35:07 hn kernel: [8111ce1b] kmemdup+0x1b/0x40 Mar 19 20:35:07 hn kernel: [a0004aab] __kvm_set_memory_region+0x1eb/0x950 [kvm] Mar 19 20:35:07 hn kernel: [8115aca0] ? mem_cgroup_uncharge_swap+0x20/0xa0 Mar 19 20:35:07 hn kernel: [8113e3a8] ? swap_entry_free+0x198/0x310 Mar 19 20:35:07 hn kernel: [a000523a] kvm_set_memory_region+0x2a/0x50 [kvm] Mar 19 20:35:07 hn kernel: [a000577f] kvm_vm_ioctl+0x51f/0x760 [kvm] Mar 19 20:35:07 hn kernel: [81170d30] do_vfs_ioctl+0x2e0/0x4e0 Mar 19 20:35:07 hn kernel: [8107666f] ? vtime_account_user+0x3f/0x60 Mar 19 20:35:07 hn kernel: [81170fb1] SyS_ioctl+0x81/0xa0 Mar 19 20:35:07 hn kernel: [81719ff2] system_call_fastpath+0x12/0x17 Is anyone working on this? thanks and best regards t. -- You are receiving this mail because: You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 93251] qemu-kvm guests randomly hangs after reboot command in guest
https://bugzilla.kernel.org/show_bug.cgi?id=93251 Bandan Das b...@makefile.in changed: What|Removed |Added CC||b...@makefile.in --- Comment #4 from Bandan Das b...@makefile.in --- Doe(In reply to Thomas Stein from comment #3) ... Is anyone working on this? thanks and best regards t. Does reverting the commit suggested in comment 2 work for you ? -- You are receiving this mail because: You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 93251] qemu-kvm guests randomly hangs after reboot command in guest
https://bugzilla.kernel.org/show_bug.cgi?id=93251 Marcelo Tosatti mtosa...@redhat.com changed: What|Removed |Added CC||mtosa...@redhat.com --- Comment #2 from Marcelo Tosatti mtosa...@redhat.com --- (In reply to Ondřej Súkup from comment #1) I run about 7 nodes, an simulate build of openstack cloud in qemu-kvm env .. nodes randomly after reboot stop working, or hangs in few sec after reboot from journalctl: úno 13 18:10:22 argus kernel: qemu-system-x86: page allocation failure: order:4, mode:0x40d0 úno 13 18:10:23 argus kernel: CPU: 2 PID: 2731 Comm: qemu-system-x86 Not tainted 3.19.0-gentoo #2 úno 13 18:10:27 argus kernel: Hardware name: Dell Inc. OptiPlex 9010/0M9KCM, BIOS A19 08/18/2014 úno 13 18:10:27 argus kernel: 0004 a9baffad 9d5cdf0c úno 13 18:10:27 argus kernel: 40d0 9d0d5f1e 0040 úno 13 18:10:27 argus kernel: 9d807080 9d0d8dda 9d806180 a9baffad úno 13 18:10:27 argus kernel: Call Trace: úno 13 18:10:27 argus kernel: [9d5cdf0c] ? dump_stack+0x47/0x67 úno 13 18:10:27 argus kernel: [9d0d5f1e] ? warn_alloc_failed+0xee/0x150 úno 13 18:10:27 argus kernel: [9d0d8dda] ? __alloc_pages_direct_compact+0x14a/0x150 úno 13 18:10:27 argus kernel: [9d0d9556] ? __alloc_pages_nodemask+0x776/0xb80 úno 13 18:10:27 argus kernel: [9d0d9a3a] ? alloc_kmem_pages+0x3a/0x110 úno 13 18:10:27 argus kernel: [9d0ee4e3] ? kmalloc_order+0x13/0x50 úno 13 18:10:27 argus kernel: [9d0e954b] ? kmemdup+0x1b/0x40 úno 13 18:10:27 argus kernel: [c0aeb1fa] ? __kvm_set_memory_region+0x24a/0x9f0 [kvm] úno 13 18:10:27 argus kernel: [c0b154b0] ? kvm_set_ioapic+0x130/0x130 [kvm] úno 13 18:10:27 argus kernel: [c0aeb9c1] ? kvm_set_memory_region+0x21/0x40 [kvm] úno 13 18:10:27 argus kernel: [c0aebe1f] ? kvm_vm_ioctl+0x43f/0x750 [kvm] úno 13 18:10:27 argus kernel: [9d2ef4eb] ? blk_finish_plug+0xb/0x40 úno 13 18:10:27 argus kernel: [9d10ae75] ? SyS_madvise+0x3c5/0x800 úno 13 18:10:27 argus kernel: [9d07ca71] ? cpuacct_account_field+0x11/0x60 úno 13 18:10:27 argus kernel: [9d147748] ? do_vfs_ioctl+0x308/0x510 úno 13 18:10:27 argus kernel: [9d06ce3f] ? vtime_account_user+0x3f/0x60 úno 13 18:10:27 argus kernel: [9d151423] ? __fget+0x63/0xa0 úno 13 18:10:27 argus kernel: [9d147986] ? SyS_ioctl+0x36/0x80 úno 13 18:10:27 argus kernel: [9d5d453d] ? int_check_syscall_exit_work+0x34/0x3d úno 13 18:10:27 argus kernel: [9d5d4312] ? system_call_fastpath+0x12/0x17 úno 13 18:10:27 argus kernel: Mem-Info: úno 13 18:10:27 argus kernel: DMA per-cpu: úno 13 18:10:27 argus kernel: CPU0: hi:0, btch: 1 usd: 0 úno 13 18:10:27 argus kernel: CPU1: hi:0, btch: 1 usd: 0 úno 13 18:10:27 argus kernel: CPU2: hi:0, btch: 1 usd: 0 úno 13 18:10:27 argus kernel: CPU3: hi:0, btch: 1 usd: 0 úno 13 18:10:27 argus kernel: CPU4: hi:0, btch: 1 usd: 0 úno 13 18:10:27 argus kernel: CPU5: hi:0, btch: 1 usd: 0 úno 13 18:10:27 argus kernel: CPU6: hi:0, btch: 1 usd: 0 úno 13 18:10:27 argus kernel: CPU7: hi:0, btch: 1 usd: 0 úno 13 18:10:27 argus kernel: DMA32 per-cpu: úno 13 18:10:27 argus kernel: CPU0: hi: 186, btch: 31 usd: 0 úno 13 18:10:27 argus kernel: CPU1: hi: 186, btch: 31 usd: 0 úno 13 18:10:27 argus kernel: CPU2: hi: 186, btch: 31 usd: 22 úno 13 18:10:27 argus kernel: CPU3: hi: 186, btch: 31 usd: 170 úno 13 18:10:27 argus kernel: CPU4: hi: 186, btch: 31 usd: 0 úno 13 18:10:27 argus kernel: CPU5: hi: 186, btch: 31 usd: 0 úno 13 18:10:27 argus kernel: CPU6: hi: 186, btch: 31 usd: 0 úno 13 18:10:27 argus kernel: CPU7: hi: 186, btch: 31 usd: 0 úno 13 18:10:27 argus kernel: Normal per-cpu: úno 13 18:10:27 argus kernel: CPU0: hi: 186, btch: 31 usd: 0 úno 13 18:10:27 argus kernel: CPU1: hi: 186, btch: 31 usd: 30 úno 13 18:10:27 argus kernel: CPU2: hi: 186, btch: 31 usd: 0 úno 13 18:10:27 argus kernel: CPU3: hi: 186, btch: 31 usd: 161 úno 13 18:10:27 argus kernel: CPU4: hi: 186, btch: 31 usd: 0 úno 13 18:10:27 argus kernel: CPU5: hi: 186, btch: 31 usd: 0 úno 13 18:10:27 argus kernel: CPU6: hi: 186, btch: 31 usd: 0 úno 13 18:10:27 argus kernel: CPU7: hi: 186, btch: 31 usd: 0 úno 13 18:10:27 argus kernel: active_anon:2109969 inactive_anon:930082 isolated_anon:0 active_file:318968 inactive_file:466719 isolated_file:0 unevictable:0 dirty:134084 writeback:3211
[Bug 93251] New: qemu-kvm guests randomly hangs after reboot command in guest
https://bugzilla.kernel.org/show_bug.cgi?id=93251 Bug ID: 93251 Summary: qemu-kvm guests randomly hangs after reboot command in guest Product: Virtualization Version: unspecified Kernel Version: 3.19.0 Hardware: Intel OS: Linux Tree: Mainline Status: NEW Severity: normal Priority: P1 Component: kvm Assignee: virtualization_...@kernel-bugs.osdl.org Reporter: mimi...@gmail.com Regression: No -- You are receiving this mail because: You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 93251] qemu-kvm guests randomly hangs after reboot command in guest
https://bugzilla.kernel.org/show_bug.cgi?id=93251 --- Comment #1 from Ondřej Súkup mimi...@gmail.com --- I run about 7 nodes, an simulate build of openstack cloud in qemu-kvm env .. nodes randomly after reboot stop working, or hangs in few sec after reboot from journalctl: úno 13 18:10:22 argus kernel: qemu-system-x86: page allocation failure: order:4, mode:0x40d0 úno 13 18:10:23 argus kernel: CPU: 2 PID: 2731 Comm: qemu-system-x86 Not tainted 3.19.0-gentoo #2 úno 13 18:10:27 argus kernel: Hardware name: Dell Inc. OptiPlex 9010/0M9KCM, BIOS A19 08/18/2014 úno 13 18:10:27 argus kernel: 0004 a9baffad 9d5cdf0c úno 13 18:10:27 argus kernel: 40d0 9d0d5f1e 0040 úno 13 18:10:27 argus kernel: 9d807080 9d0d8dda 9d806180 a9baffad úno 13 18:10:27 argus kernel: Call Trace: úno 13 18:10:27 argus kernel: [9d5cdf0c] ? dump_stack+0x47/0x67 úno 13 18:10:27 argus kernel: [9d0d5f1e] ? warn_alloc_failed+0xee/0x150 úno 13 18:10:27 argus kernel: [9d0d8dda] ? __alloc_pages_direct_compact+0x14a/0x150 úno 13 18:10:27 argus kernel: [9d0d9556] ? __alloc_pages_nodemask+0x776/0xb80 úno 13 18:10:27 argus kernel: [9d0d9a3a] ? alloc_kmem_pages+0x3a/0x110 úno 13 18:10:27 argus kernel: [9d0ee4e3] ? kmalloc_order+0x13/0x50 úno 13 18:10:27 argus kernel: [9d0e954b] ? kmemdup+0x1b/0x40 úno 13 18:10:27 argus kernel: [c0aeb1fa] ? __kvm_set_memory_region+0x24a/0x9f0 [kvm] úno 13 18:10:27 argus kernel: [c0b154b0] ? kvm_set_ioapic+0x130/0x130 [kvm] úno 13 18:10:27 argus kernel: [c0aeb9c1] ? kvm_set_memory_region+0x21/0x40 [kvm] úno 13 18:10:27 argus kernel: [c0aebe1f] ? kvm_vm_ioctl+0x43f/0x750 [kvm] úno 13 18:10:27 argus kernel: [9d2ef4eb] ? blk_finish_plug+0xb/0x40 úno 13 18:10:27 argus kernel: [9d10ae75] ? SyS_madvise+0x3c5/0x800 úno 13 18:10:27 argus kernel: [9d07ca71] ? cpuacct_account_field+0x11/0x60 úno 13 18:10:27 argus kernel: [9d147748] ? do_vfs_ioctl+0x308/0x510 úno 13 18:10:27 argus kernel: [9d06ce3f] ? vtime_account_user+0x3f/0x60 úno 13 18:10:27 argus kernel: [9d151423] ? __fget+0x63/0xa0 úno 13 18:10:27 argus kernel: [9d147986] ? SyS_ioctl+0x36/0x80 úno 13 18:10:27 argus kernel: [9d5d453d] ? int_check_syscall_exit_work+0x34/0x3d úno 13 18:10:27 argus kernel: [9d5d4312] ? system_call_fastpath+0x12/0x17 úno 13 18:10:27 argus kernel: Mem-Info: úno 13 18:10:27 argus kernel: DMA per-cpu: úno 13 18:10:27 argus kernel: CPU0: hi:0, btch: 1 usd: 0 úno 13 18:10:27 argus kernel: CPU1: hi:0, btch: 1 usd: 0 úno 13 18:10:27 argus kernel: CPU2: hi:0, btch: 1 usd: 0 úno 13 18:10:27 argus kernel: CPU3: hi:0, btch: 1 usd: 0 úno 13 18:10:27 argus kernel: CPU4: hi:0, btch: 1 usd: 0 úno 13 18:10:27 argus kernel: CPU5: hi:0, btch: 1 usd: 0 úno 13 18:10:27 argus kernel: CPU6: hi:0, btch: 1 usd: 0 úno 13 18:10:27 argus kernel: CPU7: hi:0, btch: 1 usd: 0 úno 13 18:10:27 argus kernel: DMA32 per-cpu: úno 13 18:10:27 argus kernel: CPU0: hi: 186, btch: 31 usd: 0 úno 13 18:10:27 argus kernel: CPU1: hi: 186, btch: 31 usd: 0 úno 13 18:10:27 argus kernel: CPU2: hi: 186, btch: 31 usd: 22 úno 13 18:10:27 argus kernel: CPU3: hi: 186, btch: 31 usd: 170 úno 13 18:10:27 argus kernel: CPU4: hi: 186, btch: 31 usd: 0 úno 13 18:10:27 argus kernel: CPU5: hi: 186, btch: 31 usd: 0 úno 13 18:10:27 argus kernel: CPU6: hi: 186, btch: 31 usd: 0 úno 13 18:10:27 argus kernel: CPU7: hi: 186, btch: 31 usd: 0 úno 13 18:10:27 argus kernel: Normal per-cpu: úno 13 18:10:27 argus kernel: CPU0: hi: 186, btch: 31 usd: 0 úno 13 18:10:27 argus kernel: CPU1: hi: 186, btch: 31 usd: 30 úno 13 18:10:27 argus kernel: CPU2: hi: 186, btch: 31 usd: 0 úno 13 18:10:27 argus kernel: CPU3: hi: 186, btch: 31 usd: 161 úno 13 18:10:27 argus kernel: CPU4: hi: 186, btch: 31 usd: 0 úno 13 18:10:27 argus kernel: CPU5: hi: 186, btch: 31 usd: 0 úno 13 18:10:27 argus kernel: CPU6: hi: 186, btch: 31 usd: 0 úno 13 18:10:27 argus kernel: CPU7: hi: 186, btch: 31 usd: 0 úno 13 18:10:27 argus kernel: active_anon:2109969 inactive_anon:930082 isolated_anon:0 active_file:318968 inactive_file:466719 isolated_file:0 unevictable:0 dirty:134084 writeback:3211 unstable:0 free:47211 slab_reclaimable:45577 slab_unreclaimable:74482 mapped:47028 shmem:382256 pagetables:23493 bounce:0 free_cma:43 úno 13 18:10:27 argus kernel: DMA free:15884kB min:64kB low:80kB high:96kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
Re: 3.13 hangs when I tried to start a KVM at a 32 bit stable Gentoo
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 01/23/2014 07:55 PM, Dave Hansen wrote: On 01/21/2014 08:38 AM, Toralf Förster wrote: Jan 21 17:18:57 n22 kernel: INFO: rcu_sched self-detected stall on CPU { 2} (t=60001 jiffies g=18494 c=18493 q=183951) Jan 21 17:18:57 n22 kernel: sending NMI to all CPUs: Jan 21 17:18:57 n22 kernel: NMI backtrace for cpu 2 Jan 21 17:18:57 n22 kernel: CPU: 2 PID: 6779 Comm: qemu-system-x86 Not tainted 3.13.0 #3 Jan 21 17:18:57 n22 kernel: Hardware name: LENOVO 4180F65/4180F65, BIOS 83ET75WW (1.45 ) 05/10/2013 Jan 21 17:18:57 n22 kernel: task: e921c370 ti: e5f36000 task.ti: e5f36000 I'm seeing a very similar hang with an ubuntu guest and a custom kernel. I'm on commit 0dc3fd0249a, and it's 100% reproducible every time I run KVM. Cc-ing a few more folks... I tried to bisect it - the first attempt to bisetc it between v3.12. and v3.13 blamed this commit : commit 37bf06375c90a42fe07b9bebdb07bc316ae5a0ce Merge: 6bfa687 d0e639c Author: Ingo Molnar mi...@kernel.org Date: Wed Oct 9 12:36:13 2013 +0200 Merge tag 'v3.12-rc4' into sched/core Merge Linux v3.12-rc4 to fix a conflict and also to refresh the tree before applying more scheduler patches. Conflicts: arch/avr32/include/asm/Kbuild Signed-off-by: Ingo Molnar mi...@kernel.org which seems rather to be the upper limit where to search for the first bad commit, or ? - -- MfG/Sincerely Toralf Förster pgp finger print:1A37 6F99 4A9D 026F 13E2 4DCF C4EA CDDE 0076 E94E -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iF4EAREIAAYFAlMLregACgkQxOrN3gB26U7P7gD/Wage68loQ++oP4moD94kTNpn goyT7fvssm3jyF+2HagA/0EIR5MOcl7xlBPNw5JKqcI0pPx0yBytUljtz+KTPS/Y =P8vB -END PGP SIGNATURE- -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 3.13 hangs when I tried to start a KVM at a 32 bit stable Gentoo
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 01/23/2014 07:55 PM, Dave Hansen wrote: On 01/21/2014 08:38 AM, Toralf Förster wrote: Jan 21 17:18:57 n22 kernel: INFO: rcu_sched self-detected stall on CPU { 2} (t=60001 jiffies g=18494 c=18493 q=183951) Jan 21 17:18:57 n22 kernel: sending NMI to all CPUs: Jan 21 17:18:57 n22 kernel: NMI backtrace for cpu 2 Jan 21 17:18:57 n22 kernel: CPU: 2 PID: 6779 Comm: qemu-system-x86 Not tainted 3.13.0 #3 Jan 21 17:18:57 n22 kernel: Hardware name: LENOVO 4180F65/4180F65, BIOS 83ET75WW (1.45 ) 05/10/2013 Jan 21 17:18:57 n22 kernel: task: e921c370 ti: e5f36000 task.ti: e5f36000 I'm seeing a very similar hang with an ubuntu guest and a custom kernel. I'm on commit 0dc3fd0249a, and it's 100% reproducible every time I run KVM. Cc-ing a few more folks... With 3.12.2 at the same 32 bit Gentoo Linux I experienced a similar thingin a complete different scenario: I umounted a loop-mounted file, on which I created and filled a btrfs file system before (/usr/portage FWIW - I'm a Gentoo user) and the systems becomes very slowly and /var/log/messages shows : Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: Error sending POWER_TABLE_CMD: time out after 2000ms. Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: Current CMD queue read_ptr 100 write_ptr 101 Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: Loaded firmware version: 18.168.6.1 Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | OK Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | uPc Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | branchlink1 Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | branchlink2 Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | interruptlink1 Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | interruptlink2 Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | data1 Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | data2 Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | line Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | beacon time Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | tsf low Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | tsf hi Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | time gp1 Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | time gp2 Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | time gp3 Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | uCode version Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | hw version Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | board version Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | hcmd Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | isr0 Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | isr1 Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | isr2 Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | isr3 Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | isr4 Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | isr_pref Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | wait_event Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | l2p_control Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | l2p_duration Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | l2p_mhvalid Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | l2p_addr_match Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | lmpm_pmg_sel Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | timestamp Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: 0x | flow_handler Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: Start IWL Event Log Dump: nothing in log Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: set power fail, ret = -110 Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: iwl_trans_wait_tx_queue_empty bad state = 0 Feb 7 22:46:46 n22 kernel: ieee80211 phy0: Hardware restart was requested Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: L1 Enabled; Disabling L0S Feb 7 22:46:46 n22 kernel: iwlwifi :03:00.0: Radio type=0x1-0x2-0x0 Feb 7 22:46:51 n22 kernel: iwlwifi :03:00.0: Failed to load firmware chunk! Feb 7 22:46:51 n22 kernel: iwlwifi :03:00.0: Could not load the [0] uCode section Feb 7 22:46:51 n22 kernel: iwlwifi :03:00.0: Failed to start RT ucode: -110 Feb 7 22:46:51 n22 kernel: iwlwifi :03:00.0: Unable to initialize device. Feb 7 22:46:51 n22 kernel: iwlwifi :03:00.0: Fw not loaded - dropping CMD: 18 Feb 7 22:46:51 n22 kernel: wlp3s0: HW problem - can not stop rx aggregation for 08:96:d7:05:f9:2a tid 0 Feb 7 22:46:51 n22 kernel: iwlwifi :03:00.0: Fw not loaded - dropping CMD: 18 Feb 7 22:46:51 n22 kernel: wlp3s0: HW problem - can not stop rx aggregation for 08:96:d7:05:f9:2a
Re: 3.13 hangs when I tried to start a KVM at a 32 bit stable Gentoo
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 01/24/2014 01:36 PM, Paolo Bonzini wrote: Il 23/01/2014 20:50, Toralf Förster ha scritto: | What makes the situation really annyoing - sometimes I just can | restart my wlan device it the system works normal, but sometimes | the whole system hangs and for those cases then sometimes not even | sysrq buttons do work. Can you reproduce it with the wlan driver disabled completely? yes - with CONFIG_WLAN=n I do get a similar thing : Jan 26 11:23:17 n22 kernel: NET: Registered protocol family 17 Jan 26 11:23:17 n22 kernel: device vnet0 entered promiscuous mode Jan 26 11:23:17 n22 kernel: br0: port 4(vnet0) entered forwarding state Jan 26 11:23:17 n22 kernel: br0: port 4(vnet0) entered forwarding state Jan 26 11:23:17 n22 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): br0: link becomes ready Jan 26 11:23:17 n22 kernel: cgroup: libvirtd (6113) created nested cgroup for controller memory which has incomplete hierarchy support. Nested cgroups may change behavior in the future. Jan 26 11:23:17 n22 kernel: cgroup: memory requires setting use_hierarchy to 1 on the root. Jan 26 11:23:19 n22 ntpd[6015]: Listen normally on 5 br0 fe80::837:18ff:feae:caf0 UDP 123 Jan 26 11:23:19 n22 ntpd[6015]: Listen normally on 6 vnet0 fe80::fc54:ff:fed0:f1eb UDP 123 Jan 26 11:23:19 n22 ntpd[6015]: peers refreshed Jan 26 11:23:35 n22 kernel: br0: port 4(vnet0) entered disabled state Jan 26 11:23:35 n22 kernel: device vnet0 left promiscuous mode Jan 26 11:23:35 n22 kernel: br0: port 4(vnet0) entered disabled state Jan 26 11:23:38 n22 ntpd[6015]: Deleting interface #6 vnet0, fe80::fc54:ff:fed0:f1eb#123, interface stats: received=0, sent=0, dropped=0, active_time=19 secs Jan 26 11:23:38 n22 ntpd[6015]: peers refreshed Jan 26 11:23:45 n22 kernel: device vnet0 entered promiscuous mode Jan 26 11:23:45 n22 kernel: br0: port 4(vnet0) entered forwarding state Jan 26 11:23:45 n22 kernel: br0: port 4(vnet0) entered forwarding state Jan 26 11:23:48 n22 ntpd[6015]: Listen normally on 7 vnet0 fe80::fc54:ff:fed0:f1eb UDP 123 Jan 26 11:23:48 n22 ntpd[6015]: peers refreshed Jan 26 11:24:09 n22 kernel: br0: port 4(vnet0) entered disabled state Jan 26 11:24:09 n22 kernel: device vnet0 left promiscuous mode Jan 26 11:24:09 n22 kernel: br0: port 4(vnet0) entered disabled state Jan 26 11:24:11 n22 ntpd[6015]: Deleting interface #7 vnet0, fe80::fc54:ff:fed0:f1eb#123, interface stats: received=0, sent=0, dropped=0, active_time=23 secs Jan 26 11:24:11 n22 ntpd[6015]: peers refreshed Jan 26 11:24:28 n22 kernel: device vnet0 entered promiscuous mode Jan 26 11:24:28 n22 kernel: br0: port 4(vnet0) entered forwarding state Jan 26 11:24:28 n22 kernel: br0: port 4(vnet0) entered forwarding state Jan 26 11:24:31 n22 ntpd[6015]: Listen normally on 8 vnet0 fe80::fc54:ff:fed0:f1eb UDP 123 Jan 26 11:24:31 n22 ntpd[6015]: peers refreshed Jan 26 11:24:57 n22 kernel: hda-intel: IRQ timing workaround is activated for card #0. Suggest a bigger bdl_pos_adj. Jan 26 11:25:34 n22 kernel: INFO: rcu_sched self-detected stall on CPU { 3} (t=6 jiffies g=23149 c=23148 q=7132) Jan 26 11:25:34 n22 kernel: sending NMI to all CPUs: Jan 26 11:25:34 n22 kernel: NMI backtrace for cpu 3 Jan 26 11:25:34 n22 kernel: CPU: 3 PID: 6523 Comm: qemu-system-x86 Not tainted 3.13.0 #5 Jan 26 11:25:34 n22 kernel: Hardware name: LENOVO 4180F65/4180F65, BIOS 83ET75WW (1.45 ) 05/10/2013 Jan 26 11:25:34 n22 kernel: task: e410c8a0 ti: eec6 task.ti: eec6 Jan 26 11:25:34 n22 kernel: EIP: 0060:[c1255bed] EFLAGS: 0006 CPU: 3 Jan 26 11:25:34 n22 kernel: EIP is at __const_udelay+0xd/0x20 Jan 26 11:25:34 n22 kernel: EAX: 01062560 EBX: 2710 ECX: c1557e60 EDX: 00278af4 Jan 26 11:25:34 n22 kernel: ESI: c155c500 EDI: f3627c80 EBP: eec61c64 ESP: eec61c64 Jan 26 11:25:34 n22 kernel: DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068 Jan 26 11:25:34 n22 kernel: CR0: 80050033 CR2: CR3: 23c2e000 CR4: 000427f0 Jan 26 11:25:34 n22 kernel: Stack: Jan 26 11:25:34 n22 kernel: eec61c74 c102dc75 c14b9db5 c155c500 eec61cb8 c108ba1c c14c3ac0 ea60 Jan 26 11:25:34 n22 kernel: 5a6d 5a6c 1bdc c106aadd 0001 c155c500 c1595544 f3627c80 Jan 26 11:25:34 n22 kernel: 1bdc 0003 e410c8a0 0003 eec61ccc c104a2fb f3627ec0 Jan 26 11:25:34 n22 kernel: Call Trace: Jan 26 11:25:34 n22 kernel: [c102dc75] arch_trigger_all_cpu_backtrace+0x55/0x70 Jan 26 11:25:34 n22 kernel: [c108ba1c] rcu_check_callbacks+0x38c/0x590 Jan 26 11:25:34 n22 kernel: [c106aadd] ? account_system_time+0xbd/0x170 Jan 26 11:25:34 n22 kernel: [c104a2fb] update_process_times+0x3b/0x70 Jan 26 11:25:34 n22 kernel: [c1095983] tick_sched_handle.isra.12+0x33/0x40 Jan 26 11:25:34 n22 kernel: [c1095b40] tick_sched_timer+0x40/0x70 Jan 26 11:25:34 n22 kernel: [c105dad0] ? __remove_hrtimer+0x40/0xa0 Jan 26 11:25:34 n22 kernel: [c105dd89] __run_hrtimer+0x69/0x190 Jan 26 11:25:34 n22 kernel: [c1095b00] ? tick_sched_do_timer+0x40/0x40 Jan 26 11:25:34 n22 kernel
Re: 3.13 hangs when I tried to start a KVM at a 32 bit stable Gentoo
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 01/24/2014 01:36 PM, Paolo Bonzini wrote: Il 23/01/2014 20:50, Toralf Förster ha scritto: | What makes the situation really annyoing - sometimes I just can | restart my wlan device it the system works normal, but sometimes | the whole system hangs and for those cases then sometimes not even | sysrq buttons do work. Can you reproduce it with the wlan driver disabled completely? yes - root cause is not the wlan - that's just a victim. Paolo - -- MfG/Sincerely Toralf Förster pgp finger print:1A37 6F99 4A9D 026F 13E2 4DCF C4EA CDDE 0076 E94E -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iF4EAREIAAYFAlLjiF0ACgkQxOrN3gB26U64DwD/c60zqwUORstYkSD2I1AarWLO /4QVwWo8hW8bUg6f3SEA/2Wv7jkJTtkfTUVEyZZxeEJaNP+RChsmGSoU77Dl1G7T =VMnB -END PGP SIGNATURE- -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 3.13 hangs when I tried to start a KVM at a 32 bit stable Gentoo
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Il 23/01/2014 20:50, Toralf Förster ha scritto: | What makes the situation really annyoing - sometimes I just can | restart my wlan device it the system works normal, but sometimes | the whole system hangs and for those cases then sometimes not even | sysrq buttons do work. Can you reproduce it with the wlan driver disabled completely? Paolo -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJS4l5cAAoJEBvWZb6bTYbyC0AP/jrHp8WUqDjgv/1kJYcYb0+0 cSM8jqcutYCcMAKXTsKGgQj4gWdw7q/bEo4lWAEX8fQqxOfFZjBZjhTuA66XF010 fi+nOCEwjafMUeflPZaSZr34KGSix1FTtmcdzz55Kzkh+A4hZJj2NgaAYr8KFsWW 0hCoW7+MHWTRsNLIZk1ms2H/lItz19gnO9CkNYWqSIxBtx1/le5v8LNi20OkHKCz mdemG6DSwj9sE+jfezDjZ7jV/a+RBv4Pi0v71YPEUEa7HCiyKxddVAgD0KrvMwzF 2A7H4+rTwsJLnWTKFyMdoeUUz6iC5Rntq+B4ltHVXKjN2O5bG0aqmG+cURbvXrFF y9EzChdaEikPlYJHp/ci2NOB8NJlB6oEb9uO3k2kRITdWsr3iK5gCZE3fu99ziz4 JmNPUpFGTy5dnQVI/edsuZDyafRYVq3U4CpP+drnWxbic4aH7nCTv2iR5Op4pnhd LQlmt0MkZAyF9+6XFNBAAfPnfV0YPBHydCxhC2UaOmqqNXiYILW65dCD5aQiXq1y lQi2dc6i7MYbVmc28I45LV/BFD4zn622c5ges0gKw2bshgzhQm12jWHJ+dMsLDp/ qpFFvaGoqWCNJZo9FCH6vaTiN5KNUyiUOZjvKVa3u959xC9BK8nh3vTWHxinK2Yt Do62ZMYWJbljdgTvMN7d =cfJ7 -END PGP SIGNATURE- -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 3.13 hangs when I tried to start a KVM at a 32 bit stable Gentoo
On 01/21/2014 08:38 AM, Toralf Förster wrote: Jan 21 17:18:57 n22 kernel: INFO: rcu_sched self-detected stall on CPU { 2} (t=60001 jiffies g=18494 c=18493 q=183951) Jan 21 17:18:57 n22 kernel: sending NMI to all CPUs: Jan 21 17:18:57 n22 kernel: NMI backtrace for cpu 2 Jan 21 17:18:57 n22 kernel: CPU: 2 PID: 6779 Comm: qemu-system-x86 Not tainted 3.13.0 #3 Jan 21 17:18:57 n22 kernel: Hardware name: LENOVO 4180F65/4180F65, BIOS 83ET75WW (1.45 ) 05/10/2013 Jan 21 17:18:57 n22 kernel: task: e921c370 ti: e5f36000 task.ti: e5f36000 I'm seeing a very similar hang with an ubuntu guest and a custom kernel. I'm on commit 0dc3fd0249a, and it's 100% reproducible every time I run KVM. Cc-ing a few more folks... -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 3.13 hangs when I tried to start a KVM at a 32 bit stable Gentoo
On 01/23/2014 10:55 AM, Dave Hansen wrote: On 01/21/2014 08:38 AM, Toralf Förster wrote: Jan 21 17:18:57 n22 kernel: INFO: rcu_sched self-detected stall on CPU { 2} (t=60001 jiffies g=18494 c=18493 q=183951) Jan 21 17:18:57 n22 kernel: sending NMI to all CPUs: Jan 21 17:18:57 n22 kernel: NMI backtrace for cpu 2 Jan 21 17:18:57 n22 kernel: CPU: 2 PID: 6779 Comm: qemu-system-x86 Not tainted 3.13.0 #3 Jan 21 17:18:57 n22 kernel: Hardware name: LENOVO 4180F65/4180F65, BIOS 83ET75WW (1.45 ) 05/10/2013 Jan 21 17:18:57 n22 kernel: task: e921c370 ti: e5f36000 task.ti: e5f36000 I'm seeing a very similar hang with an ubuntu guest and a custom kernel. I'm on commit 0dc3fd0249a, and it's 100% reproducible every time I run KVM. Did a little more LKML digging and found this: http://marc.info/?l=linux-kernelm=139038631607917q=raw Peter's fix works for me. I'm also running a CONFIG_PREEMPT_VOLUNTARY=y config. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 3.13 hangs when I tried to start a KVM at a 32 bit stable Gentoo
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 01/23/2014 08:33 PM, Dave Hansen wrote: On 01/23/2014 10:55 AM, Dave Hansen wrote: On 01/21/2014 08:38 AM, Toralf Förster wrote: Jan 21 17:18:57 n22 kernel: INFO: rcu_sched self-detected stall on CPU { 2} (t=60001 jiffies g=18494 c=18493 q=183951) Jan 21 17:18:57 n22 kernel: sending NMI to all CPUs: Jan 21 17:18:57 n22 kernel: NMI backtrace for cpu 2 Jan 21 17:18:57 n22 kernel: CPU: 2 PID: 6779 Comm: qemu-system-x86 Not tainted 3.13.0 #3 Jan 21 17:18:57 n22 kernel: Hardware name: LENOVO 4180F65/4180F65, BIOS 83ET75WW (1.45 ) 05/10/2013 Jan 21 17:18:57 n22 kernel: task: e921c370 ti: e5f36000 task.ti: e5f36000 I'm seeing a very similar hang with an ubuntu guest and a custom kernel. I'm on commit 0dc3fd0249a, and it's 100% reproducible every time I run KVM. Did a little more LKML digging and found this: http://marc.info/?l=linux-kernelm=139038631607917q=raw Hhm, that patch cannot be applied to 3.13.0 kernel :-( Peter's fix works for me. I'm also running a CONFIG_PREEMPT_VOLUNTARY=y config. I do have : n22 /usr/src/linux # zgrep CONFIG_PREEMPT /proc/config.gz # CONFIG_PREEMPT_RCU is not set CONFIG_PREEMPT_NOTIFIERS=y # CONFIG_PREEMPT_NONE is not set CONFIG_PREEMPT_VOLUNTARY=y # CONFIG_PREEMPT is not set What makes the situation really annyoing - sometimes I just can restart my wlan device it the system works normal, but sometimes the whole system hangs and for those cases then sometimes not even sysrq buttons do work. What I see in /var/log/messages are those lines : Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: Error sending POWER_TABLE_CMD: time out after 2000ms. Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: Current CMD queue read_ptr 158 write_ptr 159 Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: Loaded firmware version: 18.168.6.1 Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | OK Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | uPc Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | branchlink1 Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | branchlink2 Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | interruptlink1 Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | interruptlink2 Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | data1 Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | data2 Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | line Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | beacon time Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | tsf low Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | tsf hi Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | time gp1 Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | time gp2 Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | time gp3 Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | uCode version Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | hw version Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | board version Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | hcmd Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | isr0 Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | isr1 Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | isr2 Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | isr3 Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | isr4 Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | isr_pref Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | wait_event Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | l2p_control Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | l2p_duration Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | l2p_mhvalid Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | l2p_addr_match Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | lmpm_pmg_sel Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | timestamp Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: 0x | flow_handler Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: Start IWL Event Log Dump: nothing in log Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: set power fail, ret = -110 Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: iwl_trans_wait_tx_queue_empty bad state = 0 Jan 23 19:23:15 n22 kernel: ieee80211 phy0: Hardware restart was requested Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: L1 Enabled; Disabling L0S Jan 23 19:23:15 n22 kernel: iwlwifi :03:00.0: Radio type=0x1-0x2-0x0 Jan 23 19:23:20 n22 kernel: iwlwifi :03:00.0: Failed to load firmware chunk! Jan 23 19:23:20 n22 kernel: iwlwifi :03:00.0: Could not load the [0] uCode section Jan 23 19:23:20 n22 kernel: iwlwifi :03:00.0
[Bug 63291] KVM USB passthrough to Windows 7 guest fails with error -110, hangs
https://bugzilla.kernel.org/show_bug.cgi?id=63291 Greg Sheremeta g...@gregsheremeta.com changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |MOVED --- Comment #2 from Greg Sheremeta g...@gregsheremeta.com --- done, https://bugs.launchpad.net/qemu/+bug/1242765 -- You are receiving this mail because: You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 63291] New: KVM USB passthrough to Windows 7 guest fails with error -110, hangs
https://bugzilla.kernel.org/show_bug.cgi?id=63291 Bug ID: 63291 Summary: KVM USB passthrough to Windows 7 guest fails with error -110, hangs Product: Virtualization Version: unspecified Kernel Version: 3.11.1-200.fc19 Hardware: x86-64 OS: Linux Tree: Mainline Status: NEW Severity: normal Priority: P1 Component: kvm Assignee: virtualization_...@kernel-bugs.osdl.org Reporter: g...@gregsheremeta.com Regression: No Description of problem: Using a Sandisk Cruzer Fit 16GB USB thumb drive. Using virt-manager on Fedora 19 host, and Windows 7 32 bit guest. I set up a USB2 controller on Windows 7 guest in virt-manager. Windows sees the USB drive and can open the file manager and correctly show the files. I can copy a file from the thumb drive to the Fedora desktop, and then play the file on the desktop. However, any attempt to open a file directly on the thumb drive (example, play an MP3 using Windows Media Player) results in guest hang and kernel messages: Oct 19 21:15:35 localhost kernel: [187592.977839] usb 1-1.3: reset high-speed USB device number 13 using ehci-pci Oct 19 21:15:40 localhost kernel: [187598.065274] usb 1-1.3: device descriptor read/all, error -110 Oct 19 21:15:40 localhost kernel: [187598.138167] usb 1-1.3: reset high-speed USB device number 13 using ehci-pci Oct 19 21:15:56 localhost kernel: [187613.218119] usb 1-1.3: device descriptor read/64, error -110 Oct 19 21:16:11 localhost kernel: [187628.399275] usb 1-1.3: device descriptor read/64, error -110 Oct 19 21:16:11 localhost kernel: [187628.573355] usb 1-1.3: reset high-speed USB device number 13 using ehci-pci Oct 19 21:16:16 localhost kernel: [187633.587778] usb 1-1.3: device descriptor read/8, error -110 Oct 19 21:16:21 localhost kernel: [187638.702244] usb 1-1.3: device descriptor read/8, error -110 Oct 19 21:16:21 localhost kernel: [187638.876201] usb 1-1.3: reset high-speed USB device number 13 using ehci-pci Oct 19 21:16:26 localhost kernel: [187643.890642] usb 1-1.3: device descriptor read/8, error -110 Oct 19 21:16:31 localhost kernel: [187649.005071] usb 1-1.3: device descriptor read/8, error -110 Oct 19 21:16:31 localhost kernel: [187649.106188] usb 1-1.3: USB disconnect, device number 13 Oct 19 21:16:31 localhost kernel: [187649.178969] usb 1-1.3: new high-speed USB device number 14 using ehci-pci Oct 19 21:16:47 localhost kernel: [187664.258945] usb 1-1.3: device descriptor read/64, error -110 Oct 19 21:17:02 localhost kernel: [187679.440092] usb 1-1.3: device descriptor read/64, error -110 Oct 19 21:17:02 localhost kernel: [187679.614194] usb 1-1.3: new high-speed USB device number 15 using ehci-pci Oct 19 21:17:17 localhost kernel: [187694.694148] usb 1-1.3: device descriptor read/64, error -110 Oct 19 21:17:32 localhost kernel: [187709.875297] usb 1-1.3: device descriptor read/64, error -110 Oct 19 21:17:32 localhost kernel: [187710.049386] usb 1-1.3: new high-speed USB device number 16 using ehci-pci Oct 19 21:17:37 localhost kernel: [187715.063803] usb 1-1.3: device descriptor read/8, error -110 Oct 19 21:17:41 localhost kernel: [187719.005453] usb 1-1.3: device descriptor read/8, error -71 After that -71 error, the thumb drive completely disappears from the host, as if it is powered down. I read that -110 is supposedly a power issue. I can play media files directly from the thumb drive on the host, so the power seems fine on the host. How reproducible: always Steps to reproduce: 1. use virt-manager, create a Windows 7 32 bit guest 2. in virt-manager, set Controller USB to USB2 3. on host, insert Sandisk Cruser Fit thumb drive FAT32 format, with an MP3 file on it 4. in virt-manager, add a USB passthrough device and assign it to thumb drive 5. boot Windows 7 guest 6. verify that Windows 7 can see the thumb drive 7. use Windows Media Player to play MP3 Actual results: guest hangs, then host powers off thumb drive Expected results: The MP3 file should play :) Additional info: Fedora 19 Installed Packages qemu-common.x86_64 2:1.4.2-11.fc19 @updates qemu-guest-agent.x86_64 2:1.4.2-11.fc19 @updates qemu-img.x86_64 2:1.4.2-11.fc19 @updates qemu-kvm.x86_64 2:1.4.2-11.fc19 @updates qemu-system-x86.x86_64 2:1.4.2-11.fc19 @updates virt-manager.noarch 0.10.0-3.fc19 @updates kernel.x86_64
[Bug 63291] KVM USB passthrough to Windows 7 guest fails with error -110, hangs
https://bugzilla.kernel.org/show_bug.cgi?id=63291 Gleb g...@redhat.com changed: What|Removed |Added CC||g...@redhat.com --- Comment #1 from Gleb g...@redhat.com --- This is QEMU issue, not kernel. Open bug in QEMU bug tracker: https://launchpad.net/qemu -- You are receiving this mail because: You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 60642] New: guest uses 100% and completely hangs
https://bugzilla.kernel.org/show_bug.cgi?id=60642 Bug ID: 60642 Summary: guest uses 100% and completely hangs Product: Virtualization Version: unspecified Kernel Version: guest: 3.10, host: 3.10 Hardware: x86-64 OS: Linux Tree: Mainline Status: NEW Severity: normal Priority: P1 Component: kvm Assignee: virtualization_...@kernel-bugs.osdl.org Reporter: folk...@vanheusden.com Regression: No Situation: pc with 3 vms running. 2 run fine, 1 jumps to 100% cpu and completely hangs minutes after start this vm has a virtual serial port connected to /dev/ttyUMTS0 which is an umts stick. as soon as wvdial successfully started a ppp session, the guest hangs The last thing top shows in the guest is: 3 root 20 0 000 R 40.2 0.0 0:00.86 ksoftirqd/0 2320 root 20 0 21476 1648 1260 R 36.0 0.7 0:02.14 tincd 132 root 20 0 000 R 27.7 0.0 0:01.18 kworker/0:2 strace on the kvm process in the host gives me: [pid 9418] write(5, `, 1)= -1 EAGAIN (Resource temporarily unavailable) [pid 9418] write(5, `, 1)= -1 EAGAIN (Resource temporarily unavailable) [pid 9418] write(5, `, 1)= -1 EAGAIN (Resource temporarily unavailable) [pid 9418] write(5, `, 1)= -1 EAGAIN (Resource temporarily unavailable) [pid 9418] write(5, `, 1)= -1 EAGAIN (Resource temporarily unavailable) fd 5 is the /dev/ttyUMTS0 device. gdb says: #0 __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135 #1 0x7f69ced73f3c in _L_lock_974 () from /lib/x86_64-linux-gnu/libpthread.so.0 #2 0x7f69ced73d8b in __GI___pthread_mutex_lock (mutex=0x7f69d3acfa80 qemu_global_mutex) at pthread_mutex_lock.c:64 #3 0x7f69d2daedc9 in ?? () #4 0x7f69d2d83c98 in ?? () #5 0x7f69d2c93973 in main () There's absolutely _no_ logging regarding this issue in dmesg on the host. I did some googling and tried replacing utc-clock by local clock, I also tried adding clocksource=acpi_pm but none of those helped. versions: root@neo:/sys/bus/usb/devices# dpkg --list | grep -e qemu -e kvm | awk '{ print $1\t$2\t$3\t$4; }' ii ipxe-qemu 1.0.0+git-20120202.f6840ba-3all ii kvm 1:1.1.2+dfsg-6 amd64 ii qemu1.1.2+dfsg-6a amd64 ii qemu-keymaps1.1.2+dfsg-6a all ii qemu-kvm1.1.2+dfsg-6amd64 ii qemu-system 1.1.2+dfsg-6a amd64 ii qemu-user 1.1.2+dfsg-6a amd64 ii qemu-utils 1.1.2+dfsg-6a amd64 After a restart of the vm, the exact same problem happens after 10 seconds so it is 100% reproducible here. -- You are receiving this mail because: You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 60642] guest uses 100% and completely hangs
https://bugzilla.kernel.org/show_bug.cgi?id=60642 --- Comment #1 from Folkert van Heusden folk...@vanheusden.com --- In the console of the guest I see: [sched_delayed] sched: RT throttling activated -- You are receiving this mail because: You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
kernel 3.9.x kvm hangs after seabios
I have the same issue, with 3.9.1 (3.9.0 too) it hangs right after seabios... (no problem in 3.8.11) qemu-1.4.1 seabios-1.7.2.1 after setting emulate_invalid_guest_state=0 everything works just fine. virsh # qemu-monitor-command vm-jack --hmp x/8i \$pc 0x000fc46b: lgdtw %cs:-0x2c60 0x000fc471: mov%cr0,%eax 0x000fc474: or $0x1,%eax 0x000fc478: mov%eax,%cr0 0x000fc47b: ljmpl $0x8,$0xfc483 0x000fc483: mov$0x10,%ax 0x000fc486: add%al,(%bx,%si) 0x000fc488: mov%ax,%ds virsh # qemu-monitor-command vm-jack --hmp x/64b \$pc 0x000fc46b: lgdtw %cs:-0x2c60 0x000fc471: mov%cr0,%eax 0x000fc474: or $0x1,%eax 0x000fc478: mov%eax,%cr0 0x000fc47b: ljmpl $0x8,$0xfc483 0x000fc483: mov$0x10,%ax 0x000fc486: add%al,(%bx,%si) 0x000fc488: mov%ax,%ds 0x000fc48a: mov%ax,%es 0x000fc48c: mov%ax,%ss 0x000fc48e: mov%ax,%fs 0x000fc490: mov%ax,%gs 0x000fc492: mov%cx,%ax 0x000fc494: jmp*%dx 0x000fc496: mov%ax,%cx 0x000fc498: mov$0x20,%ax 0x000fc49b: add%al,(%bx,%si) 0x000fc49d: mov%ax,%ds 0x000fc49f: mov%ax,%es 0x000fc4a1: mov%ax,%ss 0x000fc4a3: mov%ax,%fs 0x000fc4a5: mov%ax,%gs 0x000fc4a7: ljmpl $0xc189,$0x18c4c4 0x000fc4af: mov$0x30,%ax 0x000fc4b2: add%al,(%bx,%si) 0x000fc4b4: mov%ax,%ds 0x000fc4b6: mov%ax,%es 0x000fc4b8: mov%ax,%ss 0x000fc4ba: mov%ax,%fs 0x000fc4bc: mov%ax,%gs 0x000fc4be: ljmpl $0x200f,$0x28c4c4 0x000fc4c6: shlb $0xe0,-0x7d(%bp) 0x000fc4ca: decb (%bx) 0x000fc4cc: and%al,%al 0x000fc4ce: ljmp $0xf000,$0xc4d3 0x000fc4d3: lidtw %cs:-0x2c18 0x000fc4d9: xor%ax,%ax 0x000fc4db: mov%ax,%fs 0x000fc4dd: mov%ax,%gs 0x000fc4df: mov%ax,%es 0x000fc4e1: mov%ax,%ds 0x000fc4e3: mov%ax,%ss 0x000fc4e5: mov%ecx,%eax 0x000fc4e8: jmpl *%edx 0x000fc4eb: push %ebp 0x000fc4ed: push %eax 0x000fc4ef: pushl %es 0x000fc4f1: push %cs 0x000fc4f2: push $0xc536 0x000fc4f5: addr32 pushw %es:0x24(%eax) 0x000fc4fa: addr32 pushl %es:0x20(%eax) 0x000fc500: addr32 mov %es:0x4(%eax),%edi 0x000fc506: addr32 mov %es:0x8(%eax),%esi 0x000fc50c: addr32 mov %es:0xc(%eax),%ebp 0x000fc512: addr32 mov %es:0x10(%eax),%ebx 0x000fc518: addr32 mov %es:0x14(%eax),%edx 0x000fc51e: addr32 mov %es:0x18(%eax),%ecx 0x000fc524: addr32 mov %es:(%eax),%ds 0x000fc528: addr32 pushl %es:0x1c(%eax) 0x000fc52e: addr32 mov %es:0x2(%eax),%es 0x000fc533: pop%eax 0x000fc535: iret 0x000fc536: pushf 0x000fc537: cli -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel 3.9.x kvm hangs after seabios
On Wed, May 08, 2013 at 11:22:01AM +, Tomas Papan wrote: I have the same issue, with 3.9.1 (3.9.0 too) it hangs right after seabios... (no problem in 3.8.11) qemu-1.4.1 seabios-1.7.2.1 Is there anything interesting in libvirt logfile? Also please send the output of qemu-monitor-command vm-jack --hmp info registers And, just in case, can you send me your bios.bin image. Mine work. after setting emulate_invalid_guest_state=0 everything works just fine. virsh # qemu-monitor-command vm-jack --hmp x/8i \$pc 0x000fc46b: lgdtw %cs:-0x2c60 0x000fc471: mov%cr0,%eax 0x000fc474: or $0x1,%eax 0x000fc478: mov%eax,%cr0 0x000fc47b: ljmpl $0x8,$0xfc483 0x000fc483: mov$0x10,%ax 0x000fc486: add%al,(%bx,%si) 0x000fc488: mov%ax,%ds virsh # qemu-monitor-command vm-jack --hmp x/64b \$pc 0x000fc46b: lgdtw %cs:-0x2c60 0x000fc471: mov%cr0,%eax 0x000fc474: or $0x1,%eax 0x000fc478: mov%eax,%cr0 0x000fc47b: ljmpl $0x8,$0xfc483 0x000fc483: mov$0x10,%ax 0x000fc486: add%al,(%bx,%si) 0x000fc488: mov%ax,%ds 0x000fc48a: mov%ax,%es 0x000fc48c: mov%ax,%ss 0x000fc48e: mov%ax,%fs 0x000fc490: mov%ax,%gs 0x000fc492: mov%cx,%ax 0x000fc494: jmp*%dx 0x000fc496: mov%ax,%cx 0x000fc498: mov$0x20,%ax 0x000fc49b: add%al,(%bx,%si) 0x000fc49d: mov%ax,%ds 0x000fc49f: mov%ax,%es 0x000fc4a1: mov%ax,%ss 0x000fc4a3: mov%ax,%fs 0x000fc4a5: mov%ax,%gs 0x000fc4a7: ljmpl $0xc189,$0x18c4c4 0x000fc4af: mov$0x30,%ax 0x000fc4b2: add%al,(%bx,%si) 0x000fc4b4: mov%ax,%ds 0x000fc4b6: mov%ax,%es 0x000fc4b8: mov%ax,%ss 0x000fc4ba: mov%ax,%fs 0x000fc4bc: mov%ax,%gs 0x000fc4be: ljmpl $0x200f,$0x28c4c4 0x000fc4c6: shlb $0xe0,-0x7d(%bp) 0x000fc4ca: decb (%bx) 0x000fc4cc: and%al,%al 0x000fc4ce: ljmp $0xf000,$0xc4d3 0x000fc4d3: lidtw %cs:-0x2c18 0x000fc4d9: xor%ax,%ax 0x000fc4db: mov%ax,%fs 0x000fc4dd: mov%ax,%gs 0x000fc4df: mov%ax,%es 0x000fc4e1: mov%ax,%ds 0x000fc4e3: mov%ax,%ss 0x000fc4e5: mov%ecx,%eax 0x000fc4e8: jmpl *%edx 0x000fc4eb: push %ebp 0x000fc4ed: push %eax 0x000fc4ef: pushl %es 0x000fc4f1: push %cs 0x000fc4f2: push $0xc536 0x000fc4f5: addr32 pushw %es:0x24(%eax) 0x000fc4fa: addr32 pushl %es:0x20(%eax) 0x000fc500: addr32 mov %es:0x4(%eax),%edi 0x000fc506: addr32 mov %es:0x8(%eax),%esi 0x000fc50c: addr32 mov %es:0xc(%eax),%ebp 0x000fc512: addr32 mov %es:0x10(%eax),%ebx 0x000fc518: addr32 mov %es:0x14(%eax),%edx 0x000fc51e: addr32 mov %es:0x18(%eax),%ecx 0x000fc524: addr32 mov %es:(%eax),%ds 0x000fc528: addr32 pushl %es:0x1c(%eax) 0x000fc52e: addr32 mov %es:0x2(%eax),%es 0x000fc533: pop%eax 0x000fc535: iret 0x000fc536: pushf 0x000fc537: cli -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel 3.9.x kvm hangs after seabios
Hi, I found this in the libvirt (but those messages are same in 3.8.x) anakin libvirt # cat libvirtd.log 2013-05-08 11:59:29.645+: 3750: info : libvirt version: 1.0.5 2013-05-08 11:59:29.645+: 3750: error : udevGetDMIData:1548 : Failed to get udev device for syspath '/sys/devices/virtual/dmi/id' or '/sys/class/dmi/id' 2013-05-08 11:59:29.680+: 3750: warning : ebiptablesDriverInitCLITools:4225 : Could not find 'ebtables' executable virsh # qemu-monitor-command vm-jack --hmp info registers EAX=0002 EBX=64a1 ECX=6e08 EDX=000fc5ab ESI=c5b8 EDI=6eec EBP=dffd83e0 ESP=6df8 EIP=c46b EFL=00010002 [---] CPL=0 II=0 A20=1 SMM=0 HLT=1 ES = 9300 CS =f000 000f 9b00 SS = 9300 DS = 9300 FS = 9300 GS = 9300 LDT= 8200 TR = 8b00 GDT= 000fd3a8 0037 IDT= 000fd3e6 CR0=0010 CR2= CR3= CR4= DR0= DR1= DR2= DR3= DR6=0ff0 DR7=0400 EFER= FCW=037f FSW= [ST=0] FTW=00 MXCSR=1f80 FPR0= FPR1= FPR2= FPR3= FPR4= FPR5= FPR6= FPR7= XMM00= XMM01= XMM02= XMM03= XMM04= XMM05= XMM06= XMM07= bios.bin can be found here http://papan.sk/share/bios.bin I should mentioned that I'm using gentoo and libvirt 1.0.5. I'm sorry if gmail interface breaks output. Regrads Tomas On Wed, May 8, 2013 at 1:55 PM, Gleb Natapov g...@redhat.com wrote: On Wed, May 08, 2013 at 11:22:01AM +, Tomas Papan wrote: I have the same issue, with 3.9.1 (3.9.0 too) it hangs right after seabios... (no problem in 3.8.11) qemu-1.4.1 seabios-1.7.2.1 Is there anything interesting in libvirt logfile? Also please send the output of qemu-monitor-command vm-jack --hmp info registers And, just in case, can you send me your bios.bin image. Mine work. after setting emulate_invalid_guest_state=0 everything works just fine. virsh # qemu-monitor-command vm-jack --hmp x/8i \$pc 0x000fc46b: lgdtw %cs:-0x2c60 0x000fc471: mov%cr0,%eax 0x000fc474: or $0x1,%eax 0x000fc478: mov%eax,%cr0 0x000fc47b: ljmpl $0x8,$0xfc483 0x000fc483: mov$0x10,%ax 0x000fc486: add%al,(%bx,%si) 0x000fc488: mov%ax,%ds virsh # qemu-monitor-command vm-jack --hmp x/64b \$pc 0x000fc46b: lgdtw %cs:-0x2c60 0x000fc471: mov%cr0,%eax 0x000fc474: or $0x1,%eax 0x000fc478: mov%eax,%cr0 0x000fc47b: ljmpl $0x8,$0xfc483 0x000fc483: mov$0x10,%ax 0x000fc486: add%al,(%bx,%si) 0x000fc488: mov%ax,%ds 0x000fc48a: mov%ax,%es 0x000fc48c: mov%ax,%ss 0x000fc48e: mov%ax,%fs 0x000fc490: mov%ax,%gs 0x000fc492: mov%cx,%ax 0x000fc494: jmp*%dx 0x000fc496: mov%ax,%cx 0x000fc498: mov$0x20,%ax 0x000fc49b: add%al,(%bx,%si) 0x000fc49d: mov%ax,%ds 0x000fc49f: mov%ax,%es 0x000fc4a1: mov%ax,%ss 0x000fc4a3: mov%ax,%fs 0x000fc4a5: mov%ax,%gs 0x000fc4a7: ljmpl $0xc189,$0x18c4c4 0x000fc4af: mov$0x30,%ax 0x000fc4b2: add%al,(%bx,%si) 0x000fc4b4: mov%ax,%ds 0x000fc4b6: mov%ax,%es 0x000fc4b8: mov%ax,%ss 0x000fc4ba: mov%ax,%fs 0x000fc4bc: mov%ax,%gs 0x000fc4be: ljmpl $0x200f,$0x28c4c4 0x000fc4c6: shlb $0xe0,-0x7d(%bp) 0x000fc4ca: decb (%bx) 0x000fc4cc: and%al,%al 0x000fc4ce: ljmp $0xf000,$0xc4d3 0x000fc4d3: lidtw %cs:-0x2c18 0x000fc4d9: xor%ax,%ax 0x000fc4db: mov%ax,%fs 0x000fc4dd: mov%ax,%gs 0x000fc4df: mov%ax,%es 0x000fc4e1: mov%ax,%ds 0x000fc4e3: mov%ax,%ss 0x000fc4e5: mov%ecx,%eax 0x000fc4e8: jmpl *%edx 0x000fc4eb: push %ebp 0x000fc4ed: push %eax 0x000fc4ef: pushl %es 0x000fc4f1: push %cs 0x000fc4f2: push $0xc536 0x000fc4f5: addr32 pushw %es:0x24(%eax) 0x000fc4fa: addr32 pushl %es:0x20(%eax) 0x000fc500: addr32 mov %es:0x4(%eax),%edi 0x000fc506
Re: kernel 3.9.x kvm hangs after seabios
On Wed, May 08, 2013 at 02:08:55PM +0200, Tomas Papan wrote: Hi, I found this in the libvirt (but those messages are same in 3.8.x) anakin libvirt # cat libvirtd.log 2013-05-08 11:59:29.645+: 3750: info : libvirt version: 1.0.5 2013-05-08 11:59:29.645+: 3750: error : udevGetDMIData:1548 : Failed to get udev device for syspath '/sys/devices/virtual/dmi/id' or '/sys/class/dmi/id' 2013-05-08 11:59:29.680+: 3750: warning : ebiptablesDriverInitCLITools:4225 : Could not find 'ebtables' executable Nothing about KVM internal error? Couple of more things please: 1. Output of qemu-monitor-command vm-jack --hmp info status. 2. command line. 3. trace http://www.linux-kvm.org/page/Tracing -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel 3.9.x kvm hangs after seabios
Hi, No nothing, I check all logs (even syslog) 1) virsh # qemu-monitor-command vm-jack --hmp info status VM status: running 2) morpheus@anakin ~ $ ps aux | grep vm-jack qemu 3822 0.5 0.1 8952256 23600 ? Sl 13:59 0:08 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name vm-jack -S -machine pc-0.14,accel=kvm,usb=off -cpu Nehalem,+rdtscp,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -m 8192 -smp 4,sockets=4,cores=1,threads=1 -uuid 03196c23-24ba-d398-a000-582b0e88b0e7 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/vm-jack.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot order=c,menu=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/jack.img,if=none,id=drive-virtio-disk0,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0 -drive file=/var/lib/libvirt/images/kernel.img,if=none,id=drive-virtio-disk1,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk1,id=virtio-disk1 -drive if=none,id=drive-ide0-1-0,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=19,id=hostnet0 -device e1000,netdev=hostnet0,id=net0,mac=52:54:00:21:1c:e0,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -vnc 127.0.0.1:0 -k en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 3) it took some time, I didn't have debug_fs, then tracing... but the file is stored here (15 MB) http://papan.sk/share/trace.dat.tar.gz Regards Tomas -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel 3.9.x kvm hangs after seabios
On Wed, May 08, 2013 at 02:08:55PM +0200, Tomas Papan wrote: Hi, I found this in the libvirt (but those messages are same in 3.8.x) anakin libvirt # cat libvirtd.log 2013-05-08 11:59:29.645+: 3750: info : libvirt version: 1.0.5 2013-05-08 11:59:29.645+: 3750: error : udevGetDMIData:1548 : Failed to get udev device for syspath '/sys/devices/virtual/dmi/id' or '/sys/class/dmi/id' 2013-05-08 11:59:29.680+: 3750: warning : ebiptablesDriverInitCLITools:4225 : Could not find 'ebtables' executable You need to look at /var/log/libvirt/qemu/$GUESTNAME.log for QEMU related messages. The libvirtd.log file only has the libvirt related messages. Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel 3.9.x kvm hangs after seabios
Sorry, I didn't write that well, I checked that log too... nothing is there... anakin qemu # cat vm-jack.log 2013-05-08 13:02:52.358+: starting up LC_ALL=C PATH=/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/opt/bin HOME=/root USER=root QEMU_AUDIO_DRV=none /usr/bin/qemu-kvm -name vm-jack -S -machine pc-0.14,accel=kvm,usb=off -cpu Nehalem,+rdtscp,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -m 8192 -smp 4,sockets=4,cores=1,threads=1 -uuid 03196c23-24ba-d398-a000-582b0e88b0e7 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/vm-jack.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot order=c,menu=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/jack.img,if=none,id=drive-virtio-disk0,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0 -drive file=/var/lib/libvirt/images/kernel.img,if=none,id=drive-virtio-disk1,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk1,id=virtio-disk1 -drive if=none,id=drive-ide0-1-0,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=19,id=hostnet0 -device e1000,netdev=hostnet0,id=net0,mac=52:54:00:21:1c:e0,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -vnc 127.0.0.1:0 -k en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 char device redirected to /dev/pts/3 (label charserial0) -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel 3.9.x kvm hangs after seabios
On Wed, May 08, 2013 at 02:51:48PM +0200, Tomas Papan wrote: Hi, No nothing, I check all logs (even syslog) Yeah, since status of the vm is running you are not suppose to see there anything. 1) virsh # qemu-monitor-command vm-jack --hmp info status VM status: running 2) morpheus@anakin ~ $ ps aux | grep vm-jack qemu 3822 0.5 0.1 8952256 23600 ? Sl 13:59 0:08 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name vm-jack -S -machine pc-0.14,accel=kvm,usb=off -cpu Nehalem,+rdtscp,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -m 8192 -smp 4,sockets=4,cores=1,threads=1 -uuid 03196c23-24ba-d398-a000-582b0e88b0e7 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/vm-jack.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot order=c,menu=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/jack.img,if=none,id=drive-virtio-disk0,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0 -drive file=/var/lib/libvirt/images/kernel.img,if=none,id=drive-virtio-disk1,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk1,id=virtio-disk1 -drive if=none,id=drive-ide0-1-0,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=19,id=hostnet0 -device e1000,netdev=hostnet0,id=net0,mac=52:54:00:21:1c:e0,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -vnc 127.0.0.1:0 -k en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 3) it took some time, I didn't have debug_fs, then tracing... but the file is stored here (15 MB) http://papan.sk/share/trace.dat.tar.gz Very interesting. In the middle of the run vcpu decides that it does not want to run any more. How much cpu time qemu takes when it happens? If it is 100% can you do the following: 1. run qemu-monitor-command vm-jack --hmp info cpus 2. note thread id for cpu #0 3. run trace-cmd record -P $pid -p function where $pid is the pid thread id that you've found in 2. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel 3.9.x kvm hangs after seabios
Ok, the cpu stays at 0% when it hangs, there is only one 100% cpu peak which happens when the vm starts ( I think this is quite normal). However I run following command, and I stop it right when it hangs: anakin trace2 # virsh start vm-jack; pid=`virsh qemu-monitor-command vm-jack --hmp info cpus | grep '\*' | awk '{print $5}' | cut -d\= -f2`; trace-cmd record -P $pid -p function if anyone is interested it produces a 1.6 GB file (the compressed version can be found here: http://papan.sk/share/trace2.dat.tar.gz (150 MB)) Tomas -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel 3.9.x kvm hangs after seabios
On Wed, May 08, 2013 at 03:50:47PM +0200, Tomas Papan wrote: Ok, the cpu stays at 0% when it hangs, there is only one 100% cpu peak which happens when the vm starts ( I think this is quite normal). However I run following command, and I stop it right when it hangs: anakin trace2 # virsh start vm-jack; pid=`virsh qemu-monitor-command vm-jack --hmp info cpus | grep '\*' | awk '{print $5}' | cut -d\= -f2`; trace-cmd record -P $pid -p function if anyone is interested it produces a 1.6 GB file (the compressed version can be found here: http://papan.sk/share/trace2.dat.tar.gz (150 MB)) Thanks! Can you test the patch below: diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c index 6667042..0af1807 100644 --- a/arch/x86/kvm/vmx.c +++ b/arch/x86/kvm/vmx.c @@ -5197,6 +5197,12 @@ static int handle_invalid_guest_state(struct kvm_vcpu *vcpu) return 0; } + if (vcpu-arch.halt_request) { + vcpu-arch.halt_request = 0; + ret = kvm_emulate_halt(vcpu); + goto out; + } + if (signal_pending(current)) goto out; if (need_resched()) -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel 3.9.x kvm hangs after seabios
-- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel 3.9.x kvm hangs after seabios
patch is working :) Thank you very much Gleb. Regards Tomas -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel 3.9.x kvm hangs after seabios
On Wed, May 08, 2013 at 04:52:52PM +0200, Tomas Papan wrote: patch is working :) Thank you very much Gleb. Thank you for your patience. Curious but it was. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs
A change in the generic code highlighted that we were running with IRQs (soft) enabled on Book3E 64-bit when trying to restart interrupts from handle_exit(). This is a lesson to configure lockdep often :) There is no reason to exit guest with soft_enabled == 1, a local_irq_enable() call will do this for us so get rid of kvmppc_layz_ee() calls. With this fix we eliminate irqs_disabled() warnings and some guest and host hangs revealed under stress tests, but guests still exhibit some unresponsiveness. The unresponsiveness has to do with the fact that arch_local_irq_restore() does not guarantees to hard enable interrupts. To do so replace exception function calls like timer_interrupt() with irq_happened flags. The local_irq_enable() call takes care of replaying them and lets the interrupts hard enabled. Signed-off-by: Mihai Caraman mihai.cara...@freescale.com --- arch/powerpc/kvm/booke.c |9 +++-- 1 files changed, 3 insertions(+), 6 deletions(-) diff --git a/arch/powerpc/kvm/booke.c b/arch/powerpc/kvm/booke.c index 1020119..82f155e 100644 --- a/arch/powerpc/kvm/booke.c +++ b/arch/powerpc/kvm/booke.c @@ -673,7 +673,6 @@ int kvmppc_vcpu_run(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu) ret = s; goto out; } - kvmppc_lazy_ee_enable(); kvm_guest_enter(); @@ -789,16 +788,16 @@ static void kvmppc_restart_interrupt(struct kvm_vcpu *vcpu, switch (exit_nr) { case BOOKE_INTERRUPT_EXTERNAL: kvmppc_fill_pt_regs(regs); - do_IRQ(regs); + local_paca-irq_happened |= PACA_IRQ_EE; break; case BOOKE_INTERRUPT_DECREMENTER: kvmppc_fill_pt_regs(regs); - timer_interrupt(regs); + local_paca-irq_happened |= PACA_IRQ_DEC; break; #if defined(CONFIG_PPC_FSL_BOOK3E) || defined(CONFIG_PPC_BOOK3E_64) case BOOKE_INTERRUPT_DOORBELL: kvmppc_fill_pt_regs(regs); - doorbell_exception(regs); + local_paca-irq_happened |= PACA_IRQ_DBELL; break; #endif case BOOKE_INTERRUPT_MACHINE_CHECK: @@ -1148,8 +1147,6 @@ int kvmppc_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu, if (s = 0) { local_irq_enable(); r = (s 2) | RESUME_HOST | (r RESUME_FLAG_NV); - } else { - kvmppc_lazy_ee_enable(); } } -- 1.7.4.1 -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs
On 03.05.2013, at 18:11, Mihai Caraman wrote: A change in the generic code highlighted that we were running with IRQs (soft) enabled on Book3E 64-bit when trying to restart interrupts from handle_exit(). This is a lesson to configure lockdep often :) There is no reason to exit guest with soft_enabled == 1, a local_irq_enable() call will do this for us so get rid of kvmppc_layz_ee() calls. With this fix we eliminate irqs_disabled() warnings and some guest and host hangs revealed under stress tests, but guests still exhibit some unresponsiveness. The unresponsiveness has to do with the fact that arch_local_irq_restore() does not guarantees to hard enable interrupts. To do so replace exception function calls like timer_interrupt() with irq_happened flags. The local_irq_enable() call takes care of replaying them and lets the interrupts hard enabled. Signed-off-by: Mihai Caraman mihai.cara...@freescale.com Ben, could you please review? Alex --- arch/powerpc/kvm/booke.c |9 +++-- 1 files changed, 3 insertions(+), 6 deletions(-) diff --git a/arch/powerpc/kvm/booke.c b/arch/powerpc/kvm/booke.c index 1020119..82f155e 100644 --- a/arch/powerpc/kvm/booke.c +++ b/arch/powerpc/kvm/booke.c @@ -673,7 +673,6 @@ int kvmppc_vcpu_run(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu) ret = s; goto out; } - kvmppc_lazy_ee_enable(); kvm_guest_enter(); @@ -789,16 +788,16 @@ static void kvmppc_restart_interrupt(struct kvm_vcpu *vcpu, switch (exit_nr) { case BOOKE_INTERRUPT_EXTERNAL: kvmppc_fill_pt_regs(regs); - do_IRQ(regs); + local_paca-irq_happened |= PACA_IRQ_EE; break; case BOOKE_INTERRUPT_DECREMENTER: kvmppc_fill_pt_regs(regs); - timer_interrupt(regs); + local_paca-irq_happened |= PACA_IRQ_DEC; break; #if defined(CONFIG_PPC_FSL_BOOK3E) || defined(CONFIG_PPC_BOOK3E_64) case BOOKE_INTERRUPT_DOORBELL: kvmppc_fill_pt_regs(regs); - doorbell_exception(regs); + local_paca-irq_happened |= PACA_IRQ_DBELL; break; #endif case BOOKE_INTERRUPT_MACHINE_CHECK: @@ -1148,8 +1147,6 @@ int kvmppc_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu, if (s = 0) { local_irq_enable(); r = (s 2) | RESUME_HOST | (r RESUME_FLAG_NV); - } else { - kvmppc_lazy_ee_enable(); } } -- 1.7.4.1 -- To unsubscribe from this list: send the line unsubscribe kvm-ppc in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs
On 05/03/2013 11:11:10 AM, Mihai Caraman wrote: A change in the generic code highlighted that we were running with IRQs (soft) enabled on Book3E 64-bit when trying to restart interrupts from handle_exit(). This is a lesson to configure lockdep often :) There is no reason to exit guest with soft_enabled == 1, a local_irq_enable() call will do this for us so get rid of kvmppc_layz_ee() calls. With this fix we eliminate irqs_disabled() warnings and some guest and host hangs revealed under stress tests, but guests still exhibit some unresponsiveness. The unresponsiveness has to do with the fact that arch_local_irq_restore() does not guarantees to hard enable interrupts. Could you elaborate? If the saved IRQ state was enabled, why wouldn't arch_local_irq_restore() hard-enable IRQs? The last thing it does is __hard_irq_enable(). Where is the arch_local_irq_restore() instance you're talking about? To do so replace exception function calls like timer_interrupt() with irq_happened flags. The local_irq_enable() call takes care of replaying them and lets the interrupts hard enabled. Not sure what you mean by lets the interrupts hard enabled... Do you mean the EE bit in regs-msr, as opposed to the EE bit in the current MSR? Signed-off-by: Mihai Caraman mihai.cara...@freescale.com --- arch/powerpc/kvm/booke.c |9 +++-- 1 files changed, 3 insertions(+), 6 deletions(-) diff --git a/arch/powerpc/kvm/booke.c b/arch/powerpc/kvm/booke.c index 1020119..82f155e 100644 --- a/arch/powerpc/kvm/booke.c +++ b/arch/powerpc/kvm/booke.c @@ -673,7 +673,6 @@ int kvmppc_vcpu_run(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu) ret = s; goto out; } - kvmppc_lazy_ee_enable(); kvm_guest_enter(); @@ -789,16 +788,16 @@ static void kvmppc_restart_interrupt(struct kvm_vcpu *vcpu, switch (exit_nr) { case BOOKE_INTERRUPT_EXTERNAL: kvmppc_fill_pt_regs(regs); - do_IRQ(regs); + local_paca-irq_happened |= PACA_IRQ_EE; break; case BOOKE_INTERRUPT_DECREMENTER: kvmppc_fill_pt_regs(regs); - timer_interrupt(regs); + local_paca-irq_happened |= PACA_IRQ_DEC; break; #if defined(CONFIG_PPC_FSL_BOOK3E) || defined(CONFIG_PPC_BOOK3E_64) case BOOKE_INTERRUPT_DOORBELL: kvmppc_fill_pt_regs(regs); - doorbell_exception(regs); + local_paca-irq_happened |= PACA_IRQ_DBELL; break; #endif Aren't you breaking 32-bit here? -Scott -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs
-Original Message- From: Wood Scott-B07421 Sent: Friday, May 03, 2013 9:05 PM To: Caraman Mihai Claudiu-B02008 Cc: kvm-...@vger.kernel.org; kvm@vger.kernel.org; linuxppc- d...@lists.ozlabs.org; Caraman Mihai Claudiu-B02008 Subject: Re: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs The unresponsiveness has to do with the fact that arch_local_irq_restore() does not guarantees to hard enable interrupts. Could you elaborate? If the saved IRQ state was enabled, why wouldn't arch_local_irq_restore() hard-enable IRQs? The last thing it does is __hard_irq_enable(). if (!irq_happened) return; Where is the arch_local_irq_restore() instance you're talking about? ./arch/power/kernel/irq.c To do so replace exception function calls like timer_interrupt() with irq_happened flags. The local_irq_enable() call takes care of replaying them and lets the interrupts hard enabled. Not sure what you mean by lets the interrupts hard enabled... Do you mean the EE bit in regs-msr, as opposed to the EE bit in the current MSR? If irq_happened the last thing it does is __hard_irq_enable(). @@ -789,16 +788,16 @@ static void kvmppc_restart_interrupt(struct kvm_vcpu *vcpu, switch (exit_nr) { case BOOKE_INTERRUPT_EXTERNAL: kvmppc_fill_pt_regs(regs); - do_IRQ(regs); + local_paca-irq_happened |= PACA_IRQ_EE; break; Aren't you breaking 32-bit here? I had eyes only for 64-bit hangs :) -Mike -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs
On 05/03/2013 03:01:26 PM, Caraman Mihai Claudiu-B02008 wrote: -Original Message- From: Wood Scott-B07421 Sent: Friday, May 03, 2013 9:05 PM To: Caraman Mihai Claudiu-B02008 Cc: kvm-...@vger.kernel.org; kvm@vger.kernel.org; linuxppc- d...@lists.ozlabs.org; Caraman Mihai Claudiu-B02008 Subject: Re: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs The unresponsiveness has to do with the fact that arch_local_irq_restore() does not guarantees to hard enable interrupts. Could you elaborate? If the saved IRQ state was enabled, why wouldn't arch_local_irq_restore() hard-enable IRQs? The last thing it does is __hard_irq_enable(). if (!irq_happened) return; OK, so the problem is that we're not setting PACA_IRQ_HARD_DIS when we hard-disable interrupts? Where is the arch_local_irq_restore() instance you're talking about? ./arch/power/kernel/irq.c I meant the caller. :-P -Scott -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs
-Original Message- From: Wood Scott-B07421 Sent: Friday, May 03, 2013 11:15 PM To: Caraman Mihai Claudiu-B02008 Cc: Wood Scott-B07421; kvm-...@vger.kernel.org; kvm@vger.kernel.org; linuxppc-...@lists.ozlabs.org Subject: Re: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs The unresponsiveness has to do with the fact that arch_local_irq_restore() does not guarantees to hard enable interrupts. Could you elaborate? If the saved IRQ state was enabled, why wouldn't arch_local_irq_restore() hard-enable IRQs? The last thing it does is __hard_irq_enable(). if (!irq_happened) return; OK, so the problem is that we're not setting PACA_IRQ_HARD_DIS when we hard-disable interrupts? We enter guest with local_irq_disable() which means soft disabled, when do we hard-disable interrupts? If we follow host exception handlers model they set PACA_IRQ_EE/DEC/DBELL but not PACA_IRQ_HARD_DIS. Can you give it a try to see how KVM behaves with PACA_IRQ_HARD_DIS? I can't do it right now. Where is the arch_local_irq_restore() instance you're talking about? ./arch/power/kernel/irq.c I meant the caller. :-P ./arch/powerpc/include/asm/hw_irq.h 55static inline unsigned long arch_local_irq_disable(void) 56{ 57unsigned long flags, zero; 58 59asm volatile( 60li %1,0; lbz %0,%2(13); stb %1,%2(13) 61: =r (flags), =r (zero) 62: i (offsetof(struct paca_struct, soft_enabled)) 63: memory); 64 65return flags; 66} 67 68extern void arch_local_irq_restore(unsigned long); 69 70static inline void arch_local_irq_enable(void) 71{ 72arch_local_irq_restore(1); 73} -Mike -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs
On Fri, 2013-05-03 at 18:24 +0200, Alexander Graf wrote: There is no reason to exit guest with soft_enabled == 1, a local_irq_enable() call will do this for us so get rid of kvmppc_layz_ee() calls. With this fix we eliminate irqs_disabled() warnings and some guest and host hangs revealed under stress tests, but guests still exhibit some unresponsiveness. The unresponsiveness has to do with the fact that arch_local_irq_restore() does not guarantees to hard enable interrupts. To do so replace exception function calls like timer_interrupt() with irq_happened flags. The local_irq_enable() call takes care of replaying them and lets the interrupts hard enabled. Signed-off-by: Mihai Caraman mihai.cara...@freescale.com Ben, could you please review? That does look like the right thing to do indeed. Acked-by: Benjamin Herrenschmidt b...@kernel.crashing.org Cheers, Ben. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs
On 05/03/2013 03:56:47 PM, Caraman Mihai Claudiu-B02008 wrote: -Original Message- From: Wood Scott-B07421 Sent: Friday, May 03, 2013 11:15 PM To: Caraman Mihai Claudiu-B02008 Cc: Wood Scott-B07421; kvm-...@vger.kernel.org; kvm@vger.kernel.org; linuxppc-...@lists.ozlabs.org Subject: Re: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs The unresponsiveness has to do with the fact that arch_local_irq_restore() does not guarantees to hard enable interrupts. Could you elaborate? If the saved IRQ state was enabled, why wouldn't arch_local_irq_restore() hard-enable IRQs? The last thing it does is __hard_irq_enable(). if (!irq_happened) return; OK, so the problem is that we're not setting PACA_IRQ_HARD_DIS when we hard-disable interrupts? We enter guest with local_irq_disable() which means soft disabled, Hmm... I don't see any obvious breakage from that, but it makes me nervous. I'd be more comfortable if we just hard-disabled interrupts there. when do we hard-disable interrupts? Interrupts will be hard-disabled when we take an exception to exit guest state. If we follow host exception handlers model they set PACA_IRQ_EE/DEC/DBELL but not PACA_IRQ_HARD_DIS. Can you give it a try to see how KVM behaves with PACA_IRQ_HARD_DIS? I can't do it right now. I replaced the two calls to kvmppc_lazy_ee_enable() with calls to hard_irq_disable(), and it seems to be working fine. Where is the arch_local_irq_restore() instance you're talking about? ./arch/power/kernel/irq.c I meant the caller. :-P ./arch/powerpc/include/asm/hw_irq.h 55static inline unsigned long arch_local_irq_disable(void) 56{ 57unsigned long flags, zero; 58 59asm volatile( 60li %1,0; lbz %0,%2(13); stb %1,%2(13) 61: =r (flags), =r (zero) 62: i (offsetof(struct paca_struct, soft_enabled)) 63: memory); 64 65return flags; 66} 67 68extern void arch_local_irq_restore(unsigned long); 69 70static inline void arch_local_irq_enable(void) 71{ 72arch_local_irq_restore(1); 73} Sigh. I meant the real caller, who's calling local_irq_restore(). -Scott -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs
-Original Message- From: Wood Scott-B07421 Sent: Saturday, May 04, 2013 1:07 AM To: Caraman Mihai Claudiu-B02008 Cc: Wood Scott-B07421; kvm-...@vger.kernel.org; kvm@vger.kernel.org; linuxppc-...@lists.ozlabs.org Subject: Re: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs I replaced the two calls to kvmppc_lazy_ee_enable() with calls to hard_irq_disable(), and it seems to be working fine. Please take a look on 'KVM: PPC64: booke: Hard disable interrupts when entering guest' RFC thread and see if your solution addresses Ben's comments. Where is the arch_local_irq_restore() instance you're talking about? ./arch/power/kernel/irq.c I meant the caller. :-P ./arch/powerpc/include/asm/hw_irq.h 55static inline unsigned long arch_local_irq_disable(void) 56{ 57unsigned long flags, zero; 58 59asm volatile( 60li %1,0; lbz %0,%2(13); stb %1,%2(13) 61: =r (flags), =r (zero) 62: i (offsetof(struct paca_struct, soft_enabled)) 63: memory); 64 65return flags; 66} 67 68extern void arch_local_irq_restore(unsigned long); 69 70static inline void arch_local_irq_enable(void) 71{ 72arch_local_irq_restore(1); 73} Sigh. I meant the real caller, who's calling local_irq_restore(). I'm not sure what you mean, arch_local_irq_restore() is called indirectly by local_irq_enable() in our case from handle_exit(). -Mike -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs
On 05/03/2013 05:59:32 PM, Caraman Mihai Claudiu-B02008 wrote: -Original Message- From: Wood Scott-B07421 Sent: Saturday, May 04, 2013 1:07 AM To: Caraman Mihai Claudiu-B02008 Cc: Wood Scott-B07421; kvm-...@vger.kernel.org; kvm@vger.kernel.org; linuxppc-...@lists.ozlabs.org Subject: Re: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs I replaced the two calls to kvmppc_lazy_ee_enable() with calls to hard_irq_disable(), and it seems to be working fine. Please take a look on 'KVM: PPC64: booke: Hard disable interrupts when entering guest' RFC thread and see if your solution addresses Ben's comments. My original one didn't (there was a race if an interrupt comes in between soft-disabling and hard-disabling, it wouldn't be received until the guest exits for some other reason). Instead, I turned the local_irq_disable() into hard_irq_disable() plus trace_hardirqs_off(). This worked without warnings. -Scott -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs
A change in the generic code highlighted that we were running with IRQs (soft) enabled on Book3E 64-bit when trying to restart interrupts from handle_exit(). This is a lesson to configure lockdep often :) There is no reason to exit guest with soft_enabled == 1, a local_irq_enable() call will do this for us so get rid of kvmppc_layz_ee() calls. With this fix we eliminate irqs_disabled() warnings and some guest and host hangs revealed under stress tests, but guests still exhibit some unresponsiveness. The unresponsiveness has to do with the fact that arch_local_irq_restore() does not guarantees to hard enable interrupts. To do so replace exception function calls like timer_interrupt() with irq_happened flags. The local_irq_enable() call takes care of replaying them and lets the interrupts hard enabled. Signed-off-by: Mihai Caraman mihai.cara...@freescale.com --- arch/powerpc/kvm/booke.c |9 +++-- 1 files changed, 3 insertions(+), 6 deletions(-) diff --git a/arch/powerpc/kvm/booke.c b/arch/powerpc/kvm/booke.c index 1020119..82f155e 100644 --- a/arch/powerpc/kvm/booke.c +++ b/arch/powerpc/kvm/booke.c @@ -673,7 +673,6 @@ int kvmppc_vcpu_run(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu) ret = s; goto out; } - kvmppc_lazy_ee_enable(); kvm_guest_enter(); @@ -789,16 +788,16 @@ static void kvmppc_restart_interrupt(struct kvm_vcpu *vcpu, switch (exit_nr) { case BOOKE_INTERRUPT_EXTERNAL: kvmppc_fill_pt_regs(regs); - do_IRQ(regs); + local_paca-irq_happened |= PACA_IRQ_EE; break; case BOOKE_INTERRUPT_DECREMENTER: kvmppc_fill_pt_regs(regs); - timer_interrupt(regs); + local_paca-irq_happened |= PACA_IRQ_DEC; break; #if defined(CONFIG_PPC_FSL_BOOK3E) || defined(CONFIG_PPC_BOOK3E_64) case BOOKE_INTERRUPT_DOORBELL: kvmppc_fill_pt_regs(regs); - doorbell_exception(regs); + local_paca-irq_happened |= PACA_IRQ_DBELL; break; #endif case BOOKE_INTERRUPT_MACHINE_CHECK: @@ -1148,8 +1147,6 @@ int kvmppc_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu, if (s = 0) { local_irq_enable(); r = (s 2) | RESUME_HOST | (r RESUME_FLAG_NV); - } else { - kvmppc_lazy_ee_enable(); } } -- 1.7.4.1 -- To unsubscribe from this list: send the line unsubscribe kvm-ppc in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs
On 03.05.2013, at 18:11, Mihai Caraman wrote: A change in the generic code highlighted that we were running with IRQs (soft) enabled on Book3E 64-bit when trying to restart interrupts from handle_exit(). This is a lesson to configure lockdep often :) There is no reason to exit guest with soft_enabled == 1, a local_irq_enable() call will do this for us so get rid of kvmppc_layz_ee() calls. With this fix we eliminate irqs_disabled() warnings and some guest and host hangs revealed under stress tests, but guests still exhibit some unresponsiveness. The unresponsiveness has to do with the fact that arch_local_irq_restore() does not guarantees to hard enable interrupts. To do so replace exception function calls like timer_interrupt() with irq_happened flags. The local_irq_enable() call takes care of replaying them and lets the interrupts hard enabled. Signed-off-by: Mihai Caraman mihai.cara...@freescale.com Ben, could you please review? Alex --- arch/powerpc/kvm/booke.c |9 +++-- 1 files changed, 3 insertions(+), 6 deletions(-) diff --git a/arch/powerpc/kvm/booke.c b/arch/powerpc/kvm/booke.c index 1020119..82f155e 100644 --- a/arch/powerpc/kvm/booke.c +++ b/arch/powerpc/kvm/booke.c @@ -673,7 +673,6 @@ int kvmppc_vcpu_run(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu) ret = s; goto out; } - kvmppc_lazy_ee_enable(); kvm_guest_enter(); @@ -789,16 +788,16 @@ static void kvmppc_restart_interrupt(struct kvm_vcpu *vcpu, switch (exit_nr) { case BOOKE_INTERRUPT_EXTERNAL: kvmppc_fill_pt_regs(regs); - do_IRQ(regs); + local_paca-irq_happened |= PACA_IRQ_EE; break; case BOOKE_INTERRUPT_DECREMENTER: kvmppc_fill_pt_regs(regs); - timer_interrupt(regs); + local_paca-irq_happened |= PACA_IRQ_DEC; break; #if defined(CONFIG_PPC_FSL_BOOK3E) || defined(CONFIG_PPC_BOOK3E_64) case BOOKE_INTERRUPT_DOORBELL: kvmppc_fill_pt_regs(regs); - doorbell_exception(regs); + local_paca-irq_happened |= PACA_IRQ_DBELL; break; #endif case BOOKE_INTERRUPT_MACHINE_CHECK: @@ -1148,8 +1147,6 @@ int kvmppc_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu, if (s = 0) { local_irq_enable(); r = (s 2) | RESUME_HOST | (r RESUME_FLAG_NV); - } else { - kvmppc_lazy_ee_enable(); } } -- 1.7.4.1 -- To unsubscribe from this list: send the line unsubscribe kvm-ppc in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe kvm-ppc in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs
On 05/03/2013 11:11:10 AM, Mihai Caraman wrote: A change in the generic code highlighted that we were running with IRQs (soft) enabled on Book3E 64-bit when trying to restart interrupts from handle_exit(). This is a lesson to configure lockdep often :) There is no reason to exit guest with soft_enabled == 1, a local_irq_enable() call will do this for us so get rid of kvmppc_layz_ee() calls. With this fix we eliminate irqs_disabled() warnings and some guest and host hangs revealed under stress tests, but guests still exhibit some unresponsiveness. The unresponsiveness has to do with the fact that arch_local_irq_restore() does not guarantees to hard enable interrupts. Could you elaborate? If the saved IRQ state was enabled, why wouldn't arch_local_irq_restore() hard-enable IRQs? The last thing it does is __hard_irq_enable(). Where is the arch_local_irq_restore() instance you're talking about? To do so replace exception function calls like timer_interrupt() with irq_happened flags. The local_irq_enable() call takes care of replaying them and lets the interrupts hard enabled. Not sure what you mean by lets the interrupts hard enabled... Do you mean the EE bit in regs-msr, as opposed to the EE bit in the current MSR? Signed-off-by: Mihai Caraman mihai.cara...@freescale.com --- arch/powerpc/kvm/booke.c |9 +++-- 1 files changed, 3 insertions(+), 6 deletions(-) diff --git a/arch/powerpc/kvm/booke.c b/arch/powerpc/kvm/booke.c index 1020119..82f155e 100644 --- a/arch/powerpc/kvm/booke.c +++ b/arch/powerpc/kvm/booke.c @@ -673,7 +673,6 @@ int kvmppc_vcpu_run(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu) ret = s; goto out; } - kvmppc_lazy_ee_enable(); kvm_guest_enter(); @@ -789,16 +788,16 @@ static void kvmppc_restart_interrupt(struct kvm_vcpu *vcpu, switch (exit_nr) { case BOOKE_INTERRUPT_EXTERNAL: kvmppc_fill_pt_regs(regs); - do_IRQ(regs); + local_paca-irq_happened |= PACA_IRQ_EE; break; case BOOKE_INTERRUPT_DECREMENTER: kvmppc_fill_pt_regs(regs); - timer_interrupt(regs); + local_paca-irq_happened |= PACA_IRQ_DEC; break; #if defined(CONFIG_PPC_FSL_BOOK3E) || defined(CONFIG_PPC_BOOK3E_64) case BOOKE_INTERRUPT_DOORBELL: kvmppc_fill_pt_regs(regs); - doorbell_exception(regs); + local_paca-irq_happened |= PACA_IRQ_DBELL; break; #endif Aren't you breaking 32-bit here? -Scott -- To unsubscribe from this list: send the line unsubscribe kvm-ppc in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs
-Original Message- From: Wood Scott-B07421 Sent: Friday, May 03, 2013 9:05 PM To: Caraman Mihai Claudiu-B02008 Cc: kvm-ppc@vger.kernel.org; k...@vger.kernel.org; linuxppc- d...@lists.ozlabs.org; Caraman Mihai Claudiu-B02008 Subject: Re: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs The unresponsiveness has to do with the fact that arch_local_irq_restore() does not guarantees to hard enable interrupts. Could you elaborate? If the saved IRQ state was enabled, why wouldn't arch_local_irq_restore() hard-enable IRQs? The last thing it does is __hard_irq_enable(). if (!irq_happened) return; Where is the arch_local_irq_restore() instance you're talking about? ./arch/power/kernel/irq.c To do so replace exception function calls like timer_interrupt() with irq_happened flags. The local_irq_enable() call takes care of replaying them and lets the interrupts hard enabled. Not sure what you mean by lets the interrupts hard enabled... Do you mean the EE bit in regs-msr, as opposed to the EE bit in the current MSR? If irq_happened the last thing it does is __hard_irq_enable(). @@ -789,16 +788,16 @@ static void kvmppc_restart_interrupt(struct kvm_vcpu *vcpu, switch (exit_nr) { case BOOKE_INTERRUPT_EXTERNAL: kvmppc_fill_pt_regs(regs); - do_IRQ(regs); + local_paca-irq_happened |= PACA_IRQ_EE; break; Aren't you breaking 32-bit here? I had eyes only for 64-bit hangs :) -Mike -- To unsubscribe from this list: send the line unsubscribe kvm-ppc in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs
On 05/03/2013 03:01:26 PM, Caraman Mihai Claudiu-B02008 wrote: -Original Message- From: Wood Scott-B07421 Sent: Friday, May 03, 2013 9:05 PM To: Caraman Mihai Claudiu-B02008 Cc: kvm-ppc@vger.kernel.org; k...@vger.kernel.org; linuxppc- d...@lists.ozlabs.org; Caraman Mihai Claudiu-B02008 Subject: Re: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs The unresponsiveness has to do with the fact that arch_local_irq_restore() does not guarantees to hard enable interrupts. Could you elaborate? If the saved IRQ state was enabled, why wouldn't arch_local_irq_restore() hard-enable IRQs? The last thing it does is __hard_irq_enable(). if (!irq_happened) return; OK, so the problem is that we're not setting PACA_IRQ_HARD_DIS when we hard-disable interrupts? Where is the arch_local_irq_restore() instance you're talking about? ./arch/power/kernel/irq.c I meant the caller. :-P -Scott -- To unsubscribe from this list: send the line unsubscribe kvm-ppc in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs
-Original Message- From: Wood Scott-B07421 Sent: Friday, May 03, 2013 11:15 PM To: Caraman Mihai Claudiu-B02008 Cc: Wood Scott-B07421; kvm-ppc@vger.kernel.org; k...@vger.kernel.org; linuxppc-...@lists.ozlabs.org Subject: Re: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs The unresponsiveness has to do with the fact that arch_local_irq_restore() does not guarantees to hard enable interrupts. Could you elaborate? If the saved IRQ state was enabled, why wouldn't arch_local_irq_restore() hard-enable IRQs? The last thing it does is __hard_irq_enable(). if (!irq_happened) return; OK, so the problem is that we're not setting PACA_IRQ_HARD_DIS when we hard-disable interrupts? We enter guest with local_irq_disable() which means soft disabled, when do we hard-disable interrupts? If we follow host exception handlers model they set PACA_IRQ_EE/DEC/DBELL but not PACA_IRQ_HARD_DIS. Can you give it a try to see how KVM behaves with PACA_IRQ_HARD_DIS? I can't do it right now. Where is the arch_local_irq_restore() instance you're talking about? ./arch/power/kernel/irq.c I meant the caller. :-P ./arch/powerpc/include/asm/hw_irq.h 55static inline unsigned long arch_local_irq_disable(void) 56{ 57unsigned long flags, zero; 58 59asm volatile( 60li %1,0; lbz %0,%2(13); stb %1,%2(13) 61: =r (flags), =r (zero) 62: i (offsetof(struct paca_struct, soft_enabled)) 63: memory); 64 65return flags; 66} 67 68extern void arch_local_irq_restore(unsigned long); 69 70static inline void arch_local_irq_enable(void) 71{ 72arch_local_irq_restore(1); 73} -Mike -- To unsubscribe from this list: send the line unsubscribe kvm-ppc in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs
On Fri, 2013-05-03 at 18:24 +0200, Alexander Graf wrote: There is no reason to exit guest with soft_enabled == 1, a local_irq_enable() call will do this for us so get rid of kvmppc_layz_ee() calls. With this fix we eliminate irqs_disabled() warnings and some guest and host hangs revealed under stress tests, but guests still exhibit some unresponsiveness. The unresponsiveness has to do with the fact that arch_local_irq_restore() does not guarantees to hard enable interrupts. To do so replace exception function calls like timer_interrupt() with irq_happened flags. The local_irq_enable() call takes care of replaying them and lets the interrupts hard enabled. Signed-off-by: Mihai Caraman mihai.cara...@freescale.com Ben, could you please review? That does look like the right thing to do indeed. Acked-by: Benjamin Herrenschmidt b...@kernel.crashing.org Cheers, Ben. -- To unsubscribe from this list: send the line unsubscribe kvm-ppc in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs
On 05/03/2013 03:56:47 PM, Caraman Mihai Claudiu-B02008 wrote: -Original Message- From: Wood Scott-B07421 Sent: Friday, May 03, 2013 11:15 PM To: Caraman Mihai Claudiu-B02008 Cc: Wood Scott-B07421; kvm-ppc@vger.kernel.org; k...@vger.kernel.org; linuxppc-...@lists.ozlabs.org Subject: Re: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs The unresponsiveness has to do with the fact that arch_local_irq_restore() does not guarantees to hard enable interrupts. Could you elaborate? If the saved IRQ state was enabled, why wouldn't arch_local_irq_restore() hard-enable IRQs? The last thing it does is __hard_irq_enable(). if (!irq_happened) return; OK, so the problem is that we're not setting PACA_IRQ_HARD_DIS when we hard-disable interrupts? We enter guest with local_irq_disable() which means soft disabled, Hmm... I don't see any obvious breakage from that, but it makes me nervous. I'd be more comfortable if we just hard-disabled interrupts there. when do we hard-disable interrupts? Interrupts will be hard-disabled when we take an exception to exit guest state. If we follow host exception handlers model they set PACA_IRQ_EE/DEC/DBELL but not PACA_IRQ_HARD_DIS. Can you give it a try to see how KVM behaves with PACA_IRQ_HARD_DIS? I can't do it right now. I replaced the two calls to kvmppc_lazy_ee_enable() with calls to hard_irq_disable(), and it seems to be working fine. Where is the arch_local_irq_restore() instance you're talking about? ./arch/power/kernel/irq.c I meant the caller. :-P ./arch/powerpc/include/asm/hw_irq.h 55static inline unsigned long arch_local_irq_disable(void) 56{ 57unsigned long flags, zero; 58 59asm volatile( 60li %1,0; lbz %0,%2(13); stb %1,%2(13) 61: =r (flags), =r (zero) 62: i (offsetof(struct paca_struct, soft_enabled)) 63: memory); 64 65return flags; 66} 67 68extern void arch_local_irq_restore(unsigned long); 69 70static inline void arch_local_irq_enable(void) 71{ 72arch_local_irq_restore(1); 73} Sigh. I meant the real caller, who's calling local_irq_restore(). -Scott -- To unsubscribe from this list: send the line unsubscribe kvm-ppc in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs
-Original Message- From: Wood Scott-B07421 Sent: Saturday, May 04, 2013 1:07 AM To: Caraman Mihai Claudiu-B02008 Cc: Wood Scott-B07421; kvm-ppc@vger.kernel.org; k...@vger.kernel.org; linuxppc-...@lists.ozlabs.org Subject: Re: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs I replaced the two calls to kvmppc_lazy_ee_enable() with calls to hard_irq_disable(), and it seems to be working fine. Please take a look on 'KVM: PPC64: booke: Hard disable interrupts when entering guest' RFC thread and see if your solution addresses Ben's comments. Where is the arch_local_irq_restore() instance you're talking about? ./arch/power/kernel/irq.c I meant the caller. :-P ./arch/powerpc/include/asm/hw_irq.h 55static inline unsigned long arch_local_irq_disable(void) 56{ 57unsigned long flags, zero; 58 59asm volatile( 60li %1,0; lbz %0,%2(13); stb %1,%2(13) 61: =r (flags), =r (zero) 62: i (offsetof(struct paca_struct, soft_enabled)) 63: memory); 64 65return flags; 66} 67 68extern void arch_local_irq_restore(unsigned long); 69 70static inline void arch_local_irq_enable(void) 71{ 72arch_local_irq_restore(1); 73} Sigh. I meant the real caller, who's calling local_irq_restore(). I'm not sure what you mean, arch_local_irq_restore() is called indirectly by local_irq_enable() in our case from handle_exit(). -Mike -- To unsubscribe from this list: send the line unsubscribe kvm-ppc in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs
On 05/03/2013 05:59:32 PM, Caraman Mihai Claudiu-B02008 wrote: -Original Message- From: Wood Scott-B07421 Sent: Saturday, May 04, 2013 1:07 AM To: Caraman Mihai Claudiu-B02008 Cc: Wood Scott-B07421; kvm-ppc@vger.kernel.org; k...@vger.kernel.org; linuxppc-...@lists.ozlabs.org Subject: Re: [PATCH] KVM: PPC: Book3E 64: Fix IRQs warnings and hangs I replaced the two calls to kvmppc_lazy_ee_enable() with calls to hard_irq_disable(), and it seems to be working fine. Please take a look on 'KVM: PPC64: booke: Hard disable interrupts when entering guest' RFC thread and see if your solution addresses Ben's comments. My original one didn't (there was a race if an interrupt comes in between soft-disabling and hard-disabling, it wouldn't be received until the guest exits for some other reason). Instead, I turned the local_irq_disable() into hard_irq_disable() plus trace_hardirqs_off(). This worked without warnings. -Scott -- To unsubscribe from this list: send the line unsubscribe kvm-ppc in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 50921] kvm hangs booting Windows 2000
https://bugzilla.kernel.org/show_bug.cgi?id=50921 --- Comment #21 from Gleb g...@redhat.com 2013-02-03 08:43:20 --- It is queued for 3.8. -- Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 50921] kvm hangs booting Windows 2000
https://bugzilla.kernel.org/show_bug.cgi?id=50921 --- Comment #22 from Gleb g...@redhat.com 2013-02-03 08:48:16 --- (In reply to comment #21) It is queued for 3.8. Sorry, for 3.9 -- Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 50921] kvm hangs booting Windows 2000
https://bugzilla.kernel.org/show_bug.cgi?id=50921 --- Comment #20 from Lucio Crusca lu...@sulweb.org 2013-02-02 22:32:39 --- Did this fix go into vanilla kernels? Is 3.7.5 patched? -- Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: qemu-kvm hangs at start up under 3.8.0-rc3-00074-gb719f43 (works with CONFIG_LOCKDEP)
On 01/14/2013 01:24 PM, Andrew Clayton wrote: On Mon, 14 Jan 2013 15:27:36 +0200, Gleb Natapov wrote: On Sun, Jan 13, 2013 at 10:29:58PM +, Andrew Clayton wrote: When running qemu-kvm under 64but Fedora 16 under current 3.8, it just hangs at start up. Dong a ps -ef hangs the process at the point where it would display the qemu process (trying to list the qemu-kvm /proc pid directory contents just hangs ls). I also noticed some other weirdness at this point like Firefox hanging for many seconds at a time and increasing load average. The qemu command I was trying to run was $ qemu-kvm -m 512 -smp 2 -vga vmware -k en-gb -drive file=/home/andrew/machines/qemu/f16-i386.img,if=virtio Here's the last few lines of a strace on it at start up. open(/home/andrew/machines/qemu/f16-i386.img, O_RDWR|O_DSYNC|O_CLOEXEC) = 8 lseek(8, 0, SEEK_END) = 9100722176 pread(8, QFI\373\0\0\0\2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\20\0\0\0\2\200\0\0\0..., 512, 0) = 512 pread(8, \200\0\0\0\0\4\0\0\200\0\0\0\0\10\0\0\200\0\0\0\2\210\0\0\200\0\0\0\2\233\0\0..., 512, 65536) = 512 brk(0) = 0x7faf12db brk(0x7faf12ddd000 It's hanging in that brk syscall. The load average also then starts to increase. However. I can make it run fine, if I enable CONFIG_LOCKDEP. But the only thing in dmesg I get is the usual. kvm: SMP vm created on host with unstable TSC; guest TSC will not be reliable I've attached both working and non-working .configs. The only difference being the lock checking enabled in config.good. The most recent kernel I had it working in was 3.7.0 System is a Quad Core Intel running 64bit Fedora 16. Can you run echo t /proc/sysrq-trigger and see where it hangs? Here you go, here's the bash process, qemu and a kvm bit. (From the above command) bashS 88013b2b0d00 0 3203 3133 0x 880114dabe58 0082 800113558065 880114dabfd8 880114dabfd8 4000 88013b0c5b00 88013b2b0d00 880114dabd88 8109067d ea0004536670 ea0004536640 Call Trace: [8109067d] ? default_wake_function+0xd/0x10 [8108a315] ? atomic_notifier_call_chain+0x15/0x20 [8133d84f] ? tty_get_pgrp+0x3f/0x50 [810819ac] ? pid_vnr+0x2c/0x30 [8133fe54] ? tty_ioctl+0x7b4/0xbd0 [8106bf62] ? wait_consider_task+0x102/0xaf0 [815c00e4] schedule+0x24/0x70 [8106cb24] do_wait+0x1d4/0x200 [8106d9cb] sys_wait4+0x9b/0xf0 [8106b9f0] ? task_stopped_code+0x50/0x50 [815c1ad2] system_call_fastpath+0x16/0x1b qemu-kvmD 88011ab8c8b8 0 3345 3203 0x 880112129cd8 0082 880112129c50 880112129fd8 880112129fd8 4000 88013b04ce00 880139da1a00 000280da 880112129d38 810d3300 Call Trace: [810d3300] ? __alloc_pages_nodemask+0xf0/0x7c0 [811273c6] ? touch_atime+0x66/0x170 [810cdabf] ? generic_file_aio_read+0x5bf/0x730 [815c00e4] schedule+0x24/0x70 [815c0cdd] rwsem_down_failed_common+0xbd/0x150 [815c0da3] rwsem_down_write_failed+0x13/0x15 [812d1be3] call_rwsem_down_write_failed+0x13/0x20 [815bf4dd] ? down_write+0x2d/0x34 [810f0724] vma_adjust+0xe4/0x610 [810f0fa4] vma_merge+0x1b4/0x270 [810f1fa6] do_brk+0x196/0x330 [810f2217] sys_brk+0xd7/0x130 [815c1ad2] system_call_fastpath+0x16/0x1b This looks like qemu-kvm getting stuck trying to get the anon_vma lock. That leads to the obvious question: what is holding the lock, and/or failed to release it? Do you have any other (qemu-kvm?) processes on your system that have any code in the VM (or strace/ptrace/...) in the backtrace, that might be holding this lock? Do you have anything in your dmesg showing threads that had a BUG_ON (and exited) while holding the lock? -- All rights reversed -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: qemu-kvm hangs at start up under 3.8.0-rc3-00074-gb719f43 (works with CONFIG_LOCKDEP)
On Tue, 15 Jan 2013 11:48:39 -0500, Rik van Riel wrote: On 01/14/2013 01:24 PM, Andrew Clayton wrote: [snip] bashS 88013b2b0d00 0 3203 3133 0x 880114dabe58 0082 800113558065 880114dabfd8 880114dabfd8 4000 88013b0c5b00 88013b2b0d00 880114dabd88 8109067d ea0004536670 ea0004536640 Call Trace: [8109067d] ? default_wake_function+0xd/0x10 [8108a315] ? atomic_notifier_call_chain+0x15/0x20 [8133d84f] ? tty_get_pgrp+0x3f/0x50 [810819ac] ? pid_vnr+0x2c/0x30 [8133fe54] ? tty_ioctl+0x7b4/0xbd0 [8106bf62] ? wait_consider_task+0x102/0xaf0 [815c00e4] schedule+0x24/0x70 [8106cb24] do_wait+0x1d4/0x200 [8106d9cb] sys_wait4+0x9b/0xf0 [8106b9f0] ? task_stopped_code+0x50/0x50 [815c1ad2] system_call_fastpath+0x16/0x1b qemu-kvmD 88011ab8c8b8 0 3345 3203 0x 880112129cd8 0082 880112129c50 880112129fd8 880112129fd8 4000 88013b04ce00 880139da1a00 000280da 880112129d38 810d3300 Call Trace: [810d3300] ? __alloc_pages_nodemask+0xf0/0x7c0 [811273c6] ? touch_atime+0x66/0x170 [810cdabf] ? generic_file_aio_read+0x5bf/0x730 [815c00e4] schedule+0x24/0x70 [815c0cdd] rwsem_down_failed_common+0xbd/0x150 [815c0da3] rwsem_down_write_failed+0x13/0x15 [812d1be3] call_rwsem_down_write_failed+0x13/0x20 [815bf4dd] ? down_write+0x2d/0x34 [810f0724] vma_adjust+0xe4/0x610 [810f0fa4] vma_merge+0x1b4/0x270 [810f1fa6] do_brk+0x196/0x330 [810f2217] sys_brk+0xd7/0x130 [815c1ad2] system_call_fastpath+0x16/0x1b This looks like qemu-kvm getting stuck trying to get the anon_vma lock. That leads to the obvious question: what is holding the lock, and/or failed to release it? Do you have any other (qemu-kvm?) processes on your system that have any code in the VM (or strace/ptrace/...) in the backtrace, that might be holding this lock? I don't think so. The above was done having just logged into gnome-shell and opened up a couple of gnome-terminals. Do you have anything in your dmesg showing threads that had a BUG_ON (and exited) while holding the lock? I never noticed anything like that. The interesting thing is that if I use basically the same kernel but with CONFIG_LOCKDEP enabled, it works fine. Hmm, I wonder if it's the same as this? http://lkml.indiana.edu/hypermail/linux/kernel/1301.1/02810.html When I get home, I'll try a kernel with those two patches reverted. They seem to be 572043c90db65b45a4efd959db7458edcf6411ad 1b963c81b14509e330e0fe3218b645ece2738dc5 in Linus' tree. CC'd Jiri as well. Cheers, Andrew -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: qemu-kvm hangs at start up under 3.8.0-rc3-00074-gb719f43 (works with CONFIG_LOCKDEP)
On Tue, 15 Jan 2013 17:17:56 +, Andrew Clayton wrote: On Tue, 15 Jan 2013 11:48:39 -0500, Rik van Riel wrote: On 01/14/2013 01:24 PM, Andrew Clayton wrote: [snip] bashS 88013b2b0d00 0 3203 3133 0x 880114dabe58 0082 800113558065 880114dabfd8 880114dabfd8 4000 88013b0c5b00 88013b2b0d00 880114dabd88 8109067d ea0004536670 ea0004536640 Call Trace: [8109067d] ? default_wake_function+0xd/0x10 [8108a315] ? atomic_notifier_call_chain+0x15/0x20 [8133d84f] ? tty_get_pgrp+0x3f/0x50 [810819ac] ? pid_vnr+0x2c/0x30 [8133fe54] ? tty_ioctl+0x7b4/0xbd0 [8106bf62] ? wait_consider_task+0x102/0xaf0 [815c00e4] schedule+0x24/0x70 [8106cb24] do_wait+0x1d4/0x200 [8106d9cb] sys_wait4+0x9b/0xf0 [8106b9f0] ? task_stopped_code+0x50/0x50 [815c1ad2] system_call_fastpath+0x16/0x1b qemu-kvmD 88011ab8c8b8 0 3345 3203 0x 880112129cd8 0082 880112129c50 880112129fd8 880112129fd8 4000 88013b04ce00 880139da1a00 000280da 880112129d38 810d3300 Call Trace: [810d3300] ? __alloc_pages_nodemask+0xf0/0x7c0 [811273c6] ? touch_atime+0x66/0x170 [810cdabf] ? generic_file_aio_read+0x5bf/0x730 [815c00e4] schedule+0x24/0x70 [815c0cdd] rwsem_down_failed_common+0xbd/0x150 [815c0da3] rwsem_down_write_failed+0x13/0x15 [812d1be3] call_rwsem_down_write_failed+0x13/0x20 [815bf4dd] ? down_write+0x2d/0x34 [810f0724] vma_adjust+0xe4/0x610 [810f0fa4] vma_merge+0x1b4/0x270 [810f1fa6] do_brk+0x196/0x330 [810f2217] sys_brk+0xd7/0x130 [815c1ad2] system_call_fastpath+0x16/0x1b This looks like qemu-kvm getting stuck trying to get the anon_vma lock. That leads to the obvious question: what is holding the lock, and/or failed to release it? Do you have any other (qemu-kvm?) processes on your system that have any code in the VM (or strace/ptrace/...) in the backtrace, that might be holding this lock? I don't think so. The above was done having just logged into gnome-shell and opened up a couple of gnome-terminals. Do you have anything in your dmesg showing threads that had a BUG_ON (and exited) while holding the lock? I never noticed anything like that. The interesting thing is that if I use basically the same kernel but with CONFIG_LOCKDEP enabled, it works fine. Hmm, I wonder if it's the same as this? http://lkml.indiana.edu/hypermail/linux/kernel/1301.1/02810.html When I get home, I'll try a kernel with those two patches reverted. They seem to be 572043c90db65b45a4efd959db7458edcf6411ad 1b963c81b14509e330e0fe3218b645ece2738dc5 Just checked the issue is still there in Linus' latest, 3.8.0-rc3-00293-g406089d, it is. However, building a kernel with those two commits reverted does indeed get it going again. Andrew -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: qemu-kvm hangs at start up under 3.8.0-rc3-00074-gb719f43 (works with CONFIG_LOCKDEP)
On Tue, 15 Jan 2013, Andrew Clayton wrote: bashS 88013b2b0d00 0 3203 3133 0x 880114dabe58 0082 800113558065 880114dabfd8 880114dabfd8 4000 88013b0c5b00 88013b2b0d00 880114dabd88 8109067d ea0004536670 ea0004536640 Call Trace: [8109067d] ? default_wake_function+0xd/0x10 [8108a315] ? atomic_notifier_call_chain+0x15/0x20 [8133d84f] ? tty_get_pgrp+0x3f/0x50 [810819ac] ? pid_vnr+0x2c/0x30 [8133fe54] ? tty_ioctl+0x7b4/0xbd0 [8106bf62] ? wait_consider_task+0x102/0xaf0 [815c00e4] schedule+0x24/0x70 [8106cb24] do_wait+0x1d4/0x200 [8106d9cb] sys_wait4+0x9b/0xf0 [8106b9f0] ? task_stopped_code+0x50/0x50 [815c1ad2] system_call_fastpath+0x16/0x1b qemu-kvmD 88011ab8c8b8 0 3345 3203 0x 880112129cd8 0082 880112129c50 880112129fd8 880112129fd8 4000 88013b04ce00 880139da1a00 000280da 880112129d38 810d3300 Call Trace: [810d3300] ? __alloc_pages_nodemask+0xf0/0x7c0 [811273c6] ? touch_atime+0x66/0x170 [810cdabf] ? generic_file_aio_read+0x5bf/0x730 [815c00e4] schedule+0x24/0x70 [815c0cdd] rwsem_down_failed_common+0xbd/0x150 [815c0da3] rwsem_down_write_failed+0x13/0x15 [812d1be3] call_rwsem_down_write_failed+0x13/0x20 [815bf4dd] ? down_write+0x2d/0x34 [810f0724] vma_adjust+0xe4/0x610 [810f0fa4] vma_merge+0x1b4/0x270 [810f1fa6] do_brk+0x196/0x330 [810f2217] sys_brk+0xd7/0x130 [815c1ad2] system_call_fastpath+0x16/0x1b This looks like qemu-kvm getting stuck trying to get the anon_vma lock. That leads to the obvious question: what is holding the lock, and/or failed to release it? Do you have any other (qemu-kvm?) processes on your system that have any code in the VM (or strace/ptrace/...) in the backtrace, that might be holding this lock? I don't think so. The above was done having just logged into gnome-shell and opened up a couple of gnome-terminals. Do you have anything in your dmesg showing threads that had a BUG_ON (and exited) while holding the lock? I never noticed anything like that. The interesting thing is that if I use basically the same kernel but with CONFIG_LOCKDEP enabled, it works fine. Thorough and careful review and analysis revealed that the rootcause very likely is that I am a complete nitwit. Could you please try the patch below and report backt? Thanks. From: Jiri Kosina jkos...@suse.cz Subject: [PATCH] lockdep, rwsem: fix down_write_nest_lock() if !CONFIG_DEBUG_LOCK_ALLOC Commit 1b963c81b1 (lockdep, rwsem: provide down_write_nest_lock()) contains a bug in a codepath when CONFIG_DEBUG_LOCK_ALLOC is disabled, which causes down_read() to be called instead of down_write() by mistake on such configurations. Fix that. Signed-off-by: Jiri Kosina jkos...@suse.cz --- include/linux/rwsem.h |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/include/linux/rwsem.h b/include/linux/rwsem.h index 413cc11..8da67d6 100644 --- a/include/linux/rwsem.h +++ b/include/linux/rwsem.h @@ -135,7 +135,7 @@ do { \ #else # define down_read_nested(sem, subclass) down_read(sem) -# define down_write_nest_lock(sem, nest_lock) down_read(sem) +# define down_write_nest_lock(sem, nest_lock) down_write(sem) # define down_write_nested(sem, subclass) down_write(sem) #endif -- Jiri Kosina SUSE Labs -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: qemu-kvm hangs at start up under 3.8.0-rc3-00074-gb719f43 (works with CONFIG_LOCKDEP)
On Tue, 15 Jan 2013 19:41:32 +0100 (CET), Jiri Kosina wrote: [snip] Could you please try the patch below and report backt? Thanks. From: Jiri Kosina jkos...@suse.cz Subject: [PATCH] lockdep, rwsem: fix down_write_nest_lock() if !CONFIG_DEBUG_LOCK_ALLOC Commit 1b963c81b1 (lockdep, rwsem: provide down_write_nest_lock()) contains a bug in a codepath when CONFIG_DEBUG_LOCK_ALLOC is disabled, which causes down_read() to be called instead of down_write() by mistake on such configurations. Fix that. Signed-off-by: Jiri Kosina jkos...@suse.cz --- include/linux/rwsem.h |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/include/linux/rwsem.h b/include/linux/rwsem.h index 413cc11..8da67d6 100644 --- a/include/linux/rwsem.h +++ b/include/linux/rwsem.h @@ -135,7 +135,7 @@ do { \ #else # define down_read_nested(sem, subclass) down_read(sem) -# define down_write_nest_lock(sem, nest_lock) down_read(sem) +# define down_write_nest_lock(sem, nest_lock) down_write(sem) # define down_write_nested(sem, subclass) down_write(sem) #endif Heh, indeed, that fix's it. Tested-by: Andrew Clayton and...@digital-domain.net Cheers, Andrew -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: qemu-kvm hangs at start up under 3.8.0-rc3-00074-gb719f43 (works with CONFIG_LOCKDEP)
On Sun, Jan 13, 2013 at 10:29:58PM +, Andrew Clayton wrote: When running qemu-kvm under 64but Fedora 16 under current 3.8, it just hangs at start up. Dong a ps -ef hangs the process at the point where it would display the qemu process (trying to list the qemu-kvm /proc pid directory contents just hangs ls). I also noticed some other weirdness at this point like Firefox hanging for many seconds at a time and increasing load average. The qemu command I was trying to run was $ qemu-kvm -m 512 -smp 2 -vga vmware -k en-gb -drive file=/home/andrew/machines/qemu/f16-i386.img,if=virtio Here's the last few lines of a strace on it at start up. open(/home/andrew/machines/qemu/f16-i386.img, O_RDWR|O_DSYNC|O_CLOEXEC) = 8 lseek(8, 0, SEEK_END) = 9100722176 pread(8, QFI\373\0\0\0\2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\20\0\0\0\2\200\0\0\0..., 512, 0) = 512 pread(8, \200\0\0\0\0\4\0\0\200\0\0\0\0\10\0\0\200\0\0\0\2\210\0\0\200\0\0\0\2\233\0\0..., 512, 65536) = 512 brk(0) = 0x7faf12db brk(0x7faf12ddd000 It's hanging in that brk syscall. The load average also then starts to increase. However. I can make it run fine, if I enable CONFIG_LOCKDEP. But the only thing in dmesg I get is the usual. kvm: SMP vm created on host with unstable TSC; guest TSC will not be reliable I've attached both working and non-working .configs. The only difference being the lock checking enabled in config.good. The most recent kernel I had it working in was 3.7.0 System is a Quad Core Intel running 64bit Fedora 16. Can you run echo t /proc/sysrq-trigger and see where it hangs? -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: qemu-kvm hangs at start up under 3.8.0-rc3-00074-gb719f43 (works with CONFIG_LOCKDEP)
On Mon, 14 Jan 2013 15:27:36 +0200, Gleb Natapov wrote: On Sun, Jan 13, 2013 at 10:29:58PM +, Andrew Clayton wrote: When running qemu-kvm under 64but Fedora 16 under current 3.8, it just hangs at start up. Dong a ps -ef hangs the process at the point where it would display the qemu process (trying to list the qemu-kvm /proc pid directory contents just hangs ls). I also noticed some other weirdness at this point like Firefox hanging for many seconds at a time and increasing load average. The qemu command I was trying to run was $ qemu-kvm -m 512 -smp 2 -vga vmware -k en-gb -drive file=/home/andrew/machines/qemu/f16-i386.img,if=virtio Here's the last few lines of a strace on it at start up. open(/home/andrew/machines/qemu/f16-i386.img, O_RDWR|O_DSYNC|O_CLOEXEC) = 8 lseek(8, 0, SEEK_END) = 9100722176 pread(8, QFI\373\0\0\0\2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\20\0\0\0\2\200\0\0\0..., 512, 0) = 512 pread(8, \200\0\0\0\0\4\0\0\200\0\0\0\0\10\0\0\200\0\0\0\2\210\0\0\200\0\0\0\2\233\0\0..., 512, 65536) = 512 brk(0) = 0x7faf12db brk(0x7faf12ddd000 It's hanging in that brk syscall. The load average also then starts to increase. However. I can make it run fine, if I enable CONFIG_LOCKDEP. But the only thing in dmesg I get is the usual. kvm: SMP vm created on host with unstable TSC; guest TSC will not be reliable I've attached both working and non-working .configs. The only difference being the lock checking enabled in config.good. The most recent kernel I had it working in was 3.7.0 System is a Quad Core Intel running 64bit Fedora 16. Can you run echo t /proc/sysrq-trigger and see where it hangs? Here you go, here's the bash process, qemu and a kvm bit. (From the above command) bashS 88013b2b0d00 0 3203 3133 0x 880114dabe58 0082 800113558065 880114dabfd8 880114dabfd8 4000 88013b0c5b00 88013b2b0d00 880114dabd88 8109067d ea0004536670 ea0004536640 Call Trace: [8109067d] ? default_wake_function+0xd/0x10 [8108a315] ? atomic_notifier_call_chain+0x15/0x20 [8133d84f] ? tty_get_pgrp+0x3f/0x50 [810819ac] ? pid_vnr+0x2c/0x30 [8133fe54] ? tty_ioctl+0x7b4/0xbd0 [8106bf62] ? wait_consider_task+0x102/0xaf0 [815c00e4] schedule+0x24/0x70 [8106cb24] do_wait+0x1d4/0x200 [8106d9cb] sys_wait4+0x9b/0xf0 [8106b9f0] ? task_stopped_code+0x50/0x50 [815c1ad2] system_call_fastpath+0x16/0x1b qemu-kvmD 88011ab8c8b8 0 3345 3203 0x 880112129cd8 0082 880112129c50 880112129fd8 880112129fd8 4000 88013b04ce00 880139da1a00 000280da 880112129d38 810d3300 Call Trace: [810d3300] ? __alloc_pages_nodemask+0xf0/0x7c0 [811273c6] ? touch_atime+0x66/0x170 [810cdabf] ? generic_file_aio_read+0x5bf/0x730 [815c00e4] schedule+0x24/0x70 [815c0cdd] rwsem_down_failed_common+0xbd/0x150 [815c0da3] rwsem_down_write_failed+0x13/0x15 [812d1be3] call_rwsem_down_write_failed+0x13/0x20 [815bf4dd] ? down_write+0x2d/0x34 [810f0724] vma_adjust+0xe4/0x610 [810f0fa4] vma_merge+0x1b4/0x270 [810f1fa6] do_brk+0x196/0x330 [810f2217] sys_brk+0xd7/0x130 [815c1ad2] system_call_fastpath+0x16/0x1b kvm-pit/3345S 0 3346 2 0x 880112149e68 0046 88013fd91340 880112149fd8 880112149fd8 4000 88013b04ce00 88013b2b6e80 880112149ea8 815bfb6c 880112149db8 880112149fd8 Call Trace: [815bfb6c] ? __schedule+0x2dc/0x830 [810904be] ? try_to_wake_up+0xbe/0x270 [8109067d] ? default_wake_function+0xd/0x10 [815c00e4] schedule+0x24/0x70 [81084975] kthread_worker_fn+0xb5/0x110 [810848c0] ? __init_kthread_worker+0x20/0x20 [810842fb] kthread+0xbb/0xc0 [81084240] ? __kthread_parkme+0x80/0x80 [815c1a2c] ret_from_fork+0x7c/0xb0 [81084240] ? __kthread_parkme+0x80/0x80 Cheers, Andrew -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: qemu-kvm hangs at start up under 3.8.0-rc3-00074-gb719f43 (works with CONFIG_LOCKDEP)
Copying linux-mm. On Mon, Jan 14, 2013 at 06:24:49PM +, Andrew Clayton wrote: On Mon, 14 Jan 2013 15:27:36 +0200, Gleb Natapov wrote: On Sun, Jan 13, 2013 at 10:29:58PM +, Andrew Clayton wrote: When running qemu-kvm under 64but Fedora 16 under current 3.8, it just hangs at start up. Dong a ps -ef hangs the process at the point where it would display the qemu process (trying to list the qemu-kvm /proc pid directory contents just hangs ls). I also noticed some other weirdness at this point like Firefox hanging for many seconds at a time and increasing load average. The qemu command I was trying to run was $ qemu-kvm -m 512 -smp 2 -vga vmware -k en-gb -drive file=/home/andrew/machines/qemu/f16-i386.img,if=virtio Here's the last few lines of a strace on it at start up. open(/home/andrew/machines/qemu/f16-i386.img, O_RDWR|O_DSYNC|O_CLOEXEC) = 8 lseek(8, 0, SEEK_END) = 9100722176 pread(8, QFI\373\0\0\0\2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\20\0\0\0\2\200\0\0\0..., 512, 0) = 512 pread(8, \200\0\0\0\0\4\0\0\200\0\0\0\0\10\0\0\200\0\0\0\2\210\0\0\200\0\0\0\2\233\0\0..., 512, 65536) = 512 brk(0) = 0x7faf12db brk(0x7faf12ddd000 It's hanging in that brk syscall. The load average also then starts to increase. However. I can make it run fine, if I enable CONFIG_LOCKDEP. But the only thing in dmesg I get is the usual. kvm: SMP vm created on host with unstable TSC; guest TSC will not be reliable I've attached both working and non-working .configs. The only difference being the lock checking enabled in config.good. The most recent kernel I had it working in was 3.7.0 System is a Quad Core Intel running 64bit Fedora 16. Can you run echo t /proc/sysrq-trigger and see where it hangs? Here you go, here's the bash process, qemu and a kvm bit. (From the above command) bashS 88013b2b0d00 0 3203 3133 0x 880114dabe58 0082 800113558065 880114dabfd8 880114dabfd8 4000 88013b0c5b00 88013b2b0d00 880114dabd88 8109067d ea0004536670 ea0004536640 Call Trace: [8109067d] ? default_wake_function+0xd/0x10 [8108a315] ? atomic_notifier_call_chain+0x15/0x20 [8133d84f] ? tty_get_pgrp+0x3f/0x50 [810819ac] ? pid_vnr+0x2c/0x30 [8133fe54] ? tty_ioctl+0x7b4/0xbd0 [8106bf62] ? wait_consider_task+0x102/0xaf0 [815c00e4] schedule+0x24/0x70 [8106cb24] do_wait+0x1d4/0x200 [8106d9cb] sys_wait4+0x9b/0xf0 [8106b9f0] ? task_stopped_code+0x50/0x50 [815c1ad2] system_call_fastpath+0x16/0x1b qemu-kvmD 88011ab8c8b8 0 3345 3203 0x 880112129cd8 0082 880112129c50 880112129fd8 880112129fd8 4000 88013b04ce00 880139da1a00 000280da 880112129d38 810d3300 Call Trace: [810d3300] ? __alloc_pages_nodemask+0xf0/0x7c0 [811273c6] ? touch_atime+0x66/0x170 [810cdabf] ? generic_file_aio_read+0x5bf/0x730 [815c00e4] schedule+0x24/0x70 [815c0cdd] rwsem_down_failed_common+0xbd/0x150 [815c0da3] rwsem_down_write_failed+0x13/0x15 [812d1be3] call_rwsem_down_write_failed+0x13/0x20 [815bf4dd] ? down_write+0x2d/0x34 [810f0724] vma_adjust+0xe4/0x610 [810f0fa4] vma_merge+0x1b4/0x270 [810f1fa6] do_brk+0x196/0x330 [810f2217] sys_brk+0xd7/0x130 [815c1ad2] system_call_fastpath+0x16/0x1b kvm-pit/3345S 0 3346 2 0x 880112149e68 0046 88013fd91340 880112149fd8 880112149fd8 4000 88013b04ce00 88013b2b6e80 880112149ea8 815bfb6c 880112149db8 880112149fd8 Call Trace: [815bfb6c] ? __schedule+0x2dc/0x830 [810904be] ? try_to_wake_up+0xbe/0x270 [8109067d] ? default_wake_function+0xd/0x10 [815c00e4] schedule+0x24/0x70 [81084975] kthread_worker_fn+0xb5/0x110 [810848c0] ? __init_kthread_worker+0x20/0x20 [810842fb] kthread+0xbb/0xc0 [81084240] ? __kthread_parkme+0x80/0x80 [815c1a2c] ret_from_fork+0x7c/0xb0 [81084240] ? __kthread_parkme+0x80/0x80 Cheers, Andrew -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
qemu-kvm hangs at start up under 3.8.0-rc3-00074-gb719f43 (works with CONFIG_LOCKDEP)
When running qemu-kvm under 64but Fedora 16 under current 3.8, it just hangs at start up. Dong a ps -ef hangs the process at the point where it would display the qemu process (trying to list the qemu-kvm /proc pid directory contents just hangs ls). I also noticed some other weirdness at this point like Firefox hanging for many seconds at a time and increasing load average. The qemu command I was trying to run was $ qemu-kvm -m 512 -smp 2 -vga vmware -k en-gb -drive file=/home/andrew/machines/qemu/f16-i386.img,if=virtio Here's the last few lines of a strace on it at start up. open(/home/andrew/machines/qemu/f16-i386.img, O_RDWR|O_DSYNC|O_CLOEXEC) = 8 lseek(8, 0, SEEK_END) = 9100722176 pread(8, QFI\373\0\0\0\2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\20\0\0\0\2\200\0\0\0..., 512, 0) = 512 pread(8, \200\0\0\0\0\4\0\0\200\0\0\0\0\10\0\0\200\0\0\0\2\210\0\0\200\0\0\0\2\233\0\0..., 512, 65536) = 512 brk(0) = 0x7faf12db brk(0x7faf12ddd000 It's hanging in that brk syscall. The load average also then starts to increase. However. I can make it run fine, if I enable CONFIG_LOCKDEP. But the only thing in dmesg I get is the usual. kvm: SMP vm created on host with unstable TSC; guest TSC will not be reliable I've attached both working and non-working .configs. The only difference being the lock checking enabled in config.good. The most recent kernel I had it working in was 3.7.0 System is a Quad Core Intel running 64bit Fedora 16. Cheers, Andrew config.bad.gz Description: GNU Zip compressed data config.good.gz Description: GNU Zip compressed data
RE: Installation of Windows 8 hangs with KVM
-Original Message- From: Gleb Natapov [mailto:g...@redhat.com] Sent: Monday, January 07, 2013 5:21 PM To: Ren, Yongjie Cc: Stefan Pietsch; kvm@vger.kernel.org Subject: Re: Installation of Windows 8 hangs with KVM On Mon, Jan 07, 2013 at 09:13:37AM +, Ren, Yongjie wrote: -Original Message- From: kvm-ow...@vger.kernel.org [mailto:kvm-ow...@vger.kernel.org] On Behalf Of Gleb Natapov Sent: Monday, January 07, 2013 4:54 PM To: Ren, Yongjie Cc: Stefan Pietsch; kvm@vger.kernel.org Subject: Re: Installation of Windows 8 hangs with KVM On Mon, Jan 07, 2013 at 08:38:59AM +, Ren, Yongjie wrote: -Original Message- From: kvm-ow...@vger.kernel.org [mailto:kvm-ow...@vger.kernel.org] On Behalf Of Stefan Pietsch Sent: Monday, January 07, 2013 2:25 AM To: Gleb Natapov Cc: kvm@vger.kernel.org Subject: Re: Installation of Windows 8 hangs with KVM * Gleb Natapov g...@redhat.com [2013-01-06 11:11]: On Fri, Jan 04, 2013 at 10:58:33PM +0100, Stefan Pietsch wrote: Hi all, when I run KVM with this command the Windows 8 installation stops with error code 0x005D: kvm -m 1024 -hda win8.img -cdrom windows_8_x86.iso After adding the option -cpu host the installation proceeds to a black screen and hangs. With Virtualbox the installation succeeds. The host CPU is an Intel Core Duo L2400. Do you have any suggestions? What is your kernel/qemu version? I'm using Debian unstable. qemu-kvm 1.1.2+dfsg-3 Linux version 3.2.0-4-686-pae (debian-ker...@lists.debian.org) (gcc version 4.6.3 (Debian 4.6.3-14) ) #1 SMP Debian 3.2.35-2 you met issue only for 32bit Win8 (not 64 bit Win8), right? I think it's the same issue as the below bug I reported. https://bugs.launchpad.net/qemu/+bug/1007269 You can try with '-cpu coreduo' or '-cpu core2duo' in qemu-kvm command line. This should be a known issue which is caused by missing 'SEP' CPU flag. See another bug in Redhat bugzilla. https://bugzilla.redhat.com/show_bug.cgi?id=821741 That was RHEL kernel bug. Doubt Debian one has it. I don't think so. It should be a qemu bug (also described in that RHEL bugzilla). https://bugzilla.redhat.com/show_bug.cgi?id=821463 is the kernel one. In my SandyBridge platform, 32bit Win8 guest can boot up with '-cpu SandyBridge,+sep' in qemu-kvm CLI. But it can't boot up with '-cpu SandyBridge'. Which qemu version? Master has sep in SandyBridge definition. In any case -cpu host should have sep enabled. Oh, sorry for my bad memory. I'm not sure it's about SEP. I shouldn't have taken 'SandyBridge' model as an example because 32bit Win8 can boot up on 'SandyBridge' model. 'qemu64' is the default CPU model in QEMU/KVM on x86-64, but its 'family number' definition is not accurate. The vendor of 'qemu64' is defined as 'AMD'. For AMD processors, AMD K8 CPU (e.g. AMD Family 15) firstly introduced SSE3 instruction set. 'qemu64' already has 'CPUID_EXT_SSE3' in its ext_features, but its family number is only 6 (not equal or bigger than 15). If I update the family number of 'qemu64' to '15', 32bit Win8 guest can boot up with the default 'qemu64' CPU model. 32bit Win8 may have some requirements of a 64bit CPU model. The following patch can be a fix for 'qemu64' definition. diff --git a/target-i386/cpu.c b/target-i386/cpu.c index 78bd61e..89b203b 100644 --- a/target-i386/cpu.c +++ b/target-i386/cpu.c @@ -363,7 +363,7 @@ static x86_def_t builtin_x86_defs[] = { .vendor1 = CPUID_VENDOR_AMD_1, .vendor2 = CPUID_VENDOR_AMD_2, .vendor3 = CPUID_VENDOR_AMD_3, -.family = 6, +.family = 15, .model = 2, .stepping = 3, .features = PPRO_FEATURES | -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Installation of Windows 8 hangs with KVM
* Ren, Yongjie yongjie@intel.com [2013-01-07 09:38]: you met issue only for 32bit Win8 (not 64 bit Win8), right? I think it's the same issue as the below bug I reported. https://bugs.launchpad.net/qemu/+bug/1007269 You can try with '-cpu coreduo' or '-cpu core2duo' in qemu-kvm command line. Yes, I'm talking about 32bit Windows 8. The CPU does not support 64bit. Booting with -cpu coreduo or core2duo does not show the 5D error but a black screen after a few minutes. I'm not able to do a successful boot of the installation ISO with KVM. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: Installation of Windows 8 hangs with KVM
-Original Message- From: kvm-ow...@vger.kernel.org [mailto:kvm-ow...@vger.kernel.org] On Behalf Of Stefan Pietsch Sent: Monday, January 07, 2013 2:25 AM To: Gleb Natapov Cc: kvm@vger.kernel.org Subject: Re: Installation of Windows 8 hangs with KVM * Gleb Natapov g...@redhat.com [2013-01-06 11:11]: On Fri, Jan 04, 2013 at 10:58:33PM +0100, Stefan Pietsch wrote: Hi all, when I run KVM with this command the Windows 8 installation stops with error code 0x005D: kvm -m 1024 -hda win8.img -cdrom windows_8_x86.iso After adding the option -cpu host the installation proceeds to a black screen and hangs. With Virtualbox the installation succeeds. The host CPU is an Intel Core Duo L2400. Do you have any suggestions? What is your kernel/qemu version? I'm using Debian unstable. qemu-kvm 1.1.2+dfsg-3 Linux version 3.2.0-4-686-pae (debian-ker...@lists.debian.org) (gcc version 4.6.3 (Debian 4.6.3-14) ) #1 SMP Debian 3.2.35-2 you met issue only for 32bit Win8 (not 64 bit Win8), right? I think it's the same issue as the below bug I reported. https://bugs.launchpad.net/qemu/+bug/1007269 You can try with '-cpu coreduo' or '-cpu core2duo' in qemu-kvm command line. This should be a known issue which is caused by missing 'SEP' CPU flag. See another bug in Redhat bugzilla. https://bugzilla.redhat.com/show_bug.cgi?id=821741 -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Installation of Windows 8 hangs with KVM
On Mon, Jan 07, 2013 at 08:38:59AM +, Ren, Yongjie wrote: -Original Message- From: kvm-ow...@vger.kernel.org [mailto:kvm-ow...@vger.kernel.org] On Behalf Of Stefan Pietsch Sent: Monday, January 07, 2013 2:25 AM To: Gleb Natapov Cc: kvm@vger.kernel.org Subject: Re: Installation of Windows 8 hangs with KVM * Gleb Natapov g...@redhat.com [2013-01-06 11:11]: On Fri, Jan 04, 2013 at 10:58:33PM +0100, Stefan Pietsch wrote: Hi all, when I run KVM with this command the Windows 8 installation stops with error code 0x005D: kvm -m 1024 -hda win8.img -cdrom windows_8_x86.iso After adding the option -cpu host the installation proceeds to a black screen and hangs. With Virtualbox the installation succeeds. The host CPU is an Intel Core Duo L2400. Do you have any suggestions? What is your kernel/qemu version? I'm using Debian unstable. qemu-kvm 1.1.2+dfsg-3 Linux version 3.2.0-4-686-pae (debian-ker...@lists.debian.org) (gcc version 4.6.3 (Debian 4.6.3-14) ) #1 SMP Debian 3.2.35-2 you met issue only for 32bit Win8 (not 64 bit Win8), right? I think it's the same issue as the below bug I reported. https://bugs.launchpad.net/qemu/+bug/1007269 You can try with '-cpu coreduo' or '-cpu core2duo' in qemu-kvm command line. This should be a known issue which is caused by missing 'SEP' CPU flag. See another bug in Redhat bugzilla. https://bugzilla.redhat.com/show_bug.cgi?id=821741 That was RHEL kernel bug. Doubt Debian one has it. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: Installation of Windows 8 hangs with KVM
-Original Message- From: kvm-ow...@vger.kernel.org [mailto:kvm-ow...@vger.kernel.org] On Behalf Of Gleb Natapov Sent: Monday, January 07, 2013 4:54 PM To: Ren, Yongjie Cc: Stefan Pietsch; kvm@vger.kernel.org Subject: Re: Installation of Windows 8 hangs with KVM On Mon, Jan 07, 2013 at 08:38:59AM +, Ren, Yongjie wrote: -Original Message- From: kvm-ow...@vger.kernel.org [mailto:kvm-ow...@vger.kernel.org] On Behalf Of Stefan Pietsch Sent: Monday, January 07, 2013 2:25 AM To: Gleb Natapov Cc: kvm@vger.kernel.org Subject: Re: Installation of Windows 8 hangs with KVM * Gleb Natapov g...@redhat.com [2013-01-06 11:11]: On Fri, Jan 04, 2013 at 10:58:33PM +0100, Stefan Pietsch wrote: Hi all, when I run KVM with this command the Windows 8 installation stops with error code 0x005D: kvm -m 1024 -hda win8.img -cdrom windows_8_x86.iso After adding the option -cpu host the installation proceeds to a black screen and hangs. With Virtualbox the installation succeeds. The host CPU is an Intel Core Duo L2400. Do you have any suggestions? What is your kernel/qemu version? I'm using Debian unstable. qemu-kvm 1.1.2+dfsg-3 Linux version 3.2.0-4-686-pae (debian-ker...@lists.debian.org) (gcc version 4.6.3 (Debian 4.6.3-14) ) #1 SMP Debian 3.2.35-2 you met issue only for 32bit Win8 (not 64 bit Win8), right? I think it's the same issue as the below bug I reported. https://bugs.launchpad.net/qemu/+bug/1007269 You can try with '-cpu coreduo' or '-cpu core2duo' in qemu-kvm command line. This should be a known issue which is caused by missing 'SEP' CPU flag. See another bug in Redhat bugzilla. https://bugzilla.redhat.com/show_bug.cgi?id=821741 That was RHEL kernel bug. Doubt Debian one has it. I don't think so. It should be a qemu bug (also described in that RHEL bugzilla). In my SandyBridge platform, 32bit Win8 guest can boot up with '-cpu SandyBridge,+sep' in qemu-kvm CLI. But it can't boot up with '-cpu SandyBridge'. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Installation of Windows 8 hangs with KVM
On Mon, Jan 07, 2013 at 09:13:37AM +, Ren, Yongjie wrote: -Original Message- From: kvm-ow...@vger.kernel.org [mailto:kvm-ow...@vger.kernel.org] On Behalf Of Gleb Natapov Sent: Monday, January 07, 2013 4:54 PM To: Ren, Yongjie Cc: Stefan Pietsch; kvm@vger.kernel.org Subject: Re: Installation of Windows 8 hangs with KVM On Mon, Jan 07, 2013 at 08:38:59AM +, Ren, Yongjie wrote: -Original Message- From: kvm-ow...@vger.kernel.org [mailto:kvm-ow...@vger.kernel.org] On Behalf Of Stefan Pietsch Sent: Monday, January 07, 2013 2:25 AM To: Gleb Natapov Cc: kvm@vger.kernel.org Subject: Re: Installation of Windows 8 hangs with KVM * Gleb Natapov g...@redhat.com [2013-01-06 11:11]: On Fri, Jan 04, 2013 at 10:58:33PM +0100, Stefan Pietsch wrote: Hi all, when I run KVM with this command the Windows 8 installation stops with error code 0x005D: kvm -m 1024 -hda win8.img -cdrom windows_8_x86.iso After adding the option -cpu host the installation proceeds to a black screen and hangs. With Virtualbox the installation succeeds. The host CPU is an Intel Core Duo L2400. Do you have any suggestions? What is your kernel/qemu version? I'm using Debian unstable. qemu-kvm 1.1.2+dfsg-3 Linux version 3.2.0-4-686-pae (debian-ker...@lists.debian.org) (gcc version 4.6.3 (Debian 4.6.3-14) ) #1 SMP Debian 3.2.35-2 you met issue only for 32bit Win8 (not 64 bit Win8), right? I think it's the same issue as the below bug I reported. https://bugs.launchpad.net/qemu/+bug/1007269 You can try with '-cpu coreduo' or '-cpu core2duo' in qemu-kvm command line. This should be a known issue which is caused by missing 'SEP' CPU flag. See another bug in Redhat bugzilla. https://bugzilla.redhat.com/show_bug.cgi?id=821741 That was RHEL kernel bug. Doubt Debian one has it. I don't think so. It should be a qemu bug (also described in that RHEL bugzilla). https://bugzilla.redhat.com/show_bug.cgi?id=821463 is the kernel one. In my SandyBridge platform, 32bit Win8 guest can boot up with '-cpu SandyBridge,+sep' in qemu-kvm CLI. But it can't boot up with '-cpu SandyBridge'. Which qemu version? Master has sep in SandyBridge definition. In any case -cpu host should have sep enabled. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Installation of Windows 8 hangs with KVM
On Fri, Jan 04, 2013 at 10:58:33PM +0100, Stefan Pietsch wrote: Hi all, when I run KVM with this command the Windows 8 installation stops with error code 0x005D: kvm -m 1024 -hda win8.img -cdrom windows_8_x86.iso After adding the option -cpu host the installation proceeds to a black screen and hangs. With Virtualbox the installation succeeds. The host CPU is an Intel Core Duo L2400. Do you have any suggestions? What is your kernel/qemu version? -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Installation of Windows 8 hangs with KVM
* Gleb Natapov g...@redhat.com [2013-01-06 11:11]: On Fri, Jan 04, 2013 at 10:58:33PM +0100, Stefan Pietsch wrote: Hi all, when I run KVM with this command the Windows 8 installation stops with error code 0x005D: kvm -m 1024 -hda win8.img -cdrom windows_8_x86.iso After adding the option -cpu host the installation proceeds to a black screen and hangs. With Virtualbox the installation succeeds. The host CPU is an Intel Core Duo L2400. Do you have any suggestions? What is your kernel/qemu version? I'm using Debian unstable. qemu-kvm 1.1.2+dfsg-3 Linux version 3.2.0-4-686-pae (debian-ker...@lists.debian.org) (gcc version 4.6.3 (Debian 4.6.3-14) ) #1 SMP Debian 3.2.35-2 Regards, Stefan -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Installation of Windows 8 hangs with KVM
Hi all, when I run KVM with this command the Windows 8 installation stops with error code 0x005D: kvm -m 1024 -hda win8.img -cdrom windows_8_x86.iso After adding the option -cpu host the installation proceeds to a black screen and hangs. With Virtualbox the installation succeeds. The host CPU is an Intel Core Duo L2400. Do you have any suggestions? Regards, Stefan -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 50921] kvm hangs booting Windows 2000
https://bugzilla.kernel.org/show_bug.cgi?id=50921 Lucio Crusca lu...@sulweb.org changed: What|Removed |Added Status|NEW |RESOLVED Resolution||CODE_FIX --- Comment #19 from Lucio Crusca lu...@sulweb.org 2012-12-09 20:46:03 --- I've tried the patch in 3.6.7 (my current kernel) and I confirm it fixes the problem. My Windows2000 guest now works without specifying emulate_invalid_guest_state = 0 to kvm-intel.ko. -- Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 50921] kvm hangs booting Windows 2000
https://bugzilla.kernel.org/show_bug.cgi?id=50921 --- Comment #18 from Gleb g...@redhat.com 2012-12-05 14:15:30 --- Created an attachment (id=88501) -- (https://bugzilla.kernel.org/attachment.cgi?id=88501) patch to implement aad (b5) instruction. Can you see if this patch helps? -- Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 50921] kvm hangs booting Windows 2000
https://bugzilla.kernel.org/show_bug.cgi?id=50921 Gleb g...@redhat.com changed: What|Removed |Added Attachment #88501|patch to implement aad (b5) |patch to implement aad (D5) description|instruction.|instruction. -- Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 50921] kvm hangs booting Windows 2000
https://bugzilla.kernel.org/show_bug.cgi?id=50921 --- Comment #17 from Gleb g...@redhat.com 2012-11-27 08:22:15 --- (In reply to comment #16) @xiaoguangrong: YOU ARE THE MAN! 'emulate_invalid_guest_state = 0' did the trick, now I have win2000 running in a 3.6.7 kvm guest! Thanks. Still guessing why it works with plain kvm-intel.ko in Debian kernels is out of my reach, but I can safely shove that mystery in the when-I'll-have-time-stuff drawer and live happy with this solution meanwhile. Because emulate_invalid_guest_state = 0 was the default before 3.6.0. This is why I wanted you to try 3.5.0. -- Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 50921] kvm hangs booting Windows 2000
https://bugzilla.kernel.org/show_bug.cgi?id=50921 Alan a...@lxorguk.ukuu.org.uk changed: What|Removed |Added CC||a...@lxorguk.ukuu.org.uk --- Comment #12 from Alan a...@lxorguk.ukuu.org.uk 2012-11-26 12:09:34 --- vboxpci 22709 0 - Live 0xf89bb000 (O) vboxnetadp 25431 0 - Live 0xf8aa6000 (O) vboxnetflt 22987 0 - Live 0xf8aae000 (O) vboxdrv 227471 3 vboxpci,vboxnetadp,vboxnetflt, Live 0xf91d4000 (O) Shouldn't be interefering but probably a good idea to test without -- Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Bug 50921] kvm hangs booting Windows 2000
https://bugzilla.kernel.org/show_bug.cgi?id=50921 --- Comment #13 from Lucio Crusca lu...@sulweb.org 2012-11-26 13:13:56 --- @Alan: see comment #5, since then I've always tested with and without vbox modules. @Gleb: can't run on 3.5.0 right now, I'll take the stack trace ASAP. -- Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are watching the assignee of the bug. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html