Re: [kvm-devel] Second KVM process hangs eating 80-100% CPU on host during startup
--- On Sat, 4/19/08, Marcelo Tosatti <[EMAIL PROTECTED]> wrote:
> From: Marcelo Tosatti <[EMAIL PROTECTED]>
> Subject: Re: [kvm-devel] Second KVM process hangs eating 80-100% CPU on host
> during startup
> To: "Alex Davis" <[EMAIL PROTECTED]>
> Cc: [EMAIL PROTECTED], kvm-devel@lists.sourceforge.net
> Date: Saturday, April 19, 2008, 7:11 PM
> On Sat, Apr 19, 2008 at 03:47:31PM -0700, Alex Davis wrote:
> > --- On Fri, 4/18/08, Avi Kivity
> <[EMAIL PROTECTED]> wrote:
> >
> > > From: Avi Kivity <[EMAIL PROTECTED]>
> > > Subject: Re: [kvm-devel] Second KVM process hangs
> eating 80-100% CPU on host during startup
> > [snip]
> >
> > I tried booting the guest with 'lpj=10682525'
> to work around the
> > calibrate_delay issue, but that gave me:
> >
> > [0.004100] ENABLING IO-APIC IRQs
> > [0.004100] ..TIMER: vector=0x31 apic1=0 pin1=0
> apic2=-1 pin2=-1
> > [0.004100] ..MP-BIOS bug: 8254 timer not connected
> to IO-APIC
> > [0.004100] ...trying to set up timer (IRQ0)
> through the 8259A ... failed.
> > [0.004100] ...trying to set up timer as Virtual
> Wire IRQ ... failed.
> > [0.004100] ...trying to set up timer as ExtINT IRQ
> ... failed :(.
> > [0.004100] Kernel panic - not syncing: IO-APIC +
> timer doesn't work! Boot
> > with apic=debug and send a report.Then try booting
> with the 'noapic' option.
> >
> > Booting with 'apic=debug' gives these
> additional lines:
> > [0.004100] Getting VERSION: 50014
> > [0.004100] Getting VERSION: 50014
> > [0.004100] Getting ID: 0
> > [0.004100] Getting LVT0: 700
> > [0.004100] Getting LVT1: 1
>
> Hi Alex,
>
> Can you please try the following.
>
> KVM: PIT: make last_injected_time per-guest
>
> Otherwise multiple guests use the same variable and boom.
>
> Also use kvm_vcpu_kick() to make sure that if a timer
> triggers on
> a different CPU the event won't be missed.
>
> Signed-off-by: Marcelo Tosatti <[EMAIL PROTECTED]>
>
>
> diff --git a/arch/x86/kvm/i8254.c b/arch/x86/kvm/i8254.c
> index 2852dd1..5697ad2 100644
> --- a/arch/x86/kvm/i8254.c
> +++ b/arch/x86/kvm/i8254.c
> @@ -200,10 +200,8 @@ int __pit_timer_fn(struct
> kvm_kpit_state *ps)
>
> atomic_inc(&pt->pending);
> smp_mb__after_atomic_inc();
> - if (vcpu0 && waitqueue_active(&vcpu0->wq))
> {
> - vcpu0->arch.mp_state = KVM_MP_STATE_RUNNABLE;
> - wake_up_interruptible(&vcpu0->wq);
> - }
> + if (vcpu0)
> + kvm_vcpu_kick(vcpu0);
>
> pt->timer.expires = ktime_add_ns(pt->timer.expires,
> pt->period);
> pt->scheduled = ktime_to_ns(pt->timer.expires);
> @@ -572,7 +570,6 @@ void kvm_inject_pit_timer_irqs(struct
> kvm_vcpu *vcpu)
> struct kvm_pit *pit = vcpu->kvm->arch.vpit;
> struct kvm *kvm = vcpu->kvm;
> struct kvm_kpit_state *ps;
> - static unsigned long last_injected_time;
>
> if (vcpu && pit) {
> ps = &pit->pit_state;
> @@ -582,11 +579,11 @@ void kvm_inject_pit_timer_irqs(struct
> kvm_vcpu *vcpu)
>* 2. Last interrupt was accepted or waited for too long
> time*/
> if (atomic_read(&ps->pit_timer.pending)
> &&
> (ps->inject_pending ||
> - (jiffies - last_injected_time
> + (jiffies - ps->last_injected_time
> >= KVM_MAX_PIT_INTR_INTERVAL))) {
> ps->inject_pending = 0;
> __inject_pit_timer_intr(kvm);
> - last_injected_time = jiffies;
> + ps->last_injected_time = jiffies;
> }
> }
> }
> diff --git a/arch/x86/kvm/i8254.h b/arch/x86/kvm/i8254.h
> index e63ef38..db25c2a 100644
> --- a/arch/x86/kvm/i8254.h
> +++ b/arch/x86/kvm/i8254.h
> @@ -35,6 +35,7 @@ struct kvm_kpit_state {
> struct mutex lock;
> struct kvm_pit *pit;
> bool inject_pending; /* if inject pending interrupts */
> + unsigned long last_injected_time;
> };
>
> struct kvm_pit {
Problem(s) solved. Everything is working now. Can now boot both with and
without 'lpj='. The BogoMIPs are also being calculated correctly in secondary
guests without 'lpj='. I'll play with it some more just to make sure, then I'll
close the original bug.
Thanks, Marcelo et al.
_
Re: [kvm-devel] Second KVM process hangs eating 80-100% CPU on host during startup
--- On Fri, 4/18/08, Avi Kivity <[EMAIL PROTECTED]> wrote: > From: Avi Kivity <[EMAIL PROTECTED]> > Subject: Re: [kvm-devel] Second KVM process hangs eating 80-100% CPU on host > during startup [snip] I tried booting the guest with 'lpj=10682525' to work around the calibrate_delay issue, but that gave me: [0.004100] ENABLING IO-APIC IRQs [0.004100] ..TIMER: vector=0x31 apic1=0 pin1=0 apic2=-1 pin2=-1 [0.004100] ..MP-BIOS bug: 8254 timer not connected to IO-APIC [0.004100] ...trying to set up timer (IRQ0) through the 8259A ... failed. [0.004100] ...trying to set up timer as Virtual Wire IRQ ... failed. [0.004100] ...trying to set up timer as ExtINT IRQ ... failed :(. [0.004100] Kernel panic - not syncing: IO-APIC + timer doesn't work! Boot with apic=debug and send a report.Then try booting with the 'noapic' option. Booting with 'apic=debug' gives these additional lines: [0.004100] Getting VERSION: 50014 [0.004100] Getting VERSION: 50014 [0.004100] Getting ID: 0 [0.004100] Getting LVT0: 700 [0.004100] Getting LVT1: 1 Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] Second KVM process hangs eating 80-100% CPU on host during startup
--- On Fri, 4/18/08, Avi Kivity <[EMAIL PROTECTED]> wrote: > From: Avi Kivity <[EMAIL PROTECTED]> > Subject: Re: [kvm-devel] Second KVM process hangs eating 80-100% CPU on host > during startup > To: "Alex Davis" <[EMAIL PROTECTED]> > Cc: kvm-devel@lists.sourceforge.net > Date: Friday, April 18, 2008, 12:12 PM > Alex Davis wrote: > > Host software: > > Linux 2.6.24.4 > > KVM 65 (I am using the kernel modules from this > release). > > X11 7.2 from Xorg > > SDL 1.2.13 > > GCC 4.1.1 > > Glibc 2.4 > > > > Host hardware: > > Asus P5B Deluxe (P965 chipset based) motherboard > > 4 GB RAM > > Intel E6700 CPU > > > > Guest software: > > Slackware 12.0 installed from CD-ROM. > > > > Command used to first KVM instance: > > /usr/local/bin/qemu-system-x86_64 -hda > /spare/vdisk1.img -cdrom /dev/cdrom -boot c -m 384 -net > > nic,macaddr=DE:AD:BE:EF:11:29 -net > tap,ifname=tap0,script=no & > > > > Command used to start second KVM instance: > > /usr/local/bin/qemu-system-x86_64 -hda > /spare/vdisk2.img -cdrom /dev/cdrom -boot c -m 384 -net > > nic,macaddr=DE:AD:BE:EF:11:30 -net > tap,ifname=tap1,script=no & > > > > tap0 and tap1 are bridged on the host. The guest OS > was installed on /spare/vdisk1.img, > > which was initially created by /usr/local/bin/qemu-img > create -f qcow /spare/vdisk.img 10G > > After the guest installation completed, vdisk1 was > copied to vdisk2. > > > > The second instance always stops after printing > > Checking if the processor honours the WP bit even in > supervisor mode... Ok. > > It stays hung until I press the return key in the > first instance; sometimes clicking in another X > > window will wake it up as well. > > > > This is a test machine so I can test patches (almost) > at will. > > > > > > Strange. Does pinning each guest to a different cpu help > (use 'taskset > 1 qemu ... vdisk1.img & ', taskset 2 qemu ... > vdisk2.img) Some additional information: I upgraded the guest to 2.6.25, and added some printk's to init_32.c and init/calibrate.c in the kernel source tree. Here's the output from dmesg for the guest boot: [0.004000] Checking if this processor honours the WP bit even in supervisor mode...Ok. [0.004000] Before cpa_init. [0.004000] CPA: page pool initialized 1 of 1 pages preallocated [0.004000] After cpa_init. [0.004000] After pagealloc [0.004000] After cpu_hotplug_init [0.004000] After kmem_cache_init [0.004000] After setup_percpu_pageset [0.004000] After numa_policy_init [0.004005] After late_time_init [0.004622] Before read_current_timer(&pre_start) [0.005314] After read_current_timer() [0.006493] Before read_current_timer(&start) [ 16.065027] Before read_current_timer(&post_start) [ 16.065753] Before read_current_timer(&post_end) [ 16.066437] Before read_current_timer(&start) [ 16.073007] Before read_current_timer(&post_start) [ 16.081007] Before read_current_timer(&post_end) [ 16.081703] Before read_current_timer(&start) [ 16.089008] Before read_current_timer(&post_start) [ 16.097008] Before read_current_timer(&post_end) [ 16.097695] Before read_current_timer(&start) [ 16.105010] Before read_current_timer(&post_start) [ 16.113009] Before read_current_timer(&post_end) [ 16.113697] Before read_current_timer(&start) [ 16.121010] Before read_current_timer(&post_start) [ 16.129010] Before read_current_timer(&post_end) [ 16.129697] calibrate_delay_direct() failed to get a good estimate for loops_per_jiffy. [ 16.129698] Probably due to long platform interrupts. Consider using "lpj=" boot option. [ 16.132180] Calibrating delay loop... 5308.41 BogoMIPS (lpj=10616832) [ 16.237019] After calibrate_delay Notice how the time jumped from about 0 seconds to 16 seconds. That's where I woke it up by typing in another window. The code seems to be hanging in the call to read_current_timer(&start) in function calibrate_delay_direct in init/calibrate.c. Also notice that calibrate_delay_direct() failed. Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] Second KVM process hangs eating 80-100% CPU on host during startup
--- On Fri, 4/18/08, Avi Kivity <[EMAIL PROTECTED]> wrote: > From: Avi Kivity <[EMAIL PROTECTED]> > Subject: Re: [kvm-devel] Second KVM process hangs eating 80-100% CPU on host > during startup > To: "Alex Davis" <[EMAIL PROTECTED]> > Cc: kvm-devel@lists.sourceforge.net > Date: Friday, April 18, 2008, 12:12 PM > Alex Davis wrote: > > Host software: > > Linux 2.6.24.4 > > KVM 65 (I am using the kernel modules from this > release). > > X11 7.2 from Xorg > > SDL 1.2.13 > > GCC 4.1.1 > > Glibc 2.4 > > > > Host hardware: > > Asus P5B Deluxe (P965 chipset based) motherboard > > 4 GB RAM > > Intel E6700 CPU > > > > Guest software: > > Slackware 12.0 installed from CD-ROM. > > > > Command used to first KVM instance: > > /usr/local/bin/qemu-system-x86_64 -hda > /spare/vdisk1.img -cdrom /dev/cdrom -boot c -m 384 -net > > nic,macaddr=DE:AD:BE:EF:11:29 -net > tap,ifname=tap0,script=no & > > > > Command used to start second KVM instance: > > /usr/local/bin/qemu-system-x86_64 -hda > /spare/vdisk2.img -cdrom /dev/cdrom -boot c -m 384 -net > > nic,macaddr=DE:AD:BE:EF:11:30 -net > tap,ifname=tap1,script=no & > > > > tap0 and tap1 are bridged on the host. The guest OS > was installed on /spare/vdisk1.img, > > which was initially created by /usr/local/bin/qemu-img > create -f qcow /spare/vdisk.img 10G > > After the guest installation completed, vdisk1 was > copied to vdisk2. > > > > The second instance always stops after printing > > Checking if the processor honours the WP bit even in > supervisor mode... Ok. > > It stays hung until I press the return key in the > first instance; sometimes clicking in another X > > window will wake it up as well. > > > > This is a test machine so I can test patches (almost) > at will. > > > > > > Strange. Does pinning each guest to a different cpu help > (use 'taskset > 1 qemu ... vdisk1.img & ', taskset 2 qemu ... > vdisk2.img) > > taskset made no difference. Upgrading to kvm-66 didn't help either. > Any sufficiently difficult bug is indistinguishable from a > feature. Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] Second KVM process hangs eating 80-100% CPU on host during startup
--- Alex Davis <[EMAIL PROTECTED]> wrote: > Host software: > Linux 2.6.24.4 > KVM 65 (I am using the kernel modules from this release). > X11 7.2 from Xorg > SDL 1.2.13 > GCC 4.1.1 > Glibc 2.4 > > Host hardware: > Asus P5B Deluxe (P965 chipset based) motherboard > 4 GB RAM > Intel E6700 CPU > > Guest software: > Slackware 12.0 installed from CD-ROM. > Additional information: host arch. is x86_64(64-bit); guest arch. is x86(32-bit). I code, therefore I am Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
[kvm-devel] Second KVM process hangs eating 80-100% CPU on host during startup
Host software: Linux 2.6.24.4 KVM 65 (I am using the kernel modules from this release). X11 7.2 from Xorg SDL 1.2.13 GCC 4.1.1 Glibc 2.4 Host hardware: Asus P5B Deluxe (P965 chipset based) motherboard 4 GB RAM Intel E6700 CPU Guest software: Slackware 12.0 installed from CD-ROM. Command used to first KVM instance: /usr/local/bin/qemu-system-x86_64 -hda /spare/vdisk1.img -cdrom /dev/cdrom -boot c -m 384 -net nic,macaddr=DE:AD:BE:EF:11:29 -net tap,ifname=tap0,script=no & Command used to start second KVM instance: /usr/local/bin/qemu-system-x86_64 -hda /spare/vdisk2.img -cdrom /dev/cdrom -boot c -m 384 -net nic,macaddr=DE:AD:BE:EF:11:30 -net tap,ifname=tap1,script=no & tap0 and tap1 are bridged on the host. The guest OS was installed on /spare/vdisk1.img, which was initially created by /usr/local/bin/qemu-img create -f qcow /spare/vdisk.img 10G After the guest installation completed, vdisk1 was copied to vdisk2. The second instance always stops after printing Checking if the processor honours the WP bit even in supervisor mode... Ok. It stays hung until I press the return key in the first instance; sometimes clicking in another X window will wake it up as well. This is a test machine so I can test patches (almost) at will. I code, therefore I am Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
