Hi Arie, I saw you also posting on some other old bugs with the same symptom (e.g. 1346917). And that exactly is the problem with this issue - the message only represents the symptom but not the root cause. Fixing the symptom makes no sense, because if your IRQ delivery is stalled then it is stalled - there is only very little the system can do other than trottling it down. If you follow the discussion around [1] you'll see that this was the thought back then and it didn't change since then AFAIK.
There were cases around KSM (bug 1346917), but also others around broken RAM, then others had just overloaded their CPUs, others had scheduler bug triggering the same while in other cases there was a thundering herd issue. The only thing all of these cases share is that eventually something (tm) happened which made IRQ/hrtimers stutter to then throttle them down. Many of the underlying issues causing that have been fixed over the years - this got more rare nowadays. But for each case still left one would not need a "this happened again here" message. But instead a way to reproduce, to then debug the root cause of this exact case and check for a fix then. Therefore three recommendations for people affected: 1. Try as good as you can to get it to reproduce reliably and then outline these steps, this will help people to hopefully get a grasp of the root cause. Do -not- just report the "hrtimer: interrupt took" being the symptom 2. Always try the same setup you have, but with the very latest virtualization stack provided - quite often things are fixed there and if that is confirmed it becomes "only" a binary search what would need to be backported. 3. open a new bug for these, because until a root cause of a given case is found and identified to be the same we don't know if it is the same issue. [1]: http://www.mail-archive.com/kvm@vger.kernel.org/msg23491.html ** Also affects: linux (Ubuntu) Importance: Undecided Status: New ** Package changed: kvm (Ubuntu) => qemu (Ubuntu) ** Changed in: qemu (Ubuntu) Status: Confirmed => Incomplete ** Changed in: linux (Ubuntu) Status: New => Incomplete -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/503138 Title: Lucid & Natty, KVM, After kernel message hrtimer: interrupt too slow.... the SMP kvm guest becomes slow. Status in linux package in Ubuntu: Incomplete Status in qemu package in Ubuntu: Incomplete Bug description: KVM Host is running a clean 9.10 server install with just qemu-kvm and virt-manager kernel=2.6.31-17-generic uname -a: Linux VMMASTER 2.6.31-17-generic #54-Ubuntu SMP Thu Dec 10 16:20:31 UTC 2009 i686 GNU/Linux version signature: Ubuntu 2.6.31-17.54-generic KVM Guest is running a clean 9.10 server install with some userland services (apache/postfix/whatnot) kernel=2.6.31-16-generic-pae (from the linux-image-virtual package) uname -a: Linux VM1 2.6.31-16-generic-pae #53-Ubuntu SMP Tue Dec 8 05:20:21 UTC 2009 i686 GNU/Linux version signature: Ubuntu 2.6.31-16.53-generic-pae KVM guest startup command (as invoked by virt-manager): /usr/bin/kvm -S -M pc-0.11 -m 1024 -smp 2 -name lexx -uuid [UUID] -monitor unix:/var/run/libvirt/qemu/VM1.monitor,server,nowait -boot c -drive file=,if=ide,media=cdrom,index=2 -drive file=/var/lib/libvirt/images/VM1.img,if=virtio,index=0,boot=on -net nic,macaddr=[MAC],vlan=0,model=virtio,name=virtio.0 -net tap,fd=16,vlan=0,name=tap.0 -serial none -parallel none -usb -vnc 127.0.0.1:0 -k en-us -vga cirrus Problem description: After a while (and high network IO) I see this pop up in my guest dmesg: hrtimer: interrupt too slow, forcing clock min delta to 215997540 ns after that the guest becomes very slow to respond, sometimes taking seconds to echo my ssh input back, on a local lan. Only a reboot of the kvm guest fixes this, until the dreaded hrtimer message pops up After a lot of googling and trying a lot of things I found this discussion on the patchwork kernel mailinglist, which contains a possible solution: http://patchwork.kernel.org/patch/51561/ Please look into it, perhaps this solves a lot of kvm-users' problems I'd like to patch my kvm guests' kernel myself to test this hrtimer patch, do you have the correct procedure for me so i can create a custom kernel? To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/503138/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
