[Bug 745785] Re: KVM HOST: Freeze every ~2 weeks
Rodrigo Campos, thank you for reporting this bug. Thank you for taking the time to report this bug and trying to help make Ubuntu better. However, 2.6.38-7.37~lucid1 PPA is not a software package provided by the official Ubuntu repositories. Because of this the Ubuntu project can not support or fix your particular bug. Please report this bug to the provider of the software package. Thanks! If you are interested in learning more about software repositories and Ubuntu the following pages should be informative: 1. http://www.ubuntu.com/project/about-ubuntu/components - information about Ubuntu repositories 2. https://help.ubuntu.com/community/Repositories - information regarding managing repositories ** Changed in: linux (Ubuntu) Status: Confirmed = Invalid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/745785 Title: KVM HOST: Freeze every ~2 weeks To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/745785/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 745785] Re: KVM HOST: Freeze every ~2 weeks
Seeing this same problem on 11.04, linux-image-generic 2.6.38.10.25. Disabling cron.d/mdadm for now, but would like to see a better solution eventually. Probably going to move our kvm images off onto NAS anyway though. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/745785 Title: KVM HOST: Freeze every ~2 weeks To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/745785/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 745785] Re: KVM HOST: Freeze every ~2 weeks
This problem is still present on fully up to date 10.04.3. Disabling the checkarray cron job does somewhat mitigate this, but any array resync will wind up reproducing this hang. Anything further needed to debug this? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/745785 Title: KVM HOST: Freeze every ~2 weeks To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/745785/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 745785] Re: KVM HOST: Freeze every ~2 weeks
The reason of our freez was the crojob /etc/cron.d/mdadm with the command /usr/share/mdadm/checkarray . checkarray made an rebuild of the array once a month, almost everytime the computer freezed. So we just delete the software raid containig the kvm images. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/745785 Title: KVM HOST: Freeze every ~2 weeks To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/745785/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 745785] Re: KVM HOST: Freeze every ~2 weeks
While that avoids the problem, it seems like the the wrong overall step. The array rebuild needs to be able to work, even under load. I too tracked my problem down to the checkarray script's execution and have file a separate report about. Seems there's a number of people experiencing the same problem. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/745785 Title: KVM HOST: Freeze every ~2 weeks To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/745785/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 745785] Re: KVM HOST: Freeze every ~2 weeks
I'm also seeing this problem on a 10.04 host, running software RAID1, along with KVM. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/745785 Title: KVM HOST: Freeze every ~2 weeks To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/745785/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 745785] Re: KVM HOST: Freeze every ~2 weeks
Same problem here. It freezes every month or under heavy load. KVM running an an software RAID. Every month because there is an montly cronjob to check the software Raid: /usr/share/mdadm/checkarray. Testing the Partition, on which the kvm images are one it, triggers the bug. The kvm images takes 100% cpu, only reboot can stop it: syslog: May 24 06:02:47 localhost kernel: [1682547.453843] INFO: task kdmflush:465 blocked for more than 120 seconds. May 24 06:02:47 localhost kernel: [1682547.453845] echo 0 /proc/sys/kernel/hung_task_timeout_secs disables this message. May 24 06:02:47 localhost kernel: [1682547.453848] kdmflush D 0 465 2 0x May 24 06:02:47 localhost kernel: [1682547.453852] 88032e0b99d0 0046 00015dc0 00015dc0 May 24 06:02:47 localhost kernel: [1682547.453856] 88032ec8dfd0 88032e0b9fd8 00015dc0 88032ec8dc00 May 24 06:02:47 localhost kernel: [1682547.453860] 00015dc0 88032e0b9fd8 00015dc0 88032ec8dfd0 May 24 06:02:47 localhost kernel: [1682547.453864] Call Trace: May 24 06:02:47 localhost kernel: [1682547.453879] [a0074685] wait_barrier+0xf5/0x140 [raid1] May 24 06:02:47 localhost kernel: [1682547.453885] [8105ded0] ? default_wake_function+0x0/0x20 May 24 06:02:47 localhost kernel: [1682547.453890] [a0077651] make_request+0x51/0x750 [raid1] May 24 06:02:47 localhost kernel: [1682547.453894] [81064304] ? check_preempt_wakeup+0x1c4/0x3c0 May 24 06:02:47 localhost kernel: [1682547.453897] [8105f10b] ? enqueue_task_fair+0x9b/0xa0 May 24 06:02:47 localhost kernel: [1682547.453902] [8142b6b0] md_make_request+0xc0/0x130 May 24 06:02:47 localhost kernel: [1682547.453907] [812a1d01] generic_make_request+0x1b1/0x4f0 May 24 06:02:47 localhost kernel: [1682547.453911] [810f8475] ? mempool_alloc_slab+0x15/0x20 May 24 06:02:47 localhost kernel: [1682547.453915] [810f860d] ? mempool_alloc+0x5d/0x130 May 24 06:02:47 localhost kernel: [1682547.453919] [814382ad] __map_bio+0xad/0x130 May 24 06:02:47 localhost kernel: [1682547.453922] [814387dd] __clone_and_map+0x4ad/0x4c0 May 24 06:02:47 localhost kernel: [1682547.453925] [810f860d] ? mempool_alloc+0x5d/0x130 May 24 06:02:47 localhost kernel: [1682547.453929] [814398b8] __split_and_process_bio+0x108/0x190 May 24 06:02:47 localhost kernel: [1682547.453932] [81439996] dm_flush+0x56/0x70 May 24 06:02:47 localhost kernel: [1682547.453935] [814399fc] dm_wq_work+0x4c/0x1c0 May 24 06:02:47 localhost kernel: [1682547.453938] [814399b0] ? dm_wq_work+0x0/0x1c0 May 24 06:02:47 localhost kernel: [1682547.453942] [81081457] run_workqueue+0xc7/0x1a0 May 24 06:02:47 localhost kernel: [1682547.453946] [810815d3] worker_thread+0xa3/0x110 May 24 06:02:47 localhost kernel: [1682547.453950] [81085ff0] ? autoremove_wake_function+0x0/0x40 May 24 06:02:47 localhost kernel: [1682547.453954] [81081530] ? worker_thread+0x0/0x110 May 24 06:02:47 localhost kernel: [1682547.453957] [81085c76] kthread+0x96/0xa0 May 24 06:02:47 localhost kernel: [1682547.453961] [810141ea] child_rip+0xa/0x20 May 24 06:02:47 localhost kernel: [1682547.453964] [81085be0] ? kthread+0x0/0xa0 May 24 06:02:47 localhost kernel: [1682547.453967] [810141e0] ? child_rip+0x0/0x20 May 24 06:02:47 localhost kernel: [1682547.453971] INFO: task jbd2/dm-0-8:610 blocked for more than 120 seconds. May 24 06:02:47 localhost kernel: [1682547.453973] echo 0 /proc/sys/kernel/hung_task_timeout_secs disables this message. May 24 06:02:47 localhost kernel: [1682547.453975] jbd2/dm-0-8 D 0 610 2 0x May 24 06:02:47 localhost kernel: [1682547.453979] 880325db1d20 0046 00015dc0 00015dc0 May 24 06:02:47 localhost kernel: [1682547.453983] 8803265703d0 880325db1fd8 00015dc0 88032657 May 24 06:02:47 localhost kernel: [1682547.453986] 00015dc0 880325db1fd8 00015dc0 8803265703d0 May 24 06:02:47 localhost kernel: [1682547.453990] Call Trace: May 24 06:02:47 localhost kernel: [1682547.453995] [8121e741] jbd2_journal_commit_transaction+0x1c1/0x1280 May 24 06:02:47 localhost kernel: [1682547.453999] [81077bbc] ? lock_timer_base+0x3c/0x70 May 24 06:02:47 localhost kernel: [1682547.454002] [81085ff0] ? autoremove_wake_function+0x0/0x40 May 24 06:02:47 localhost kernel: [1682547.454006] [81225d7d] kjournald2+0xbd/0x220 May 24 06:02:47 localhost kernel: [1682547.454010] [81085ff0] ? autoremove_wake_function+0x0/0x40 May 24 06:02:47 localhost kernel: [1682547.454013] [81225cc0] ? kjournald2+0x0/0x220 May 24 06:02:47 localhost kernel: [1682547.454016] [81085c76] kthread+0x96/0xa0 May 24 06:02:47
[Bug 745785] Re: KVM HOST: Freeze every ~2 weeks
** Attachment added: lspci.txt https://bugs.launchpad.net/ubuntu/+source/linux/+bug/745785/+attachment/2143343/+files/lspci.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/745785 Title: KVM HOST: Freeze every ~2 weeks -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 745785] Re: KVM HOST: Freeze every ~2 weeks
** Changed in: linux (Ubuntu) Status: New = Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/745785 Title: KVM HOST: Freeze every ~2 weeks -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 745785] Re: KVM HOST: Freeze every ~2 weeks
** Attachment added: lspci https://bugs.launchpad.net/bugs/745785/+attachment/1956797/+files/lspci -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/745785 Title: KVM HOST: Freeze every ~2 weeks -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 745785] Re: KVM HOST: Freeze every ~2 weeks
** Attachment added: dmesg https://bugs.launchpad.net/ubuntu/+source/linux/+bug/745785/+attachment/1956799/+files/dmesg -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/745785 Title: KVM HOST: Freeze every ~2 weeks -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 745785] Re: KVM HOST: Freeze every ~2 weeks
Also, forgot to mention: I'm using lucid with a 2.6.38 kernel from the kernel ppa. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/745785 Title: KVM HOST: Freeze every ~2 weeks -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 745785] Re: KVM HOST: Freeze every ~2 weeks
** Attachment added: heavy-mkfs.sh https://bugs.launchpad.net/ubuntu/+source/linux/+bug/745785/+attachment/1956808/+files/heavy-mkfs.sh -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/745785 Title: KVM HOST: Freeze every ~2 weeks -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 745785] Re: KVM HOST: Freeze every ~2 weeks
Also, I've just noted that after the 10 mkfs run, 3 minutes later, collectd started logging: Mar 30 11:42:27 hydra collectd[1767]: uc_update: Value too old: name = hydra.integratech.net/irq/irq-0; value time = 1301496136; last cache update = 1301496136; Mar 30 11:42:27 hydra collectd[1767]: uc_update: Value too old: name = hydra.integratech.net/disk-sdb/disk_octets; value time = 1301496136; last cache update = 130149 6136; Mar 30 11:42:27 hydra collectd[1767]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed with status -1. Mar 30 11:42:27 hydra collectd[1767]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed with status -1. Mar 30 11:42:27 hydra collectd[1767]: uc_update: Value too old: name = hydra.integratech.net/swap/swap-used; value time = 1301496136; last cache update = 1301496136; Mar 30 11:42:27 hydra collectd[1767]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed with status -1. Mar 30 11:42:27 hydra collectd[1767]: uc_update: Value too old: name = hydra.integratech.net/irq/irq-1; value time = 1301496147; last cache update = 1301496147; Mar 30 11:42:27 hydra collectd[1767]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed with status -1. Mar 30 11:42:27 hydra collectd[1767]: uc_update: Value too old: name = hydra.integratech.net/irq/irq-7; value time = 1301496147; last cache update = 1301496147; Mar 30 11:42:27 hydra collectd[1767]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed with status -1. Mar 30 11:42:27 hydra collectd[1767]: uc_update: Value too old: name = hydra.integratech.net/irq/irq-8; value time = 1301496147; last cache update = 1301496147; Mar 30 11:42:27 hydra collectd[1767]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed with status -1. Mar 30 11:42:27 hydra collectd[1767]: uc_update: Value too old: name = hydra.integratech.net/irq/irq-9; value time = 1301496147; last cache update = 1301496147; Mar 30 11:42:27 hydra collectd[1767]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed with status -1. Mar 30 11:42:27 hydra collectd[1767]: uc_update: Value too old: name = hydra.integratech.net/irq/irq-12; value time = 1301496147; last cache update = 1301496147; Mar 30 11:42:28 hydra collectd[1767]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed with status -1. Mar 30 11:42:28 hydra collectd[1767]: uc_update: Value too old: name = hydra.integratech.net/irq/irq-14; value time = 1301496147; last cache update = 1301496147; Mar 30 11:42:28 hydra collectd[1767]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed with status -1. Mar 30 11:42:28 hydra collectd[1767]: uc_update: Value too old: name = hydra.integratech.net/irq/irq-15; value time = 1301496147; last cache update = 1301496147; Mar 30 11:42:28 hydra collectd[1767]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed with status -1. Mar 30 11:42:28 hydra collectd[1767]: uc_update: Value too old: name = hydra.integratech.net/irq/irq-16; value time = 1301496147; last cache update = 1301496147; Mar 30 11:42:28 hydra collectd[1767]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed with status -1. Mar 30 11:42:28 hydra collectd[1767]: uc_update: Value too old: name = hydra.integratech.net/irq/irq-18; value time = 1301496147; last cache update = 1301496147; Mar 30 11:42:28 hydra collectd[1767]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed with status -1. Mar 30 11:42:28 hydra collectd[1767]: uc_update: Value too old: name = hydra.integratech.net/irq/irq-19; value time = 1301496147; last cache update = 1301496147; Mar 30 11:42:28 hydra collectd[1767]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed with status -1. Mar 30 11:42:28 hydra collectd[1767]: uc_update: Value too old: name = hydra.integratech.net/irq/irq-22; value time = 1301496147; last cache update = 1301496147; Mar 30 11:42:28 hydra collectd[1767]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed with status -1. Mar 30 11:42:28 hydra collectd[1767]: uc_update: Value too old: name = hydra.integratech.net/disk-sdb/disk_ops; value time = 1301496147; last cache update = 1301496147; Mar 30 11:42:28 hydra collectd[1767]: uc_update: Value too old: name = hydra.integratech.net/swap/swap-free; value time = 1301496147; last cache update = 1301496147; Mar 30 11:42:28 hydra collectd[1767]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed with status -1. Mar 30 11:42:28 hydra collectd[1767]: Filter subsystem: Built-in target `write': Dispatching
