[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
I'm experiencing quite similar problems on Ubuntu 12.04.1 LTS running on brand new Fujitsu servers :( ** Attachment added: "messages.tar.gz" https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/614853/+attachment/3432126/+files/messages.tar.gz -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
** Changed in: linux Status: Confirmed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
This is also affecting Maverick, on physical hardware. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
Has there been an AKI released for this new patch? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
This bug was fixed in the package linux - 2.6.32-35.78 --- linux (2.6.32-35.78) lucid-proposed; urgency=low [Herton R. Krzesinski] * Release Tracking Bug - LP: #871899 [ Andrew Dickinson ] * SAUCE: sched: Prevent divide by zero when cpu_power is 0 - LP: #614853 [ Stefan Bader ] * [Config] Force perf to use libiberty for demangling - LP: #783660 [ Tim Gardner ] * [Config] Simplify binary-udebs dependencies - LP: #832352 * [Config] kernel preparation cannot be parallelized - LP: #832352 * [Config] Linearize module/abi checks - LP: #832352 * [Config] Linearize and simplify tree preparation rules - LP: #832352 * [Config] Build kernel image in parallel with modules - LP: #832352 * [Config] Set concurrency for kmake invocations - LP: #832352 * [Config] Improve install-arch-headers speed - LP: #832352 * [Config] Fix binary-perarch dependencies - LP: #832352 * [Config] Removed stamp-flavours target - LP: #832352 * [Config] Serialize binary indep targets - LP: #832352 * [Config] Use build stamp directly - LP: #832352 * [Config] Restore prepare-% target - LP: #832352 * [Config] Fix binary-% build target * [Config] Fix install-headers target - LP: #832352 * SAUCE: igb: Protect stats update - LP: #829566 * SAUCE: rtl8192se spams log - LP: #859702 [ Upstream Kernel Changes ] * Add mount option to check uid of device being mounted = expect uid, CVE-2011-1833 - LP: #732628 - CVE-2011-1833 * crypto: Move md5_transform to lib/md5.c - LP: #827462 * net: Compute protocol sequence numbers and fragment IDs using MD5. - LP: #827462 * ALSA: timer - Fix Oops at closing slave timer - LP: #827462 * ALSA: snd-usb-caiaq: Fix keymap for RigKontrol3 - LP: #827462 * powerpc: Fix device tree claim code - LP: #827462 * powerpc: pseries: Fix kexec on machines with more than 4TB of RAM - LP: #827462 * Linux 2.6.32.45+drm33.19 - LP: #827462 * ipv6: make fragment identifications less predictable, CVE-2011-2699 - LP: #827685 - CVE-2011-2699 * tunnels: fix netns vs proto registration ordering - LP: #823296 * Fix broken backport for IPv6 tunnels in 2.6.32-longterm kernels. * USB: xhci: fix OS want to own HC - LP: #837669 * USB: assign instead of equal in usbtmc.c - LP: #837669 * USB: usb-storage: unusual_devs entry for ARM V2M motherboard. - LP: #837669 * USB: Serial: Added device ID for Qualcomm Modem in Sagemcom's HiLo3G - LP: #837669 * atm: br2864: sent packets truncated in VC routed mode - LP: #837669 * hwmon: (ibmaem) add missing kfree - LP: #837669 * ALSA: snd-usb-caiaq: Correct offset fields of outbound iso_frame_desc - LP: #837669 * mm: fix wrong vmap address calculations with odd NR_CPUS values - LP: #837669 * perf tools: do not look at ./config for configuration - LP: #837669 * fs/partitions/efi.c: corrupted GUID partition tables can cause kernel oops - LP: #837669 * befs: Validate length of long symbolic links. - LP: #837669 * ALSA: snd_usb_caiaq: track submitted output urbs - LP: #837669 * ALSA: ac97: Add HP Compaq dc5100 SFF(PT003AW) to Headphone Jack Sense whitelist - LP: #826081, #837669 * futex: Fix regression with read only mappings - LP: #837669 * x86-32, vdso: On system call restart after SYSENTER, use int $0x80 - LP: #837669 * x86, UV: Remove UV delay in starting slave cpus - LP: #837669 * drm/ttm: fix ttm_bo_add_ttm(user) failure path - LP: #837669 * fuse: check size of FUSE_NOTIFY_INVAL_ENTRY message - LP: #837669 * igb: Fix lack of flush after register write and before delay - LP: #837669 * Linux 2.6.32.46 - LP: #837669 * cifs: fix possible memory corruption in CIFSFindNext, CVE-2011-3191 - LP: #834135 - CVE-2011-3191 * Bluetooth: Prevent buffer overflow in l2cap config request, CVE-2011-2497 - LP: #838423 - CVE-2011-2497 * core: Fix memory leak/corruption on VLAN GRO_DROP, CVE-2011-1576 - LP: #844361 - CVE-2011-1576 * ext4: Fix max file size and logical block counting of extent format file, CVE-2011-2695 - LP: #819574 - CVE-2011-2695 * drm/i915: prepare for fair lru eviction - LP: #843904 * drm/i915: Move the eviction logic to its own file. - LP: #843904 * drm/i915: Implement fair lru eviction across both rings. (v2) - LP: #843904 * drm/i915: Maintain LRU order of inactive objects upon access by CPU (v2) - LP: #843904 * drm/i915/evict: Ensure we completely cleanup on failure - LP: #843904 * drm/i915: Periodically flush the active lists and requests - LP: #843904 * Make TASKSTATS require root access, CVE-2011-2494 - LP: #866021 - CVE-2011-2494 * proc: fix a race in do_io_accounting(), CVE-2011-2495 - LP: #866025 - CVE-2011-2495 * drm/i915: Remove BUG_ON from i915_gem_evict_som
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
See: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/871899 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
It looks like everything has been qualified in the bug for the proposed package, and everyone has signed off on it. As of the 27th of October. I wonder if there's anything else preventing it from being promoted? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
Any ETA on promoting the 2.6.32-35.78 kernel package from -proposed to -updates? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
** Branch linked: lp:ubuntu/lucid-proposed/linux-mvl-dove ** Branch linked: lp:ubuntu/maverick-proposed/linux-mvl-dove -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
Since this bug is hard to verify, looking to require more than 1 week with the pristine kernel, and the same patch is already for some time in lucid-ec2 without issues, I'm marking verified for lucid update. ** Tags removed: verification-needed-lucid ** Tags added: verification-done-lucid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
The patch is now in -proposed 2.6.32-35.78 kernel for Lucid (it is already included in current ec2 flavour, just main kernel for lucid didn't have it). Just noted that on master, the debugging patch "UBUNTU: SAUCE: sched: Try tp catch cpu_power being set to 0" isn't included, not sure this was intended. As this can take a long time to verify, probably it can be tagged verification-done-lucid, unless there is some way/testcase to make the crash happen earlier. Anyone wanting to test 2.6.32-35.78 kernel, should enable -proposed for now, see https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
mine has been running for 212 days when it crashed after what i can read on the internet, it _seems_ this bug happens when the server's uptime is 200+ days -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
That syslog is from a 2.6.32 kernel (and a quite old one 2.6.32-29.58). However the current 2.6.32-34.77 would not have the work-around patch, yet. It is staged for the next round of updates. Was that the correct syslog (because crashes with 2.6.35 were mentioned). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
It was 244 days actually. Syslog output attached. ** Attachment added: "syslog output" https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/614853/+attachment/2512258/+files/syslog.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
Yes, the scheduler code changed since 2.6.32 and so the syslog is valuable. Also, was this actually 200+ day uptime or quicker? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
Ok we just got hit again by this bug is there still need to attach the output from syslog to this bug? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
Hi. today, on my filer, running debian squeeze with kernel 2.6.32-5-amd64, i had the same bug in "find_busiest_group" see the screen here : http://pic.twitter.com/sAih9DlN after rebooting, my server runs fine... but i'm affraid it can happen again :( -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
It seems like a regression to me or introduced by a new feature. We still have some karmic KVM hosts that are running 2.6.31-20-server kernel but they are definitely not affected by this issue. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
Whether using 2.6.35 or 2.6.38 would make no difference if the patch which is upstream helps. There was a patch claimed to cause the problem sooner in the upstream discussion but it did not seem to work for me when I tried it. So unfortunately I know of now way to speed up testing. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
Since the latest kvm machine is running 2.6.35-30-server #59~lucid1-Ubuntu and has an uptime of 13 days, 22:03. Will report back in 206 days from now to see if that fix is working as intended. Is there any other workaround available? Like upgrading to another backports kernel, 2.6.38 perhaps? Or can we trigger this bug in another way? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
This is currently only committed to the repository and will be included in the next proposed kernel update. There will be a message to this report, asking for verification when the package is prepared. Note this is 2.6.32. For 2.6.35 see comment #39: Ubuntu-2.6.35-29.51 had a fix that was said to fix some crashes. But the last confirmation of 2.6.35 crashing was using an older kernel. So for the moment there is nothing planned for that. First there needs to be some feedback that the latest kernel is still crashing that way. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
As James Sellman I'm quite curious if someone could point me to the changelog of the 2.6.35.xx kernel version where this was fixed as I'm unable to find it. I just want to make sure that this issue is fixed or has a workaround so that we don't get this oops again. So far 20 KVM servers have been hit by this bug. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
Can I be pointed to the commit with the diff where the fix went into linux generic (server, etc.) and what package rev it will go into testing on? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
Thanks Tim. If we can at least keep the div by zero from happening and keep the kernel from dying, if the underlying problem occurs again we can at least gather more information to determine what happened to put it the situation in the first place. In the meantime, at least we don't have to keep a planned reboot cycle to avoid unplanned oopses. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
This bug will be next to impossible to verify given its 219 day cycle. ** Changed in: linux (Ubuntu Lucid) Status: New => Fix Committed ** Changed in: linux (Ubuntu Lucid) Assignee: (unassigned) => Tim Gardner (timg-tpi) ** Changed in: linux (Ubuntu) Status: New => Invalid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
** Also affects: linux (Ubuntu) Importance: Undecided Status: New ** Also affects: linux (Ubuntu Lucid) Importance: Undecided Status: New ** Also affects: linux-ec2 (Ubuntu Lucid) Importance: Undecided Status: New ** Changed in: linux-ec2 (Ubuntu Lucid) Status: New => Fix Released ** Changed in: linux-ec2 (Ubuntu) Status: Fix Released => Invalid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
I wound up opening a separate bug for the generic/server packages over at https://bugs.launchpad.net/ubuntu/+source/linux/+bug/824304?comments=all ... There didn't seem to be a way for me to add those packages to this ticket (just other projects). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
This looks like the work-around used for the ec2 kernels. So it sounds like the same problem can in fact happen on real hardware (which was not really clear). That, the fact that it is clearly only papering over some other issue and no reports about this happening on other kernels prevented any action on later kernels. I think this report should be a good place, we just need another task for the "normal" kernel package. Probably the real fix could be to not mark the sched_clock as stable as it was brought up in that upstream discussion. Though obviously the 219 day delay makes it hard to verify. But before that I would like to make sure the second part is actually needed. Your report for Maverick was using a 2.6.35-24 kernel and the patch above came in much later (2.6.35-29, sorry the 2.6.38 in my last comment was a mistype). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
Apparently a patch will be included in Debian to fix the 219 days issue, as per http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=636797 It's for 2.6.32, but would you think it could be ported to the 2.6.35 on Maverick? Should I file a different bug? Thanks, ** Bug watch added: Debian Bug tracker #636797 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=636797 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
No, as this report was only observed on ec2 kernels and also quicker. There has been some upstream stable discussion about crashes after 219 days of uptime (in 2.6.32 based kernels). One of the patches mentioned commit 305e6835e05513406fa12820e40e4a8ecb63743c Author: Venkatesh Pallipadi Date: Mon Oct 4 17:03:21 2010 -0700 sched: Do not account irq time to current task would be upstream now, but is not in 2.6.38 kernels before Ubuntu-2.6.35-29.51. The other change seems not yet being pushed forward. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
Hi all. This happened to me with 2.6.35-24-server, it is a MySQL (Percona, 5.1.54) machine running not so heavy load but slightly heavier IO. Please find attached the crash log. The uptime of the server was ~219 days, which is relevant according to the original bug at the kernel. Was the patch on this bug ported to newer kernerls? Thanks, ** Attachment added: "Crash for kernel 2.6.35-24." https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/614853/+attachment/2331494/+files/mysql_crash_log.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
Scott, you're right! I think what happened is, that we were running 312 and had a crash after which we rebooted the machine and installed the newest kernel (314 at that time). But we didn't reboot the machine after the upgrade, so 312 was still running. Please ignore comment #36! Let's see how 316 performs... -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
Rudolf, your console log shows: [0.00] Linux version 2.6.32-312-ec2 (buildd@yellow) (gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #24-Ubuntu SMP Fri Jan 7 18:30:50 UTC 2011 (Ubuntu 2.6.32-312.24-ec2 2.6.32.27+drm33.12) That definitely indicates that you've either collected the wrong console output, or you're not running the kernel you think you are. It does appear that pv-grub is loading the kernel, but that its not a 3.6.32-314 kernel. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
I can confirm, that this bug is still happening in (see attached log): Ubuntu 10.04.2 LTS, kernel 2.6.32-314-ec2 We're running a Postgres server on AWS with linux software raid10. After the crash we upgraded to: Linux db6.i.bluereport.net 2.6.32-316-ec2 #31-Ubuntu SMP Wed May 18 14:10:36 UTC 2011 x86_64 GNU/Linux Will report if it's still happening in 316! ** Attachment added: "2.6.32-314-ec2 kernel crash @ 17.06.2011" https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/614853/+attachment/2172551/+files/db-17.06.2011.log -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/614853/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
This bug was fixed in the package linux-ec2 - 2.6.32-313.26 --- linux-ec2 (2.6.32-313.26) lucid-proposed; urgency=low [ Brad Figg ] * Release Tracking Bug - LP: #716657 [ Brad Figg ] * Release Tracking Bug - LP: #712864 [ Brad Figg ] * Rebased to 2.6.32-29.58 [ Ubuntu: 2.6.32-29.58 ] * Release Tracking Bug - LP: #716551 * net: fix rds_iovec page count overflow, CVE-2010-3865 - LP: #709153 - CVE-2010-3865 * net: ax25: fix information leak to userland, CVE-2010-3875 - LP: #710714 - CVE-2010-3875 * net: ax25: fix information leak to userland harder, CVE-2010-3875 - LP: #710714 - CVE-2010-3875 * net: packet: fix information leak to userland, CVE-2010-3876 - LP: #710714 - CVE-2010-3876 * net: tipc: fix information leak to userland, CVE-2010-3877 - LP: #711291 - CVE-2010-3877 * inet_diag: Make sure we actually run the same bytecode we audited, CVE-2010-3880 - LP: #711865 - CVE-2010-3880 linux-ec2 (2.6.32-313.25) lucid-proposed; urgency=low [ Brad Figg ] * Tracking Bug - LP: #708890 [ Andrew Dickinson ] * SAUCE: sched: Prevent divide by zero when cpu_power is 0 - LP: #614853 [ Brad Figg ] * Rebased to 2.6.32-29.57 [ Stefan Bader ] * SAUCE: sched: Try tp catch cpu_power being set to 0 - LP: #614853 [ Upstream Kernel Changes ] * SRU: xen: events: do not unmask event channels on resume - LP: #681083 [ Ubuntu: 2.6.32-29.57 ] * Tracking Bug - LP: #708864 * [Config] Set CONFIG_NR_CPUS=256 for amd64 server - LP: #706058 * Input: i8042 - introduce 'notimeout' blacklist for Dell Vostro V13 - LP: #380126 * tun: avoid BUG, dump packet on GSO errors - LP: #698883 * TTY: Fix error return from tty_ldisc_open() - LP: #705045 * x86, hotplug: Use mwait to offline a processor, fix the legacy case - LP: #705045 * fuse: verify ioctl retries - LP: #705045 * fuse: fix ioctl when server is 32bit - LP: #705045 * ALSA: hda: Use model=lg quirk for LG P1 Express to enable playback and capture - LP: #595482, #705045 * nohz: Fix printk_needs_cpu() return value on offline cpus - LP: #705045 * nohz: Fix get_next_timer_interrupt() vs cpu hotplug - LP: #705045 * nfsd: Fix possible BUG_ON firing in set_change_info - LP: #705045 * NFS: Fix fcntl F_GETLK not reporting some conflicts - LP: #705045 * sunrpc: prevent use-after-free on clearing XPT_BUSY - LP: #705045 * hwmon: (adm1026) Allow 1 as a valid divider value - LP: #705045 * hwmon: (adm1026) Fix setting fan_div - LP: #705045 * amd64_edac: Fix interleaving check - LP: #705045 * IB/uverbs: Handle large number of entries in poll CQ - LP: #705045 * PM / Hibernate: Fix PM_POST_* notification with user-space suspend - LP: #705045 * ACPICA: Fix Scope() op in module level code - LP: #705045 * ACPI: EC: Add another dmi match entry for MSI hardware - LP: #705045 * orinoco: fix TKIP countermeasure behaviour - LP: #705045 * orinoco: clear countermeasure setting on commit - LP: #705045 * x86, amd: Fix panic on AMD CPU family 0x15 - LP: #705045 * md: fix bug with re-adding of partially recovered device. - LP: #705045 * tracing: Fix panic when lseek() called on "trace" opened for writing - LP: #705045 * x86, gcc-4.6: Use gcc -m options when building vdso - LP: #705045 * x86: Enable the intr-remap fault handling after local APIC setup - LP: #705045 * x86, vt-d: Handle previous faults after enabling fault handling - LP: #705045 * x86, vt-d: Fix the vt-d fault handling irq migration in the x2apic mode - LP: #705045 * x86, vt-d: Quirk for masking vtd spec errors to platform error handling logic - LP: #705045 * hvc_console: Fix race between hvc_close and hvc_remove - LP: #705045 * hvc_console: Fix race between hvc_close and hvc_remove, again - LP: #705045 * HID: hidraw: fix window in hidraw_release - LP: #705045 * bfa: fix system crash when reading sysfs fc_host statistics - LP: #705045 * net: release dst entry while cache-hot for GSO case too - LP: #705045 * install_special_mapping skips security_file_mmap check. - LP: #705045 * USB: misc: uss720.c: add another vendor/product ID - LP: #705045 * USB: ftdi_sio: Add D.O.Tec PID - LP: #705045 * USB: usb-storage: unusual_devs entry for the Samsung YP-CP3 - LP: #705045 * p54usb: add 5 more USBIDs - LP: #705045 * p54usb: New USB ID for Gemtek WUBI-100GW - LP: #705045 * sound: Prevent buffer overflow in OSS load_mixer_volumes - LP: #705045 * mv_xor: fix race in tasklet function - LP: #705045 * ima: fix add LSM rule bug - LP: #705045 * ALSA: hda: Use LPIB for Dell Latitude 131L - LP: #530346, #705045 * ALSA: hda: Use LPIB quirk for Dell Inspiron m101z/1120 - LP: #705045 * block: Deprecate QUEUE_FLAG_CLUSTER and use queue_li
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
** Tags added: verification-needed-lucid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
** Branch linked: lp:ubuntu/lucid-proposed/linux-ec2 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
Patch is in 2.6.32-313.25 ** Changed in: linux-ec2 (Ubuntu) Status: Confirmed => Fix Committed ** Changed in: linux-ec2 (Ubuntu) Assignee: (unassigned) => Stefan Bader (stefan-bader-canonical) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
** Changed in: linux Status: Unknown => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
SRU Justification: Impact: When trying to find the busiest group for the scheduler, there are rare (but it seems more likely in EC2) cases where cpu_power is zero when the code tries to divide by that variable. Fix: There is no real fix yet (and therefor both patches are not upstream) but users have tested the first patch which works around the issue by avoiding the divide whenever cpu_power actually is zero. The second patch is an optional companion to the first one which hopefully will yell when cpu_power is set to zero by accident. While it is neither a bug fix nor really needed I would like to add it, too. That way we could potentially catch the real bug in real usage (which seems to be the only way to get it after an extended period of time) and then revert both changes in future, when there is a fix. Testcase: Not being able to reproduce in test. But this has been reported to happen after around a week of uptime on production servers. (boot tested this approach to make sure this does not introduce obvious regressions by hitting the warning too often). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
Not yet, but in the end maybe the pragmatic approach will have to do until there is something better. I tried to reproduce this with the other patch from the upstream bug (to possible catch setting the value to zero) but have not been able to get anything. I have packages with those kernels at http://people.canonical.com/~smb/lp614853/ which could be used by booting with the pv-grub aki as described in https://lists.ubuntu.com/archives/ubuntu- cloud/2010-December/000466.html. If those being able to get the bug could try to do so with that kernel to see whether that adds more information for upstream. Meanwhile I would try to get the paper-over patch accepted for SRU. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
Has this been merged into 10.04? If not, the "paper over" patch should really get included in my opinion and then be replaced when the correct fix is available. Myself and others have been running the custom kernel that includes the fix for a while now with success. I guess I am a bit more pragmatic than the LKML guys in that I have to make sure my machines stay up or my bills don't get paid. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
We had at least 4 crashes related to this bug (all within 2 months). Attached the messages of the latest two panics. It's a DB server running postgres and a linux software raid10 setup for storage. On all occasions the machine had a higher load than normal ~20 - 30 (normally ~15), on the latest crash there was also a raid rebuild in the background. Running on AWS Instance: m2.2xlarge Region: EU-West Kernel-id: aki-4feec43b (2.6.32-309-ec2 kernel via pvgrub) Linux version 2.6.32-309-ec2 (bui...@yellow) (gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #18-Ubuntu SMP Mon Oct 18 21:00:50 UTC 2010 (Ubuntu 2.6.32-309.18-ec2 2.6.32.21+drm33.7) Will try to upgrade to linux-image-2.6.32-311-ec2 as there are a lot of changes in the sched code, although I didn't find anything that would address this issue explicitly. ** Attachment added: "oops.txt" https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/614853/+attachment/1784164/+files/oops.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
There was more action on the linux bug (https://bugzilla.kernel.org/show_bug.cgi?id=16991#c17), and a paper- over patch sent upstream http://lkml.indiana.edu/hypermail/linux/kernel/1010.2/02058.html . The upstream post got the expected response (no... fix it right). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
It has been reported that Bug #671001 was encountered before the ran the test kernel with the above patch. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
The patch posted above may be causing Bug #671001. The patch "fixes" this bug by simply checking for 0 before doing the division it does not address the underlying issue causing group->cpu_power to be 0 in the first place. So instead of oopsing at the divide by zero, the kernel continues until the underlying problem causes a different bug to surface. To be clear this is just speculation that this might be the cause, and has not be verified yet. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/614853 Title: kernel panic divide error: [#1] SMP -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
This patch seems solid, the panics don't seem to happen any longer on my machines. -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
** Tags added: patch -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
** Changed in: linux-ec2 (Ubuntu) Importance: Undecided => Medium ** Changed in: linux-ec2 (Ubuntu) Status: New => Confirmed -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
ubuntu-kernels-sandbox/ubuntu-lucid-amd64-linux- image-2.6.32-310-ec2_2.6.32-310.190-lp614853-kernel.img.manifest.xml I uploaded to each region john's kernel from http://kernel.ubuntu.com/~jj/linux-image-2.6.32-310-ec2_2.6.32-310.19~lp614853_amd64.deb us-west-1 aki-3e23737b x86_64 us-east-1 aki-2433c44d x86_64 eu-west-1 aki-6c063318 x86_64 ap-southeast-1 aki-d8740a8a x86_64 -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
I have been running this patch in production for a couple days and it seems solid thus far. I'm going to wait a few more days before I call it fixed though. -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
This is the patch from comment #17 backported to Lucid. ** Patch added: "lp614853.patch" https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/614853/+attachment/1729278/+files/lp614853.patch -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
** Also affects: linux via http://bugzilla.kernel.org/show_bug.cgi?id=16991 Importance: Unknown Status: Unknown -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
@Scott I do not believe its the same bug, see the discussion at https://bugzilla.kernel.org/show_bug.cgi?id=16991 I have gotten a patched kernel from canonical support and applied it to some of my machines this morning, we'll see if it will fix the panics. -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
@Joe, Do you think that this bug is a duplicate (or vice versa) of bug 651370 ? The thing that makes me think it might be is that your console log and all linked images show massive timestamps in the kernel at the time of the failure. Ie, "3229228" is ~ 897 hours uptime. Was your system up for anywheres near that long ? Maybe the timestamps is just aftermath of the failure. -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
I believe I have found this bug reported in the kernel bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=16991 Anything that can be done to expedite a fix is appreciated. ** Bug watch added: Linux Kernel Bug Tracker #16991 http://bugzilla.kernel.org/show_bug.cgi?id=16991 -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
I doubt they are related but figured it was worth mentioning last night we got soft lockups (not the divide by zero panics we've seen in the past) on a machine. Our hosting provider's KVM software didnt allow me to get the text but i got some screenshots. http://img.skitch.com/20100914-nkskuxfcucgrigj95bqqtbids1.jpg http://img.skitch.com/20100914-xir2hce4rt1p83m9jyy9agr4dk.jpg http://img.skitch.com/20100914-tx6nuuf86sp552u118m1uebcd.jpg >From the first function call in the trace it looks like its in the meta >information block cache. Maybe due to the spinlock or a bug in xfs? j...@der-dieb ~/Downloads/linux-2.6.32.21 $ ack mb_cache_shrink_fn . fs/mbcache.c 118:static int mb_cache_shrink_fn(int nr_to_scan, gfp_t gfp_mask); 121:.shrink = mb_cache_shrink_fn, 189: * mb_cache_shrink_fn() memory pressure callback 200:mb_cache_shrink_fn(int nr_to_scan, gfp_t gfp_mask) -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
Verified my disks are not CFQ, so it seems to effect all schedulers. $ cat /sys/block/*/queue/scheduler | grep -v none noop anticipatory [deadline] cfq noop anticipatory [deadline] cfq noop anticipatory [deadline] cfq noop anticipatory [deadline] cfq noop anticipatory [deadline] cfq noop anticipatory [deadline] cfq j...@der-dieb ~/Downloads/linux-2.6.32.21 $ ack "find_busiest_group" . kernel/sched.c 3389:/** Helpers for find_busiest_group / 4011:/*** find_busiest_group() helpers end here */ 4014: * find_busiest_group - Returns the busiest group within the sched_domain 4039:find_busiest_group(struct sched_domain *sd, int this_cpu, 4182: group = find_busiest_group(sd, this_cpu, &imbalance, idle, &sd_idle, 4206:* Attempt to move tasks. If find_busiest_group has found 4344: group = find_busiest_group(sd, this_cpu, &imbalance, CPU_NEWLY_IDLE, 4401:* find_busiest_group(). If there are no imbalance, then -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
apport information ** Description changed: I have seen this both on EC2 and physical hardware. I installed linux- crashdump on these machines to see if I can get more information but I will have to wait for another crash. divide error: [#1] SMP [1449293.452514] last sysfs file: /sys/devices/xen/vbd-16756/block/sdx4/stat [1449293.452518] CPU 0 [1449293.452521] Modules linked in: raid0 ipt_REJECT ipt_LOG xt_limit xt_tcpudp ipt_addrtype xt_state ip6table_filter ip6_tables ipv6 md_mod nf_nat_irc nf_conntrack_irc nf_nat_ftp nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack_ftp nf_conntrack iptable_filter ip_tables x_tables [1449293.452547] Pid: 10926, comm: beam.smp Not tainted 2.6.32-305-ec2 #9-Ubuntu [1449293.452551] RIP: e030:[] [] update_sd_lb_stats+0x3a4/0x4e0 [1449293.452563] RSP: e02b:880443e137e8 EFLAGS: 00010046 [1449293.452566] RAX: RBX: 880443e139d4 RCX: 0001 [1449293.452570] RDX: RSI: RDI: [1449293.452574] RBP: 880443e138c8 R08: 88000184dbc8 R09: 0040 [1449293.452578] R10: 880444358240 R11: R12: [1449293.452582] R13: a380 R14: R15: [1449293.452591] FS: 7f0f8fac3710() GS:880001846000() knlGS: [1449293.452595] CS: e033 DS: ES: CR0: 8005003b [1449293.452599] CR2: 7f0f80825008 CR3: 00043ed16000 CR4: 2620 [1449293.452603] DR0: DR1: DR2: [1449293.452607] DR3: DR6: 0ff0 DR7: [1449293.452611] Process beam.smp (pid: 10926, threadinfo 880443e12000, task 88043ee94200) [1449293.452616] Stack: [1449293.452618] 0246 880443e13868 2179d200 [1449293.452623] <0> 88000184daa0 0008 [1449293.452630] <0> a380 a380 88000184dbb0 a380 [1449293.452637] Call Trace: [1449293.452644] [] find_busiest_group+0x4d/0x460 [1449293.452652] [] ? do_mpage_readpage+0x330/0x630 [1449293.452656] [] load_balance_newidle+0xa4/0x320 [1449293.452662] [] thread_return+0x3cc/0x429 [1449293.452668] [] ? xfs_get_blocks+0x0/0x20 [1449293.452673] [] ? blk_unplug+0x2f/0x70 [1449293.452679] [] ? raid0_unplug+0x52/0x70 [raid0] [1449293.452685] [] ? sync_page_killable+0x0/0x40 [1449293.452689] [] io_schedule+0x42/0x60 [1449293.452693] [] sync_page+0x3d/0x50 [1449293.452697] [] sync_page_killable+0x9/0x40 [1449293.452701] [] __wait_on_bit_lock+0x52/0xb0 [1449293.452707] [] ? radix_tree_prev_hole+0x4d/0x60 [1449293.452711] [] __lock_page_killable+0x62/0x70 [1449293.452717] [] ? wake_bit_function+0x0/0x40 [1449293.452721] [] ? find_get_page+0x19/0xa0 [1449293.452725] [] T.769+0x1b7/0x410 [1449293.452729] [] generic_file_aio_read+0xb6/0x1d0 [1449293.452734] [] ? __down_read+0xf3/0x110 [1449293.452740] [] xfs_read+0x11a/0x2a0 [1449293.452747] [] ? unqueue_me+0x79/0xd0 [1449293.452751] [] ? futex_wait+0x257/0x290 [1449293.452756] [] xfs_file_aio_read+0x5b/0x70 [1449293.452761] [] do_sync_read+0xf2/0x130 [1449293.452765] [] ? autoremove_wake_function+0x0/0x40 [1449293.452770] [] ? futex_wake+0x112/0x130 [1449293.452776] [] ? security_file_permission+0x11/0x20 [1449293.452780] [] vfs_read+0xb5/0x1a0 [1449293.452784] [] sys_pread64+0x7a/0x90 [1449293.452790] [] system_call_fastpath+0x16/0x1b [1449293.452794] [] ? system_call+0x0/0x52 [1449293.452797] Code: 06 89 85 50 ff ff ff c7 85 54 ff ff ff 01 00 00 00 e9 cf fd ff ff 90 48 8b 95 70 ff ff ff 48 8b 45 a8 8b 72 08 48 c1 e0 0a 31 d2 <48> f7 f6 48 8b 75 b0 48 89 45 a0 31 c0 48 85 f6 74 0c 48 8b 45 [1449293.452837] RIP [] update_sd_lb_stats+0x3a4/0x4e0 [1449293.452842] RSP [1449293.452848] ---[ end trace 9b3628a023db21fb ]--- ProblemType: Bug DistroRelease: Ubuntu 10.04 Package: linux-image-2.6.32-305-ec2 2.6.32-305.9 ProcVersionSignature: Ubuntu 2.6.32-305.9-ec2 2.6.32.11+drm33.2 Uname: Linux 2.6.32-305-ec2 x86_64 Architecture: amd64 Date: Sat Aug 7 21:55:31 2010 Ec2AMI: ami-fd4aa494 Ec2AMIManifest: ubuntu-images-us/ubuntu-lucid-10.04-amd64-server-20100427.1.manifest.xml Ec2AvailabilityZone: us-east-1b Ec2InstanceType: m2.xlarge Ec2Kernel: aki-0b4aa462 Ec2Ramdisk: unavailable ProcEnviron: PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash SourcePackage: linux-ec2 --- Architecture: amd64 DistroRelease: Ubuntu 10.04 Ec2AMI: ami-c997c68c Ec2AMIManifest: ubuntu-images-us-west-1/ubuntu-lucid-10.04-amd64-server-20100427.1.manifest.xml Ec2AvailabilityZone: us-west-1b Ec2InstanceType: m2.xlarge Ec2Kernel: aki-c397c686 Ec2Ramdisk: unavailable Package: linux-ec2 2.6.32.30
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
Got it again ... ** Attachment added: "panic.log" https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/614853/+attachment/1545147/+files/panic.log -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
** Attachment added: "lsmod" https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/614853/+attachment/1543562/+files/lsmod -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
I ran "apport-collect 614853" on the aformentioned EC2 node and all it seemed to produce was the above dependency list. I have attached uname and lsmod should they be helpful. ** Attachment added: "uname" https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/614853/+attachment/1543561/+files/uname -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
apport information ** Description changed: I have seen this both on EC2 and physical hardware. I installed linux- crashdump on these machines to see if I can get more information but I will have to wait for another crash. divide error: [#1] SMP [1449293.452514] last sysfs file: /sys/devices/xen/vbd-16756/block/sdx4/stat [1449293.452518] CPU 0 [1449293.452521] Modules linked in: raid0 ipt_REJECT ipt_LOG xt_limit xt_tcpudp ipt_addrtype xt_state ip6table_filter ip6_tables ipv6 md_mod nf_nat_irc nf_conntrack_irc nf_nat_ftp nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack_ftp nf_conntrack iptable_filter ip_tables x_tables [1449293.452547] Pid: 10926, comm: beam.smp Not tainted 2.6.32-305-ec2 #9-Ubuntu [1449293.452551] RIP: e030:[] [] update_sd_lb_stats+0x3a4/0x4e0 [1449293.452563] RSP: e02b:880443e137e8 EFLAGS: 00010046 [1449293.452566] RAX: RBX: 880443e139d4 RCX: 0001 [1449293.452570] RDX: RSI: RDI: [1449293.452574] RBP: 880443e138c8 R08: 88000184dbc8 R09: 0040 [1449293.452578] R10: 880444358240 R11: R12: [1449293.452582] R13: a380 R14: R15: [1449293.452591] FS: 7f0f8fac3710() GS:880001846000() knlGS: [1449293.452595] CS: e033 DS: ES: CR0: 8005003b [1449293.452599] CR2: 7f0f80825008 CR3: 00043ed16000 CR4: 2620 [1449293.452603] DR0: DR1: DR2: [1449293.452607] DR3: DR6: 0ff0 DR7: [1449293.452611] Process beam.smp (pid: 10926, threadinfo 880443e12000, task 88043ee94200) [1449293.452616] Stack: [1449293.452618] 0246 880443e13868 2179d200 [1449293.452623] <0> 88000184daa0 0008 [1449293.452630] <0> a380 a380 88000184dbb0 a380 [1449293.452637] Call Trace: [1449293.452644] [] find_busiest_group+0x4d/0x460 [1449293.452652] [] ? do_mpage_readpage+0x330/0x630 [1449293.452656] [] load_balance_newidle+0xa4/0x320 [1449293.452662] [] thread_return+0x3cc/0x429 [1449293.452668] [] ? xfs_get_blocks+0x0/0x20 [1449293.452673] [] ? blk_unplug+0x2f/0x70 [1449293.452679] [] ? raid0_unplug+0x52/0x70 [raid0] [1449293.452685] [] ? sync_page_killable+0x0/0x40 [1449293.452689] [] io_schedule+0x42/0x60 [1449293.452693] [] sync_page+0x3d/0x50 [1449293.452697] [] sync_page_killable+0x9/0x40 [1449293.452701] [] __wait_on_bit_lock+0x52/0xb0 [1449293.452707] [] ? radix_tree_prev_hole+0x4d/0x60 [1449293.452711] [] __lock_page_killable+0x62/0x70 [1449293.452717] [] ? wake_bit_function+0x0/0x40 [1449293.452721] [] ? find_get_page+0x19/0xa0 [1449293.452725] [] T.769+0x1b7/0x410 [1449293.452729] [] generic_file_aio_read+0xb6/0x1d0 [1449293.452734] [] ? __down_read+0xf3/0x110 [1449293.452740] [] xfs_read+0x11a/0x2a0 [1449293.452747] [] ? unqueue_me+0x79/0xd0 [1449293.452751] [] ? futex_wait+0x257/0x290 [1449293.452756] [] xfs_file_aio_read+0x5b/0x70 [1449293.452761] [] do_sync_read+0xf2/0x130 [1449293.452765] [] ? autoremove_wake_function+0x0/0x40 [1449293.452770] [] ? futex_wake+0x112/0x130 [1449293.452776] [] ? security_file_permission+0x11/0x20 [1449293.452780] [] vfs_read+0xb5/0x1a0 [1449293.452784] [] sys_pread64+0x7a/0x90 [1449293.452790] [] system_call_fastpath+0x16/0x1b [1449293.452794] [] ? system_call+0x0/0x52 [1449293.452797] Code: 06 89 85 50 ff ff ff c7 85 54 ff ff ff 01 00 00 00 e9 cf fd ff ff 90 48 8b 95 70 ff ff ff 48 8b 45 a8 8b 72 08 48 c1 e0 0a 31 d2 <48> f7 f6 48 8b 75 b0 48 89 45 a0 31 c0 48 85 f6 74 0c 48 8b 45 [1449293.452837] RIP [] update_sd_lb_stats+0x3a4/0x4e0 [1449293.452842] RSP [1449293.452848] ---[ end trace 9b3628a023db21fb ]--- ProblemType: Bug DistroRelease: Ubuntu 10.04 Package: linux-image-2.6.32-305-ec2 2.6.32-305.9 ProcVersionSignature: Ubuntu 2.6.32-305.9-ec2 2.6.32.11+drm33.2 Uname: Linux 2.6.32-305-ec2 x86_64 Architecture: amd64 Date: Sat Aug 7 21:55:31 2010 Ec2AMI: ami-fd4aa494 Ec2AMIManifest: ubuntu-images-us/ubuntu-lucid-10.04-amd64-server-20100427.1.manifest.xml Ec2AvailabilityZone: us-east-1b Ec2InstanceType: m2.xlarge Ec2Kernel: aki-0b4aa462 Ec2Ramdisk: unavailable ProcEnviron: PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash SourcePackage: linux-ec2 --- Architecture: amd64 DistroRelease: Ubuntu 10.04 Ec2AMI: ami-c997c68c Ec2AMIManifest: ubuntu-images-us-west-1/ubuntu-lucid-10.04-amd64-server-20100427.1.manifest.xml Ec2AvailabilityZone: us-west-1b Ec2InstanceType: m2.xlarge Ec2Kernel: aki-c397c686 Ec2Ramdisk: unavailable Package: linux-ec2 2.6.32.30
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
Not sure if it's related but I noticed the following on boot up of the EC2 machine: Checking for running unattended-upgrades: [ 132.079264] BUG: soft lockup - CPU#0 stuck for 61s! [udevd:219] [ 197.577155] BUG: soft lockup - CPU#0 stuck for 61s! [udevd:219] [ 240.073502] INFO: task mount:609 blocked for more than 120 seconds. [ 240.073513] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 240.073609] INFO: task sync:627 blocked for more than 120 seconds. [ 240.073613] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 263.074746] BUG: soft lockup - CPU#0 stuck for 61s! [udevd:219] [ 328.573703] BUG: soft lockup - CPU#0 stuck for 61s! [udevd:219] -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
Here's another panic screenshot (physical hardware) and console output (EC2). http://img.skitch.com/20100904-bitg4476jipband75g38g5wjcb.jpg ** Attachment added: "panic.log" https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/614853/+attachment/1543483/+files/panic.log -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
I have confirmed this happens with the deadline IO scheduler. Today an EC2 node of ours running deadline on all the disks got the same "divide error: [#1] SMP" panic. -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
In an attempt to figure out the issue I decided to change the IO scheduler thinking it might help considering the contents of the trace. I set one group of nodes to noop and another to deadline. I have seen panics on both groups of machines since doing so. From the (partial) traces I've gotten from those machines the noop trace looks quite a bit different while the deadline trace looks pretty similar with lots of bits regarding xfs. Screenshots: http://img.skitch.com/20100816-buaaqf6ggdfp6m8y41x4wfyhfy.jpg http://img.skitch.com/20100816-dnwh8sijt8jnck5k18ewrdcdeu.jpg Unfortunately the console in the remote management card cuts the top of the trace off. -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
apport information ** Tags added: apport-collected ** Description changed: I have seen this both on EC2 and physical hardware. I installed linux- crashdump on these machines to see if I can get more information but I will have to wait for another crash. divide error: [#1] SMP [1449293.452514] last sysfs file: /sys/devices/xen/vbd-16756/block/sdx4/stat [1449293.452518] CPU 0 [1449293.452521] Modules linked in: raid0 ipt_REJECT ipt_LOG xt_limit xt_tcpudp ipt_addrtype xt_state ip6table_filter ip6_tables ipv6 md_mod nf_nat_irc nf_conntrack_irc nf_nat_ftp nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack_ftp nf_conntrack iptable_filter ip_tables x_tables [1449293.452547] Pid: 10926, comm: beam.smp Not tainted 2.6.32-305-ec2 #9-Ubuntu [1449293.452551] RIP: e030:[] [] update_sd_lb_stats+0x3a4/0x4e0 [1449293.452563] RSP: e02b:880443e137e8 EFLAGS: 00010046 [1449293.452566] RAX: RBX: 880443e139d4 RCX: 0001 [1449293.452570] RDX: RSI: RDI: [1449293.452574] RBP: 880443e138c8 R08: 88000184dbc8 R09: 0040 [1449293.452578] R10: 880444358240 R11: R12: [1449293.452582] R13: a380 R14: R15: [1449293.452591] FS: 7f0f8fac3710() GS:880001846000() knlGS: [1449293.452595] CS: e033 DS: ES: CR0: 8005003b [1449293.452599] CR2: 7f0f80825008 CR3: 00043ed16000 CR4: 2620 [1449293.452603] DR0: DR1: DR2: [1449293.452607] DR3: DR6: 0ff0 DR7: [1449293.452611] Process beam.smp (pid: 10926, threadinfo 880443e12000, task 88043ee94200) [1449293.452616] Stack: [1449293.452618] 0246 880443e13868 2179d200 [1449293.452623] <0> 88000184daa0 0008 [1449293.452630] <0> a380 a380 88000184dbb0 a380 [1449293.452637] Call Trace: [1449293.452644] [] find_busiest_group+0x4d/0x460 [1449293.452652] [] ? do_mpage_readpage+0x330/0x630 [1449293.452656] [] load_balance_newidle+0xa4/0x320 [1449293.452662] [] thread_return+0x3cc/0x429 [1449293.452668] [] ? xfs_get_blocks+0x0/0x20 [1449293.452673] [] ? blk_unplug+0x2f/0x70 [1449293.452679] [] ? raid0_unplug+0x52/0x70 [raid0] [1449293.452685] [] ? sync_page_killable+0x0/0x40 [1449293.452689] [] io_schedule+0x42/0x60 [1449293.452693] [] sync_page+0x3d/0x50 [1449293.452697] [] sync_page_killable+0x9/0x40 [1449293.452701] [] __wait_on_bit_lock+0x52/0xb0 [1449293.452707] [] ? radix_tree_prev_hole+0x4d/0x60 [1449293.452711] [] __lock_page_killable+0x62/0x70 [1449293.452717] [] ? wake_bit_function+0x0/0x40 [1449293.452721] [] ? find_get_page+0x19/0xa0 [1449293.452725] [] T.769+0x1b7/0x410 [1449293.452729] [] generic_file_aio_read+0xb6/0x1d0 [1449293.452734] [] ? __down_read+0xf3/0x110 [1449293.452740] [] xfs_read+0x11a/0x2a0 [1449293.452747] [] ? unqueue_me+0x79/0xd0 [1449293.452751] [] ? futex_wait+0x257/0x290 [1449293.452756] [] xfs_file_aio_read+0x5b/0x70 [1449293.452761] [] do_sync_read+0xf2/0x130 [1449293.452765] [] ? autoremove_wake_function+0x0/0x40 [1449293.452770] [] ? futex_wake+0x112/0x130 [1449293.452776] [] ? security_file_permission+0x11/0x20 [1449293.452780] [] vfs_read+0xb5/0x1a0 [1449293.452784] [] sys_pread64+0x7a/0x90 [1449293.452790] [] system_call_fastpath+0x16/0x1b [1449293.452794] [] ? system_call+0x0/0x52 [1449293.452797] Code: 06 89 85 50 ff ff ff c7 85 54 ff ff ff 01 00 00 00 e9 cf fd ff ff 90 48 8b 95 70 ff ff ff 48 8b 45 a8 8b 72 08 48 c1 e0 0a 31 d2 <48> f7 f6 48 8b 75 b0 48 89 45 a0 31 c0 48 85 f6 74 0c 48 8b 45 [1449293.452837] RIP [] update_sd_lb_stats+0x3a4/0x4e0 [1449293.452842] RSP [1449293.452848] ---[ end trace 9b3628a023db21fb ]--- ProblemType: Bug DistroRelease: Ubuntu 10.04 Package: linux-image-2.6.32-305-ec2 2.6.32-305.9 ProcVersionSignature: Ubuntu 2.6.32-305.9-ec2 2.6.32.11+drm33.2 Uname: Linux 2.6.32-305-ec2 x86_64 Architecture: amd64 Date: Sat Aug 7 21:55:31 2010 Ec2AMI: ami-fd4aa494 Ec2AMIManifest: ubuntu-images-us/ubuntu-lucid-10.04-amd64-server-20100427.1.manifest.xml Ec2AvailabilityZone: us-east-1b Ec2InstanceType: m2.xlarge Ec2Kernel: aki-0b4aa462 Ec2Ramdisk: unavailable ProcEnviron: PATH=(custom, no user) LANG=en_US.UTF-8 SHELL=/bin/bash SourcePackage: linux-ec2 + --- + Architecture: amd64 + DistroRelease: Ubuntu 10.04 + Ec2AMI: ami-c997c68c + Ec2AMIManifest: ubuntu-images-us-west-1/ubuntu-lucid-10.04-amd64-server-20100427.1.manifest.xml + Ec2AvailabilityZone: us-west-1b + Ec2InstanceType: m2.xlarge + Ec2Kernel: aki-c397c686 + Ec2Ramdisk: unavailab
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
All of these servers are doing high throughput database work, specifically CouchDB. -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
Joe, what kind of work loads are you running to trigger this? also after you hit this bug again could you run apport-collect 614853 -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
I have been unable to collect a core using linux-crashdump on my physical machines, it doesn't seem dump it and reboot automatically. However it does seem to load the crash kernel (kdump init script). I did collect another stack trace from one of my EC2 machines: [2498228.006101] divide error: [#1] SMP [2498228.006113] last sysfs file: /sys/devices/xen/vbd-16756/block/sdx4/stat [2498228.006117] CPU 0 [2498228.006120] Modules linked in: btrfs zlib_deflate crc32c libcrc32c ufs qnx4 hfsplus hfs minix ntfs vfat msdos fat raid0 ipt_REJECT ipt_LOG xt_limit xt_tcpudp ipt_addrtype xt_state ip6table_filter ip6_tables ipv6 md_mod nf_nat_irc nf_conntrack_irc nf_nat_ftp nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack_ftp nf_conntrack iptable_filter ip_tables x_tables [2498228.006157] Pid: 2128, comm: beam.smp Not tainted 2.6.32-305-ec2 #9-Ubuntu [2498228.006161] RIP: e030:[] [] update_sd_lb_stats+0x3a4/0x4e0 [2498228.006172] RSP: e02b:88044188f9f8 EFLAGS: 00010046 [2498228.006176] RAX: RBX: 88044188fbe4 RCX: 0001 [2498228.006179] RDX: RSI: RDI: [2498228.006183] RBP: 88044188fad8 R08: 88000184dbc8 R09: 0040 [2498228.006187] R10: R11: R12: [2498228.006191] R13: a380 R14: R15: [2498228.006199] FS: 7fd37084f710() GS:880001846000() knlGS: [2498228.006204] CS: e033 DS: ES: CR0: 8005003b [2498228.006207] CR2: 7fd35e7bd000 CR3: 000440f9b000 CR4: 2620 [2498228.006211] DR0: DR1: DR2: [2498228.006215] DR3: DR6: 0ff0 DR7: [2498228.006220] Process beam.smp (pid: 2128, threadinfo 88044188e000, task 880440f982c0) [2498228.006224] Stack: [2498228.006226] 8804407fec40 88044188fa78 88044188fb18 [2498228.006231] <0> 88000184daa0 0008 [2498228.006238] <0> a380 a380 88000184dbb0 a380 [2498228.006245] Call Trace: [2498228.006251] [] find_busiest_group+0x4d/0x460 [2498228.006258] [] ? __wait_on_bit_lock+0x73/0xb0 [2498228.006262] [] load_balance_newidle+0xa4/0x320 [2498228.006266] [] thread_return+0x3cc/0x429 [2498228.006272] [] ? __up_read+0x9a/0xc0 [2498228.006277] [] ? get_futex_value_locked+0x27/0x40 [2498228.006282] [] futex_wait_queue_me+0xcd/0x110 [2498228.006286] [] futex_wait+0x128/0x290 [2498228.006291] [] ? _spin_lock+0x2d/0x60 [2498228.006295] [] ? futex_wake+0x112/0x130 [2498228.006299] [] do_futex+0xc9/0x1b0 [2498228.006303] [] sys_futex+0x76/0x170 [2498228.006308] [] ? sys_pread64+0x88/0x90 [2498228.006315] [] system_call_fastpath+0x16/0x1b [2498228.006319] [] ? system_call+0x0/0x52 [2498228.006322] Code: 06 89 85 50 ff ff ff c7 85 54 ff ff ff 01 00 00 00 e9 cf fd ff ff 90 48 8b 95 70 ff ff ff 48 8b 45 a8 8b 72 08 48 c1 e0 0a 31 d2 <48> f7 f6 48 8b 75 b0 48 89 45 a0 31 c0 48 85 f6 74 0c 48 8b 45 [2498228.006363] RIP [] update_sd_lb_stats+0x3a4/0x4e0 [2498228.006368] RSP [2498228.006372] ---[ end trace 18faee40e07dc443 ]--- -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
On the physical hardware I am running 2.6.32-24-generic, without any virtualization of any sort. -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
Joe, can you elaborate on which kernel and the setup you were using when you saw this on physical hardware, ie. were you running lucid's generic kernel on physical hardware, or where you running the ec2 kernel under a Xen dom0, etc. -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 614853] Re: kernel panic divide error: 0000 [#1] SMP
** Attachment added: "Dependencies.txt" http://launchpadlibrarian.net/53250006/Dependencies.txt -- kernel panic divide error: [#1] SMP https://bugs.launchpad.net/bugs/614853 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
