[Bug 1483343] Comment bridged from LTC Bugzilla
--- Comment From boqun.f...@cn.ibm.com 2017-02-06 03:07 EDT--- (In reply to comment #71) > Hi, > > Ran runltp on 17.04, and ended up with continuous call traces on console. > > ---uname output--- > Linux ltc-garri1 4.9.0-15-generic #16-Ubuntu SMP Fri Jan 20 15:28:49 UTC > 2017 ppc64le ppc64le ppc64le GNU/Linux > > ---Call Trace--- > [332850.158228] Call Trace: > [332850.158253] [c01fcec4fb50] [c001c3a0] __switch_to+0x2e0/0x4c0 > [332850.159375] [c01fcec4fbb0] [c0b193a0] __schedule+0x300/0x990 > [332850.159872] [c01fcec4fc90] [c0b19e88] _cond_resched+0x78/0xa0 > [332850.160475] [c01fcec4fcc0] [d0001c810278] > torture_rwlock_write_lock_irq+0x30/0x60 [locktorture] This one is weird, because torture_rwlock_write_lock_irq will acquire a lock with irq disabled, so it will not get any change to reschedule.. And the asm code for /lib/modules/4.9.0-15-generic/kernel/kernel/locking/locktorture.ko: 0248 : 248: f8 ff 4c e8 ld r2,-8(r12) 248: R_PPC64_ENTRY *ABS* 24c: 14 62 42 7c add r2,r2,r12 250: a6 02 08 7c mflrr0 254: 01 00 00 48 bl 254 254: R_PPC64_REL24 _mcount 258: a6 02 08 7c mflrr0 25c: f8 ff e1 fb std r31,-8(r1) 260: 10 00 01 f8 std r0,16(r1) 264: d1 ff 21 f8 stdur1,-48(r1) 268: 00 00 22 3d addis r9,r2,0 268: R_PPC64_TOC16_HA .toc 26c: 00 00 e9 eb ld r31,0(r9) 26c: R_PPC64_TOC16_LO_DS.toc 270: 30 00 7f 38 addir3,r31,48 274: 01 00 00 48 bl 274 274: R_PPC64_REL24 _raw_write_lock_irqsave 278: 00 00 00 60 nop 27c: 18 00 3f e9 ld r9,24(r31) 280: 40 00 69 f8 std r3,64(r9) 284: 00 00 60 38 li r3,0 288: 30 00 21 38 addir1,r1,48 so torture_rwlock_write_lock_irq+0x30/0x60 is 278: 00 00 00 60 nop right after the return of call to _raw_write_lock_irqsave, which should be fixed up by the instruction that recover the toc, IIUC. How that end up being a call to _cond_resched() is still unknown. Besides, Pooja, I cannot reproduce this in a PowerKVM guest with 16 VCPUs. What's your setup for the machine that hit the problem? And how many times you need to run the testcase in order to hit one? Regards, Boqun > [332850.161157] [c01fcec4fcf0] [d0001c810988] > lock_torture_writer+0xc0/0x270 [locktorture] > [332850.170046] [c01fcec4fd80] [c00f53c0] kthread+0x110/0x130 > [332850.173108] [c01fcec4fe30] [c000c0e8] > ret_from_kernel_thread+0x5c/0x74 > [332850.174169] Task dump for CPU 10: > [332850.174202] lock_torture_wr R running task0 37019 2 > 0x0804 > [332850.174734] Call Trace: > [332850.175218] [c01fcf3ab980] [c00d0008] irq_exit+0xe8/0x120 > (unreliable) > [332850.175813] [c01fcf3ab9a0] [c0024e10] > timer_interrupt+0xa0/0xe0 > [332850.176438] --- interrupt: d0001c6008c0 at > finish_task_switch+0xb0/0x340 > [332850.176438] LR = stutter_wait+0x8c/0x130 [torture] > [332850.177950] [c01fcf3ab9d0] [c0009514] > decrementer_common+0x114/0x180 (unreliable) > [332850.178517] --- interrupt: 901 at torture_shutdown_init+0x2a4/0x770 > [torture] > [332850.178517] LR = stutter_wait+0x8c/0x130 [torture] > [332850.179557] [c01fcf3abcc0] [d0001c810278] > torture_rwlock_write_lock_irq+0x30/0x60 [locktorture] (unreliable) > [332850.180675] [c01fcf3abcf0] [d0001c810988] > lock_torture_writer+0xc0/0x270 [locktorture] > [332850.181646] [c01fcf3abd80] [c00f53c0] kthread+0x110/0x130 > [332850.194048] [c01fcf3abe30] [c000c0e8] > ret_from_kernel_thread+0x5c/0x74 > [332850.196658] Task dump for CPU 12: > [332850.198840] lock_torture_wr R running task0 36921 2 > 0x0804 > [332850.201377] Call Trace: > [332850.201402] [c01fc3747890] [c016127c] > hrtimer_interrupt+0x11c/0x2a0 (unreliable) > [332850.206296] [c01fc3747a60] [c001c3a0] __switch_to+0x2e0/0x4c0 > [332850.208647] [c01fc3747ac0] [c0b193a0] __schedule+0x300/0x990 > [332850.211007] [c01fc3747b00] [c015bc78] > lock_timer_base+0xa8/0x100 > [332850.214641] [c01fc3747b60] [c015bde8] > try_to_del_timer_sync+0x68/0xa0 > [332850.215236] [c01fc3747ba0] [c015be90] > del_timer_sync+0x70/0x90 > [332850.215817] [c01fc3747bd0] [c0b1de14] > schedule_timeout+0x234/0x470 > [332850.216369] [c01fc3747cc0] [d0001c810278] > torture_rwlock_write_lock_irq+0x30/0x60 [locktorture] > [332850.217397] [c01fc3747cf0] [d0001c810988] > lock_torture_writer+0xc0/0x270 [locktorture] > [332850.218433] [c01fc3747d80] [c00f53c0] kthread+0x110/0x130 > [332850.219013] [c01fc3747e30] [c000c0e8] > ret_from_kernel_thread+0x5c/0x74 > > > Thanks, > Pooja -- You received this bug notification because you are a member of Ubuntu B
[Bug 1483343] Comment bridged from LTC Bugzilla
--- Comment From poojabsu...@in.ibm.com 2017-02-06 01:36 EDT--- Hi, Ran runltp on 17.04, and ended up with continuous call traces on console. ---uname output--- Linux ltc-garri1 4.9.0-15-generic #16-Ubuntu SMP Fri Jan 20 15:28:49 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux ---Call Trace--- [332850.158228] Call Trace: [332850.158253] [c01fcec4fb50] [c001c3a0] __switch_to+0x2e0/0x4c0 [332850.159375] [c01fcec4fbb0] [c0b193a0] __schedule+0x300/0x990 [332850.159872] [c01fcec4fc90] [c0b19e88] _cond_resched+0x78/0xa0 [332850.160475] [c01fcec4fcc0] [d0001c810278] torture_rwlock_write_lock_irq+0x30/0x60 [locktorture] [332850.161157] [c01fcec4fcf0] [d0001c810988] lock_torture_writer+0xc0/0x270 [locktorture] [332850.170046] [c01fcec4fd80] [c00f53c0] kthread+0x110/0x130 [332850.173108] [c01fcec4fe30] [c000c0e8] ret_from_kernel_thread+0x5c/0x74 [332850.174169] Task dump for CPU 10: [332850.174202] lock_torture_wr R running task0 37019 2 0x0804 [332850.174734] Call Trace: [332850.175218] [c01fcf3ab980] [c00d0008] irq_exit+0xe8/0x120 (unreliable) [332850.175813] [c01fcf3ab9a0] [c0024e10] timer_interrupt+0xa0/0xe0 [332850.176438] --- interrupt: d0001c6008c0 at finish_task_switch+0xb0/0x340 [332850.176438] LR = stutter_wait+0x8c/0x130 [torture] [332850.177950] [c01fcf3ab9d0] [c0009514] decrementer_common+0x114/0x180 (unreliable) [332850.178517] --- interrupt: 901 at torture_shutdown_init+0x2a4/0x770 [torture] [332850.178517] LR = stutter_wait+0x8c/0x130 [torture] [332850.179557] [c01fcf3abcc0] [d0001c810278] torture_rwlock_write_lock_irq+0x30/0x60 [locktorture] (unreliable) [332850.180675] [c01fcf3abcf0] [d0001c810988] lock_torture_writer+0xc0/0x270 [locktorture] [332850.181646] [c01fcf3abd80] [c00f53c0] kthread+0x110/0x130 [332850.194048] [c01fcf3abe30] [c000c0e8] ret_from_kernel_thread+0x5c/0x74 [332850.196658] Task dump for CPU 12: [332850.198840] lock_torture_wr R running task0 36921 2 0x0804 [332850.201377] Call Trace: [332850.201402] [c01fc3747890] [c016127c] hrtimer_interrupt+0x11c/0x2a0 (unreliable) [332850.206296] [c01fc3747a60] [c001c3a0] __switch_to+0x2e0/0x4c0 [332850.208647] [c01fc3747ac0] [c0b193a0] __schedule+0x300/0x990 [332850.211007] [c01fc3747b00] [c015bc78] lock_timer_base+0xa8/0x100 [332850.214641] [c01fc3747b60] [c015bde8] try_to_del_timer_sync+0x68/0xa0 [332850.215236] [c01fc3747ba0] [c015be90] del_timer_sync+0x70/0x90 [332850.215817] [c01fc3747bd0] [c0b1de14] schedule_timeout+0x234/0x470 [332850.216369] [c01fc3747cc0] [d0001c810278] torture_rwlock_write_lock_irq+0x30/0x60 [locktorture] [332850.217397] [c01fc3747cf0] [d0001c810988] lock_torture_writer+0xc0/0x270 [locktorture] [332850.218433] [c01fc3747d80] [c00f53c0] kthread+0x110/0x130 [332850.219013] [c01fc3747e30] [c000c0e8] ret_from_kernel_thread+0x5c/0x74 Thanks, Pooja -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1483343 Title: NMI watchdog: BUG: soft lockup errors when we execute lock_torture_wr tests To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1483343/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1483343] Comment bridged from LTC Bugzilla
--- Comment From ckuma...@in.ibm.com 2016-10-06 01:24 EDT--- Externalizing comment 61 and 62 - The Same test is causing problem in ubuntu kernel version 4.8.0-17-generic. Got below call traces: [48755.933755] 112-...: (2 ticks this GP) idle=b95/140/0 softirq=11738/11738 fqs=2829432 [48755.933979] 118-...: (5 ticks this GP) idle=011/140/0 softirq=6041/6041 fqs=2829432 [48755.934264] (detected by 29, t=11947979 jiffies, g=5047, c=5046, q=1984242) [48692.907932] Call Trace: [48692.907934] [c017e8a83980] [c00d57f8] irq_exit+0x98/0x100 (unreliable) [48692.907936] [c017e8a839a0] [c0024810] timer_interrupt+0xa0/0xe0 [48692.907938] [c017e8a839d0] [c0002814] decrementer_common+0x114/0x180 [48692.907942] --- interrupt: 901 at _cond_resched+0x0/0xa0 LR = stutter_wait+0x8c/0x130 [torture] [48692.907945] [c017e8a83cc0] [d000158e0348] torture_rwlock_write_lock_irq+0x30/0x60 [locktorture] (unreliable) [48692.907948] [c017e8a83cf0] [d000158e0a88] lock_torture_writer+0xc0/0x270 [locktorture] [48692.907950] [c017e8a83d80] [c00fd120] kthread+0x110/0x130 [48692.907954] [c017e8a83e30] [c00098f0] ret_from_kernel_thread+0x5c/0x6c This call traces keeps displaying from the time we execute the script and system is in hanged state. --- Issue is seen in ubuntu 16.10 as well. --- Canonical, May we have some attention on this issue. Thanks, Chandan -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1483343 Title: NMI watchdog: BUG: soft lockup errors when we execute lock_torture_wr tests To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1483343/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1483343] Comment bridged from LTC Bugzilla
--- Comment From ckuma...@in.ibm.com 2016-05-30 01:57 EDT--- (In reply to comment #52) > I'll try to reproduce the bug, so we can perform a reverse bisect and > identify the fix for this bug. > > Does this bug only happen on ppc64le? According to comment #31, this may > also be happening on x86_64? I can confirm only for ppl64le, not sure about x86_64 as we have not tested on x86_64 but based on LP comment#31 it seems the issue happens on x86_64 as well. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1483343 Title: NMI watchdog: BUG: soft lockup errors when we execute lock_torture_wr tests To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1483343/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1483343] Comment bridged from LTC Bugzilla
--- Comment From pavsu...@in.ibm.com 2016-04-19 01:25 EDT--- I don't have any machine currently for repeatedly testing with different upstream kernels as and when they are released. Since the test steps are quite simple and straight forward, I think it would be better to re-create the issue at your end with different kernels as required. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1483343 Title: NMI watchdog: BUG: soft lockup errors when we execute lock_torture_wr tests To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1483343/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1483343] Comment bridged from LTC Bugzilla
--- Comment From pavsu...@in.ibm.com 2016-04-06 07:38 EDT--- I have installed the 4.6-rc1 test kernel and executed the tests on a Bare Metal Open Power Hardware. I could not see the NMI softlockup call traces occurring. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1483343 Title: NMI watchdog: BUG: soft lockup errors when we execute lock_torture_wr tests To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1483343/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1483343] Comment bridged from LTC Bugzilla
--- Comment From pavsu...@in.ibm.com 2016-01-16 05:05 EDT--- I have copied lock_torture script on to the P8 server with the 3.13 kernel installed. Then executed the test script again in-order to re-create the scenario. root@powerkvmpok002:/opt/ltp/testcases/bin# ./lock_torture.sh lock_torture 1 TCONF : Test requires locktorture module lock_torture 1 TINFO : cleanup root@powerkvmpok002:/opt/ltp/testcases/bin# modprobe locktorture modprobe: FATAL: Module locktorture not found. As can be seen above the locktorture module is required in 3.13 kernel. I think, this module is not supported in the 3.13 kernel. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1483343 Title: NMI watchdog: BUG: soft lockup errors when we execute lock_torture_wr tests To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1483343/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1483343] Comment bridged from LTC Bugzilla
--- Comment From pavsu...@in.ibm.com 2016-01-11 01:01 EDT--- I have installed 3.18 kernel provided in the link: https://launchpad.net/ubuntu/+source/linux/3.18.0-8.9/+build/6690705 root@powerkvmpok002:~/6690705# dpkg -i linux-image-3.18.0-8-generic_3.18.0-8.9_ppc64el.deb Selecting previously unselected package linux-image-3.18.0-8-generic. (Reading database ... 164194 files and directories currently installed.) Preparing to unpack linux-image-3.18.0-8-generic_3.18.0-8.9_ppc64el.deb ... Done. Unpacking linux-image-3.18.0-8-generic (3.18.0-8.9) ... Setting up linux-image-3.18.0-8-generic (3.18.0-8.9) ... Running depmod. update-initramfs: deferring update (hook will be called later) Examining /etc/kernel/postinst.d. run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 3.18.0-8-generic /boot/vmlinux-3.18.0-8-generic run-parts: executing /etc/kernel/postinst.d/initramfs-tools 3.18.0-8-generic /boot/vmlinux-3.18.0-8-generic update-initramfs: Generating /boot/initrd.img-3.18.0-8-generic run-parts: executing /etc/kernel/postinst.d/update-notifier 3.18.0-8-generic /boot/vmlinux-3.18.0-8-generic run-parts: executing /etc/kernel/postinst.d/zz-update-grub 3.18.0-8-generic /boot/vmlinux-3.18.0-8-generic Generating grub configuration file ... Found linux image: /boot/vmlinux-4.4.0-040400rc4-generic Found initrd image: /boot/initrd.img-4.4.0-040400rc4-generic Found linux image: /boot/vmlinux-3.19.0-41-generic Found initrd image: /boot/initrd.img-3.19.0-41-generic Found linux image: /boot/vmlinux-3.19.0-39-generic Found initrd image: /boot/initrd.img-3.19.0-39-generic Found linux image: /boot/vmlinux-3.18.0-8-generic Found initrd image: /boot/initrd.img-3.18.0-8-generic Found linux image: /boot/vmlinux-3.13.0-10-generic Found initrd image: /boot/initrd.img-3.13.0-10-generic Found Debian GNU/Linux (8.2) on /dev/sdb2 Found unknown Linux distribution on /dev/mapper/ibmpkvm_vg_root01-ibmpkvm_lv_system done root@powerkvmpok002:~# uname -a Linux powerkvmpok002 3.18.0-8-generic #9-Ubuntu SMP Mon Jan 5 22:52:15 UTC 2015 ppc64le ppc64le ppc64le GNU/Linux Then executed the lock_torture tests again on the 3.18 Vivid kernel. root@powerkvmpok002:~# cd /opt/ltp/testcases/bin/ root@powerkvmpok002:/opt/ltp/testcases/bin# ./lock_torture.sh ./lock_torture.sh: 55: ./lock_torture.sh: tst_kvercmp: not found [ 148.10] NMI watchdog: BUG: soft lockup - CPU#100 stuck for 23s! [lock_torture_wr:2994] [ 148.13] NMI watchdog: BUG: soft lockup - CPU#101 stuck for 23s! [lock_torture_wr:2986] [ 148.15] NMI watchdog: BUG: soft lockup - CPU#99 stuck for 23s! [lock_torture_wr:2960] [ 148.177780] NMI watchdog: BUG: soft lockup - CPU#102 stuck for 23s! [lock_torture_wr:2999] [ 148.177783] NMI watchdog: BUG: soft lockup - CPU#103 stuck for 23s! [lock_torture_wr:2995] [ 148.181769] NMI watchdog: BUG: soft lockup - CPU#104 stuck for 23s! [lock_torture_wr:2987] [ 148.185771] NMI watchdog: BUG: soft lockup - CPU#107 stuck for 23s! [lock_torture_wr:2990] [ 148.185774] NMI watchdog: BUG: soft lockup - CPU#106 stuck for 23s! [lock_torture_wr:2992] [ 148.189769] NMI watchdog: BUG: soft lockup - CPU#108 stuck for 23s! [lock_torture_wr:2984] [ 148.189773] NMI watchdog: BUG: soft lockup - CPU#110 stuck for 23s! [lock_torture_wr:2983] [ 148.189775] NMI watchdog: BUG: soft lockup - CPU#109 stuck for 23s! [lock_torture_wr:2989] [ 148.189787] NMI watchdog: BUG: soft lockup - CPU#111 stuck for 23s! [lock_torture_wr:2988] [ 148.193770] NMI watchdog: BUG: soft lockup - CPU#113 stuck for 23s! [lock_torture_wr:3023] [ 148.193773] NMI watchdog: BUG: soft lockup - CPU#112 stuck for 23s! [lock_torture_wr:3000] [ 148.197768] NMI watchdog: BUG: soft lockup - CPU#115 stuck for 23s! [lock_torture_wr:3017] [ 204.521765] INFO: rcu_sched self-detected stall on CPU [ 204.521768] INFO: rcu_sched self-detected stall on CPU [ 204.521780] INFO: rcu_sched self-detected stall on CPU { 71} [ 204.521793] INFO: rcu_sched self-detected stall on CPU { 39} [ 204.521794] INFO: rcu_sched self-detected stall on CPU [ 204.521794] INFO: rcu_sched self-detected stall on CPU The above messages are displayed immediately on the console once the tests are started. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1483343 Title: NMI watchdog: BUG: soft lockup errors when we execute lock_torture_wr tests To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1483343/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1483343] Comment bridged from LTC Bugzilla
--- Comment From pavsu...@in.ibm.com 2016-01-08 08:03 EDT--- I have installed the proposed kernel which you have mentioned on the P8 Hardware. root@powerkvmpok002:~/5613571# dpkg -i linux-image-3.13.0-10-generic_3.13.0-10.30_ppc64el.deb Selecting previously unselected package linux-image-3.13.0-10-generic. (Reading database ... 155249 files and directories currently installed.) Preparing to unpack linux-image-3.13.0-10-generic_3.13.0-10.30_ppc64el.deb ... Done. Unpacking linux-image-3.13.0-10-generic (3.13.0-10.30) ... Setting up linux-image-3.13.0-10-generic (3.13.0-10.30) ... Running depmod. update-initramfs: deferring update (hook will be called later) Examining /etc/kernel/postinst.d. run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 3.13.0-10-generic /boot/vmlinux-3.13.0-10-generic run-parts: executing /etc/kernel/postinst.d/initramfs-tools 3.13.0-10-generic /boot/vmlinux-3.13.0-10-generic update-initramfs: Generating /boot/initrd.img-3.13.0-10-generic run-parts: executing /etc/kernel/postinst.d/update-notifier 3.13.0-10-generic /boot/vmlinux-3.13.0-10-generic run-parts: executing /etc/kernel/postinst.d/zz-update-grub 3.13.0-10-generic /boot/vmlinux-3.13.0-10-generic Generating grub configuration file ... Found linux image: /boot/vmlinux-4.4.0-040400rc4-generic Found initrd image: /boot/initrd.img-4.4.0-040400rc4-generic Found linux image: /boot/vmlinux-3.19.0-41-generic Found initrd image: /boot/initrd.img-3.19.0-41-generic Found linux image: /boot/vmlinux-3.19.0-39-generic Found initrd image: /boot/initrd.img-3.19.0-39-generic Found linux image: /boot/vmlinux-3.13.0-10-generic Found initrd image: /boot/initrd.img-3.13.0-10-generic Found Debian GNU/Linux (8.2) on /dev/sdb2 Found unknown Linux distribution on /dev/mapper/ibmpkvm_vg_root01-ibmpkvm_lv_system done root@powerkvmpok002:~/5613571# dpkg -i linux-image-extra-3.13.0-10-generic_3.13.0-10.30_ppc64el.deb Selecting previously unselected package linux-image-extra-3.13.0-10-generic. (Reading database ... 155936 files and directories currently installed.) Preparing to unpack linux-image-extra-3.13.0-10-generic_3.13.0-10.30_ppc64el.deb ... Unpacking linux-image-extra-3.13.0-10-generic (3.13.0-10.30) ... Setting up linux-image-extra-3.13.0-10-generic (3.13.0-10.30) ... Running depmod. update-initramfs: deferring update (hook will be called later) initrd.img(/boot/initrd.img-3.13.0-10-generic ) points to /boot/initrd.img-3.13.0-10-generic (/boot/initrd.img-3.13.0-10-generic) -- doing nothing at /var/lib/dpkg/info/linux-image-extra-3.13.0-10-generic.postinst line 491. vmlinux(/boot/vmlinux-3.13.0-10-generic ) points to /boot/vmlinux-3.13.0-10-generic (/boot/vmlinux-3.13.0-10-generic) -- doing nothing at /var/lib/dpkg/info/linux-image-extra-3.13.0-10-generic.postinst line 491. Examining /etc/kernel/postinst.d. run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 3.13.0-10-generic /boot/vmlinux-3.13.0-10-generic run-parts: executing /etc/kernel/postinst.d/initramfs-tools 3.13.0-10-generic /boot/vmlinux-3.13.0-10-generic update-initramfs: Generating /boot/initrd.img-3.13.0-10-generic run-parts: executing /etc/kernel/postinst.d/update-notifier 3.13.0-10-generic /boot/vmlinux-3.13.0-10-generic run-parts: executing /etc/kernel/postinst.d/zz-update-grub 3.13.0-10-generic /boot/vmlinux-3.13.0-10-generic Generating grub configuration file ... Found linux image: /boot/vmlinux-4.4.0-040400rc4-generic Found initrd image: /boot/initrd.img-4.4.0-040400rc4-generic Found linux image: /boot/vmlinux-3.19.0-41-generic Found initrd image: /boot/initrd.img-3.19.0-41-generic Found linux image: /boot/vmlinux-3.19.0-39-generic Found initrd image: /boot/initrd.img-3.19.0-39-generic Found linux image: /boot/vmlinux-3.13.0-10-generic Found initrd image: /boot/initrd.img-3.13.0-10-generic Found Debian GNU/Linux (8.2) on /dev/sdb2 Found unknown Linux distribution on /dev/mapper/ibmpkvm_vg_root01-ibmpkvm_lv_system done root@powerkvmpok002:~# uname -a Linux powerkvmpok002 3.13.0-10-generic #30-Ubuntu SMP Tue Feb 18 23:06:18 UTC 2014 ppc64le ppc64le ppc64le GNU/Linux root@powerkvmpok002:~# cat /etc/os-release NAME="Ubuntu" VERSION="14.04.3 LTS, Trusty Tahr" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 14.04.3 LTS" VERSION_ID="14.04" HOME_URL="http://www.ubuntu.com/"; SUPPORT_URL="http://help.ubuntu.com/"; BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"; root@powerkvmpok002:~# cat /proc/cpuinfo | tail processor : 159 cpu : POWER8E (raw), altivec supported clock : 3425.00MHz revision: 2.1 (pvr 004b 0201) timebase: 51200 platform: PowerNV model : 8284-22A machine : PowerNV 8284-22A firmware: OPAL v3 Then executed the same test cases again on the installed kernel. root@powerkvmpok002:/opt/ltp/testcases/bin# ./lock_torture.sh lock_torture 1 TCONF : test must be run with kernel 3.18 or newer lock_torture 1 TINFO : cleanup As can be seen, the kernel supp
[Bug 1483343] Comment bridged from LTC Bugzilla
--- Comment From cha...@us.ibm.com 2016-01-05 17:08 EDT--- Re-posting most recent comment so Canonical can see it: Similar behaviour is observed in ubuntu 14.04.4 testing. runltp test execution stopped after executing rcu_torture test Fllowing Call trace is observed in "dmesg" output [63437.060777] INFO: rcu_sched self-detected stall on CPU { 6} (t=11284398 jiffies g=511721 c=511720 q=147264) [63437.060796] Task dump for CPU 6: [63437.060798] lock_torture_wr R running task0 32581 2 0x0804 [63437.060803] Call Trace: [63437.060806] [c00306ec7560] [c00f55f0] sched_show_task+0xe0/0x180 (unreliable) [63437.060812] [c00306ec75d0] [c0133c58] rcu_dump_cpu_stacks+0xe8/0x160 [63437.060817] [c00306ec7620] [c01384a4] rcu_check_callbacks+0x634/0x960 [63437.060822] [c00306ec7750] [c013fec8] update_process_times+0x58/0xa0 [63437.060826] [c00306ec7780] [c0156438] tick_sched_handle.isra.15+0x48/0xe0 [63437.060831] [c00306ec77c0] [c0156534] tick_sched_timer+0x64/0xc0 [63437.060836] [c00306ec7800] [c0140954] __hrtimer_run_queues+0x124/0x350 [63437.060841] [c00306ec7890] [c014174c] hrtimer_interrupt+0xec/0x290 [63437.060846] [c00306ec7950] [c001f3c0] __timer_interrupt+0x90/0x230 [63437.060851] [c00306ec79a0] [c001f710] timer_interrupt+0xa0/0xe0 [63437.060855] [c00306ec79d0] [c0002714] decrementer_common+0x114/0x180 [63437.060861] --- interrupt: 901 at udelay+0x44/0x60 [63437.060861] LR = torture_rwlock_write_delay+0x5c/0x90 [locktorture] [63437.060868] [c00306ec7cc0] [d4010bc8] torture_rwlock_write_delay+0x28/0x90 [locktorture] (unreliable) [63437.060874] [c00306ec7cf0] [d40105c0] lock_torture_writer+0xe0/0x210 [locktorture] [63437.060880] [c00306ec7d80] [c00e0a74] kthread+0x114/0x140 [63437.060884] [c00306ec7e30] [c0009538] ret_from_kernel_thread+0x5c/0xa4 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1483343 Title: NMI watchdog: BUG: soft lockup errors when we execute lock_torture_wr tests To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1483343/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1483343] Comment bridged from LTC Bugzilla
--- Comment From pavsu...@in.ibm.com 2015-12-11 08:36 EDT--- I have downloaded the test kernel as suggested and executed the scenario again. root@powerkvmpok002:~/lp1483343# dpkg -i linux-image-3.19.0-41-generic_3.19.0-41.46~14.04.2_ppc64el.deb Selecting previously unselected package linux-image-3.19.0-41-generic. (Reading database ... 142221 files and directories currently installed.) Preparing to unpack linux-image-3.19.0-41-generic_3.19.0-41.46~14.04.2_ppc64el.deb ... Done. Unpacking linux-image-3.19.0-41-generic (3.19.0-41.46~14.04.2) ... Setting up linux-image-3.19.0-41-generic (3.19.0-41.46~14.04.2) ... Running depmod. update-initramfs: deferring update (hook will be called later) Examining /etc/kernel/postinst.d. run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 3.19.0-41-generic /boot/vmlinux-3.19.0-41-generic run-parts: executing /etc/kernel/postinst.d/initramfs-tools 3.19.0-41-generic /boot/vmlinux-3.19.0-41-generic update-initramfs: Generating /boot/initrd.img-3.19.0-41-generic run-parts: executing /etc/kernel/postinst.d/update-notifier 3.19.0-41-generic /boot/vmlinux-3.19.0-41-generic run-parts: executing /etc/kernel/postinst.d/zz-update-grub 3.19.0-41-generic /boot/vmlinux-3.19.0-41-generic Generating grub configuration file ... Found linux image: /boot/vmlinux-4.4.0-040400rc4-generic Found initrd image: /boot/initrd.img-4.4.0-040400rc4-generic Found linux image: /boot/vmlinux-3.19.0-41-generic Found initrd image: /boot/initrd.img-3.19.0-41-generic Found linux image: /boot/vmlinux-3.19.0-39-generic Found initrd image: /boot/initrd.img-3.19.0-39-generic Found Debian GNU/Linux (8.2) on /dev/sdb2 Found unknown Linux distribution on /dev/mapper/ibmpkvm_vg_root01-ibmpkvm_lv_system done root@powerkvmpok002:~/lp1483343# dpkg -i linux-image-extra-3.19.0-41-generic_3.19.0-41.46~14.04.2_ppc64el.deb Selecting previously unselected package linux-image-extra-3.19.0-41-generic. (Reading database ... 143108 files and directories currently installed.) Preparing to unpack linux-image-extra-3.19.0-41-generic_3.19.0-41.46~14.04.2_ppc64el.deb ... Unpacking linux-image-extra-3.19.0-41-generic (3.19.0-41.46~14.04.2) ... Setting up linux-image-extra-3.19.0-41-generic (3.19.0-41.46~14.04.2) ... run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 3.19.0-41-generic /boot/vmlinux-3.19.0-41-generic run-parts: executing /etc/kernel/postinst.d/initramfs-tools 3.19.0-41-generic /boot/vmlinux-3.19.0-41-generic update-initramfs: Generating /boot/initrd.img-3.19.0-41-generic run-parts: executing /etc/kernel/postinst.d/update-notifier 3.19.0-41-generic /boot/vmlinux-3.19.0-41-generic run-parts: executing /etc/kernel/postinst.d/zz-update-grub 3.19.0-41-generic /boot/vmlinux-3.19.0-41-generic Generating grub configuration file ... Found linux image: /boot/vmlinux-4.4.0-040400rc4-generic Found initrd image: /boot/initrd.img-4.4.0-040400rc4-generic Found linux image: /boot/vmlinux-3.19.0-41-generic Found initrd image: /boot/initrd.img-3.19.0-41-generic Found linux image: /boot/vmlinux-3.19.0-39-generic Found initrd image: /boot/initrd.img-3.19.0-39-generic Found Debian GNU/Linux (8.2) on /dev/sdb2 Found unknown Linux distribution on /dev/mapper/ibmpkvm_vg_root01-ibmpkvm_lv_system done root@powerkvmpok002:~# uname -a Linux powerkvmpok002 3.19.0-41-generic #46~14.04.2 SMP Thu Dec 10 15:55:32 UTC 2015 ppc64le ppc64le ppc64le GNU/Linux root@powerkvmpok002:~# cd /opt/ltp/testcases/bin/ root@powerkvmpok002:/opt/ltp/testcases/bin# ./lock_torture.sh ./lock_torture.sh: 55: ./lock_torture.sh: tst_kvercmp: not found lock_torture 1 TINFO : estimate time 6.00 min lock_torture 1 TINFO : spin_lock: running 60 sec... lock_torture 1 TPASS : spin_lock: completed lock_torture 2 TINFO : spin_lock_irq: running 60 sec... root@powerkvmpok002:~# [ 6816.082218] NMI watchdog: BUG: soft lockup - CPU#6 stuck for 22s! [lock_torture_wr:3486] [ 6828.106219] NMI watchdog: BUG: soft lockup - CPU#57 stuck for 21s! [lock_torture_wr:3403] [ 6828.214216] NMI watchdog: BUG: soft lockup - CPU#141 stuck for 21s! [lock_torture_wr:3606] [ 6844.15] NMI watchdog: BUG: soft lockup - CPU#144 stuck for 23s! [lock_torture_wr:3507] [ 6844.234220] NMI watchdog: BUG: soft lockup - CPU#149 stuck for 21s! [lock_torture_wr:3475] [ 6856.082221] NMI watchdog: BUG: soft lockup - CPU#8 stuck for 23s! [lock_torture_wr:3345] [ 6856.094227] NMI watchdog: BUG: soft lockup - CPU#35 stuck for 21s! [lock_torture_wr:3329] [ 6856.142223] NMI watchdog: BUG: soft lockup - CPU#95 stuck for 21s! [lock_torture_wr:3283] [ 6856.162221] NMI watchdog: BUG: soft lockup - CPU#109 stuck for 23s! [lock_torture_wr:3265] [ 6856.210221] NMI watchdog: BUG: soft lockup - CPU#137 stuck for 21s! [lock_torture_wr:3458] [ 6860.118223] NMI watchdog: BUG: soft lockup - CPU#68 stuck for 22s! [lock_torture_wr:3409] [ 6860.182227] NMI watchdog: BUG: soft lockup - CPU#121 stuck for 22s! [lock_torture_wr:3341] [ 6862.110245] INFO: rcu_sched self-detected stall on
[Bug 1483343] Comment bridged from LTC Bugzilla
--- Comment From pavsu...@in.ibm.com 2015-12-11 06:16 EDT--- I have downloaded the mainline wily kernel as suggested and executed the scenario again. root@powerkvmpok002:~/mainline# wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.4-rc4-wily/linux-image-4.4.0-040400rc4-generic_4.4.0-040400rc4.201512061930_ppc64el.deb --2015-12-11 00:28:48-- http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.4-rc4-wily/linux-image-4.4.0-040400rc4-generic_4.4.0-040400rc4.201512061930_ppc64el.deb Resolving kernel.ubuntu.com (kernel.ubuntu.com)... 91.189.94.216 Connecting to kernel.ubuntu.com (kernel.ubuntu.com)|91.189.94.216|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 56391678 (54M) [application/x-debian-package] Saving to: ?linux-image-4.4.0-040400rc4-generic_4.4.0-040400rc4.201512061930_ppc64el.deb? 100%[===>] 56,391,678 846KB/s in 66s 2015-12-11 00:29:54 (832 KB/s) - ?linux- image-4.4.0-040400rc4-generic_4.4.0-040400rc4.201512061930_ppc64el.deb? saved [56391678/56391678] root@powerkvmpok002:~/mainline# dpkg -i linux-image-4.4.0-040400rc4-generic_4.4.0-040400rc4.201512061930_ppc64el.deb Selecting previously unselected package linux-image-4.4.0-040400rc4-generic. (Reading database ... 137006 files and directories currently installed.) Preparing to unpack linux-image-4.4.0-040400rc4-generic_4.4.0-040400rc4.201512061930_ppc64el.deb ... Done. Unpacking linux-image-4.4.0-040400rc4-generic (4.4.0-040400rc4.201512061930) ... Setting up linux-image-4.4.0-040400rc4-generic (4.4.0-040400rc4.201512061930) ... Running depmod. update-initramfs: deferring update (hook will be called later) Examining /etc/kernel/postinst.d. run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.4.0-040400rc4-generic /boot/vmlinux-4.4.0-040400rc4-generic run-parts: executing /etc/kernel/postinst.d/initramfs-tools 4.4.0-040400rc4-generic /boot/vmlinux-4.4.0-040400rc4-generic update-initramfs: Generating /boot/initrd.img-4.4.0-040400rc4-generic W: Possible missing firmware /lib/firmware/ql2700_fw.bin for module qla2xxx W: Possible missing firmware /lib/firmware/ql8300_fw.bin for module qla2xxx W: Possible missing firmware /lib/firmware/ql2600_fw.bin for module qla2xxx run-parts: executing /etc/kernel/postinst.d/update-notifier 4.4.0-040400rc4-generic /boot/vmlinux-4.4.0-040400rc4-generic run-parts: executing /etc/kernel/postinst.d/zz-update-grub 4.4.0-040400rc4-generic /boot/vmlinux-4.4.0-040400rc4-generic Generating grub configuration file ... Found linux image: /boot/vmlinux-4.4.0-040400rc4-generic Found initrd image: /boot/initrd.img-4.4.0-040400rc4-generic Found linux image: /boot/vmlinux-3.19.0-39-generic Found initrd image: /boot/initrd.img-3.19.0-39-generic Found Debian GNU/Linux (8.2) on /dev/sdb2 Found unknown Linux distribution on /dev/mapper/ibmpkvm_vg_root01-ibmpkvm_lv_system done root@powerkvmpok002:~# uname -a Linux powerkvmpok002 4.4.0-040400rc4-generic #201512061930 SMP Mon Dec 7 01:14:51 UTC 2015 ppc64le ppc64le ppc64le GNU/Linux root@powerkvmpok002:~# cd /opt/ltp/testcases/bin/ root@powerkvmpok002:/opt/ltp/testcases/bin# ./lock_torture.sh ./lock_torture.sh: 55: ./lock_torture.sh: tst_kvercmp: not found lock_torture 1 TINFO : estimate time 6.00 min lock_torture 1 TINFO : spin_lock: running 60 sec... lock_torture 1 TPASS : spin_lock: completed lock_torture 2 TINFO : spin_lock_irq: running 60 sec... lock_torture 2 TPASS : spin_lock_irq: completed lock_torture 3 TINFO : rw_lock: running 60 sec... lock_torture 3 TPASS : rw_lock: completed lock_torture 4 TINFO : rw_lock_irq: running 60 sec... root@powerkvmpok002:~# [ 614.289969] NMI watchdog: BUG: soft lockup - CPU#150 stuck for 140s! [lock_torture_wr:4325] [ 619.293972] NMI watchdog: BUG: soft lockup - CPU#23 stuck for 244s! [lock_torture_wr:4399] [ 619.293987] NMI watchdog: BUG: soft lockup - CPU#98 stuck for 51s! [lock_torture_wr:4387] [ 627.301973] NMI watchdog: BUG: soft lockup - CPU#39 stuck for 242s! [lock_torture_wr:4403] [ 669.345931] INFO: rcu_sched detected stalls on CPUs/tasks: [ 669.346071] 1-...: (2 GPs behind) idle=02f/140/0 softirq=10477/10480 fqs=2908 [ 669.346162] 8-...: (0 ticks this GP) idle=58f/140/0 softirq=9878/9878 fqs=2908 [ 669.346290] 11-...: (0 ticks this GP) idle=e0f/140/0 softirq=10060/10060 fqs=2908 [ 669.346383] 12-...: (0 ticks this GP) idle=3dd/140/0 softirq=10024/10024 fqs=2908 [ 669.346470] 14-...: (1 GPs behind) idle=141/140/0 softirq=10118/10118 fqs=2908 [ 669.346560] 19-...: (1 GPs behind) idle=83d/140/0 softirq=10273/10273 fqs=2908 [ 669.346660] (detected by 104, t=84094 jiffies, g=3980, c=3979, q=7125) [ 672.213960] NMI watchdog: BUG: soft lockup - CPU#116 stuck for 53s! [lock_torture_wr:4425] [ 672.213980] NMI watchdog: BUG: soft lockup - CPU#32 stuck for 50s! [lock_tor
[Bug 1483343] Comment bridged from LTC Bugzilla
--- Comment From pavsu...@in.ibm.com 2015-12-10 08:00 EDT--- I have installed Ubuntu 14.04.3 ISO on PowerNV 8284-22A P8 Hardware. Then executed the LTP lock_torture tests again on the same. root@powerkvmpok002:~# uname -a Linux powerkvmpok002 3.19.0-39-generic #44~14.04.1-Ubuntu SMP Wed Dec 2 10:00:52 UTC 2015 ppc64le ppc64le ppc64le GNU/Linux root@powerkvmpok002:~# cat /etc/os-release NAME="Ubuntu" VERSION="14.04.3 LTS, Trusty Tahr" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 14.04.3 LTS" VERSION_ID="14.04" HOME_URL="http://www.ubuntu.com/"; SUPPORT_URL="http://help.ubuntu.com/"; BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"; root@powerkvmpok002:/opt/ltp/testcases/bin# ./lock_torture.sh ./lock_torture.sh: 55: ./lock_torture.sh: tst_kvercmp: not found [ 3173.326014] INFO: rcu_sched self-detected stall on CPU [ 3173.326017] INFO: rcu_sched self-detected stall on CPU [ 3173.326019] INFO: rcu_sched self-detected stall on CPU [ 3173.326022] INFO: rcu_sched self-detected stall on CPU [ 3173.326023] INFO: rcu_sched self-detected stall on CPU [ 3173.326044] INFO: rcu_sched self-detected stall on CPU { [ 3173.326045] INFO: rcu_sched self-detected stall on CPU [ 3173.326054] INFO: rcu_sched self-detected stall on CPU [ 3180.118019] NMI watchdog: BUG: soft lockup - CPU#69 stuck for 21s! [lock_torture_wr:25460] [ 3184.102014] NMI watchdog: BUG: soft lockup - CPU#47 stuck for 24s! [lock_torture_wr:25357] [ 3184.122017] NMI watchdog: BUG: soft lockup - CPU#74 stuck for 22s! [lock_torture_wr:25466] [ 3184.246010] NMI watchdog: BUG: soft lockup - CPU#155 stuck for 22s! [lock_torture_wr:25527] [ 3204.090023] NMI watchdog: BUG: soft lockup - CPU#23 stuck for 22s! [lock_torture_wr:25484] [ 3208.106009] NMI watchdog: BUG: soft lockup - CPU#53 stuck for 21s! [lock_torture_wr:25482] [ 3212.110015] NMI watchdog: BUG: soft lockup - CPU#60 stuck for 22s! [lock_torture_wr:25422] [ 3212.142006] NMI watchdog: BUG: soft lockup - CPU#90 stuck for 21s! [lock_torture_wr:25395] [ 3212.222010] NMI watchdog: BUG: soft lockup - CPU#144 stuck for 23s! [lock_torture_wr:25415] [ 3216.090008] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 24s! [lock_torture_wr:25495] [ 3216.118032] NMI watchdog: BUG: soft lockup - CPU#67 stuck for 21s! [lock_torture_wr:25353] [ 3216.198006] NMI watchdog: BUG: soft lockup - CPU#130 stuck for 21s! [lock_torture_wr:25380] [ 3224.094005] NMI watchdog: BUG: soft lockup - CPU#38 stuck for 22s! [lock_torture_wr:25330] [ 3228.101998] NMI watchdog: BUG: soft lockup - CPU#49 stuck for 23s! [lock_torture_wr:25455] [ 3228.162005] NMI watchdog: BUG: soft lockup - CPU#110 stuck for 22s! [lock_torture_wr:25490] The issue is still occurring with the latest Ubuntu 14.04.3 OS kernel. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1483343 Title: NMI watchdog: BUG: soft lockup errors when we execute lock_torture_wr tests To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1483343/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1483343] Comment bridged from LTC Bugzilla
--- Comment From cha...@us.ibm.com 2015-08-11 16:52 EDT--- (In reply to comment #18) > Created attachment 100943 [details] > dmesg logs are attached when tests are done with new kernel provided So does the system lockup as before? Also, the dmesg attached doesn't show any softlockups. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1483343 Title: NMI watchdog: BUG: soft lockup errors when we execute lock_torture_wr tests To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1483343/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1483343] Comment bridged from LTC Bugzilla
--- Comment From pavsu...@in.ibm.com 2015-08-11 10:13 EDT--- I have installed this kernel on a Power NV Hardware which is installed with Ubuntu 14.04.3. Then executed the tests again on the same OS. root@p7pite26os:~# uname -a Linux p7pite26os 3.19.0-26-generic #27 SMP Mon Aug 10 20:05:24 UTC 2015 ppc64le ppc64le ppc64le GNU/Linux root@p7pite26os:~# cat /etc/issue Ubuntu 14.04.3 LTS \n \l root@p7pite26os:~# cat /etc/os-release NAME="Ubuntu" VERSION="14.04.3 LTS, Trusty Tahr" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 14.04.3 LTS" VERSION_ID="14.04" HOME_URL="http://www.ubuntu.com/"; SUPPORT_URL="http://help.ubuntu.com/"; BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"; root@p7pite26os:~# dmesg | grep NMI [ 4824.652595] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [lock_torture_wr:29508] [ 4824.652603] NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [lock_torture_wr:29402] [ 4824.656608] NMI watchdog: BUG: soft lockup - CPU#11 stuck for 23s! [lock_torture_wr:29374] [ 4824.656610] NMI watchdog: BUG: soft lockup - CPU#8 stuck for 23s! [lock_torture_wr:29468] [ 4824.656616] NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [lock_torture_wr:29450] [ 4824.656619] NMI watchdog: BUG: soft lockup - CPU#7 stuck for 22s! [lock_torture_re:29411] [ 4824.656633] NMI watchdog: BUG: soft lockup - CPU#9 stuck for 23s! [lock_torture_re:29387] [ 4824.660592] NMI watchdog: BUG: soft lockup - CPU#12 stuck for 22s! [lock_torture_re:29403] [ 4824.660603] NMI watchdog: BUG: soft lockup - CPU#15 stuck for 23s! [lock_torture_re:29407] [ 4824.660605] NMI watchdog: BUG: soft lockup - CPU#16 stuck for 23s! [lock_torture_wr:29376] [ 4824.660610] NMI watchdog: BUG: soft lockup - CPU#13 stuck for 23s! [lock_torture_re:29423] [ 4824.664587] NMI watchdog: BUG: soft lockup - CPU#19 stuck for 22s! [lock_torture_wr:29476] [ 4824.664589] NMI watchdog: BUG: soft lockup - CPU#22 stuck for 23s! [lock_torture_wr:29466] [ 4824.664591] NMI watchdog: BUG: soft lockup - CPU#18 stuck for 23s! [lock_torture_wr:29494] [ 4824.664622] NMI watchdog: BUG: soft lockup - CPU#23 stuck for 23s! [lock_torture_wr:29386] [ 4824.668596] NMI watchdog: BUG: soft lockup - CPU#27 stuck for 23s! [lock_torture_re:29457] [ 4824.668599] NMI watchdog: BUG: soft lockup - CPU#25 stuck for 22s! [lock_torture_re:29377] [ 4824.668616] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [lock_torture_re:29409] [ 4824.668623] NMI watchdog: BUG: soft lockup - CPU#26 stuck for 23s! [lock_torture_wr:29356] [ 4824.672600] NMI watchdog: BUG: soft lockup - CPU#33 stuck for 23s! [lock_torture_wr:29414] [ 4824.672631] NMI watchdog: BUG: soft lockup - CPU#32 stuck for 23s! [lock_torture_wr:29396] [ 4824.672650] NMI watchdog: BUG: soft lockup - CPU#31 stuck for 23s! [lock_torture_re:29441] [ 4824.672657] NMI watchdog: BUG: soft lockup - CPU#29 stuck for 23s! [lock_torture_re:29455] [ 4824.676598] NMI watchdog: BUG: soft lockup - CPU#37 stuck for 23s! [lock_torture_wr:29368] [ 4824.67] NMI watchdog: BUG: soft lockup - CPU#34 stuck for 23s! [lock_torture_wr:29442] [ 4824.684608] NMI watchdog: BUG: soft lockup - CPU#45 stuck for 22s! [lock_torture_re:29379] [ 4824.684626] NMI watchdog: BUG: soft lockup - CPU#43 stuck for 23s! [lock_torture_wr:29406] [ 4824.688611] NMI watchdog: BUG: soft lockup - CPU#47 stuck for 23s! [lock_torture_wr:29454] [ 4824.688613] NMI watchdog: BUG: soft lockup - CPU#46 stuck for 23s! [lock_torture_wr:29502] [ 4824.688647] NMI watchdog: BUG: soft lockup - CPU#48 stuck for 23s! [lock_torture_wr:29420] [ 4824.696650] NMI watchdog: BUG: soft lockup - CPU#55 stuck for 22s! [lock_torture_wr:29412] [ 4824.696777] NMI watchdog: BUG: soft lockup - CPU#52 stuck for 23s! [lock_torture_wr:29380] [ 4824.700602] NMI watchdog: BUG: soft lockup - CPU#57 stuck for 22s! [lock_torture_re:29367] [ 4824.700609] NMI watchdog: BUG: soft lockup - CPU#56 stuck for 22s! [lock_torture_wr:29418] [ 4824.704597] NMI watchdog: BUG: soft lockup - CPU#59 stuck for 22s! [lock_torture_re:29401] [ 4824.704659] NMI watchdog: BUG: soft lockup - CPU#58 stuck for 22s! [lock_torture_wr:29410] [ 4824.712597] NMI watchdog: BUG: soft lockup - CPU#64 stuck for 22s! [lock_torture_re:29429] [ 4824.716864] NMI watchdog: BUG: soft lockup - CPU#66 stuck for 22s! [lock_torture_wr:29372] [ 4824.720595] NMI watchdog: BUG: soft lockup - CPU#71 stuck for 22s! [lock_torture_wr:29382] [ 4824.720672] NMI watchdog: BUG: soft lockup - CPU#70 stuck for 23s! [lock_torture_wr:29514] [ 4824.720702] NMI watchdog: BUG: soft lockup - CPU#69 stuck for 23s! [lock_torture_wr:29474] [ 4824.728618] NMI watchdog: BUG: soft lockup - CPU#75 stuck for 22s! [lock_torture_re:29505] [ 4824.732596] NMI watchdog: BUG: soft lockup - CPU#76 stuck for 23s! [lock_torture_re:29453] [ 4824.732622] NMI watchdog: BUG: soft lockup - CPU#77 stuck for 22s! [lock_torture_re:29485] [ 4824.736597] NMI watchdog: BUG: soft lockup - CPU#79 stuck for 22s! [lock_torture_re:29499