[Bug 1777857] Re: [LTCTest][OPAL][OP920] INFO: rcu_sched self-detected stall on CPU
*** This bug is a duplicate of bug 1773964 *** https://bugs.launchpad.net/bugs/1773964 ** Tags added: cscc -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1777857 Title: [LTCTest][OPAL][OP920] INFO: rcu_sched self-detected stall on CPU To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-power-systems/+bug/1777857/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1777857] Re: [LTCTest][OPAL][OP920] INFO: rcu_sched self-detected stall on CPU
*** This bug is a duplicate of bug 1773964 *** https://bugs.launchpad.net/bugs/1773964 This bug has been marked as a duplicate of LP#1773964 in Launchpad. Please see that bug for updates. Thanks. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1777857 Title: [LTCTest][OPAL][OP920] INFO: rcu_sched self-detected stall on CPU To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-power-systems/+bug/1777857/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1777857] Re: [LTCTest][OPAL][OP920] INFO: rcu_sched self-detected stall on CPU
*** This bug is a duplicate of bug 1773964 *** https://bugs.launchpad.net/bugs/1773964 ** This bug has been marked a duplicate of bug 1773964 IPL: ppc64_cpu --frequency hang with INFO: rcu_sched detected stalls on CPUs/tasks on w34 and wsbmc016 with 920.1714.20170330n -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1777857 Title: [LTCTest][OPAL][OP920] INFO: rcu_sched self-detected stall on CPU To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-power-systems/+bug/1777857/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1777857] Re: [LTCTest][OPAL][OP920] INFO: rcu_sched self-detected stall on CPU
SRU Request Submitted:
https://lists.ubuntu.com/archives/kernel-team/2018-June/093483.html
** Description changed:
+ == SRU Justification ==
+ IBM is seeing kernel traces during testing. This is due to a missing
+ backport of some kernel fixes in the RTC driver, which is commit
+ 682e6b4da5cb. Commit 682e6b4da5cb was also cc'd to upstream stable, but
+ it has not landed in Bionic as of yet. It is also a fix to upstream
+ commit 628daa8d5abf.
+
+ Commit 34dd25de9fe3 is also needed as a prereq to define
+ OPAL_BUSY_DELAY_MS.
+
+ == Fixes ==
+ 34dd25de9fe3 ("powerpc/powernv: define a standard delay for OPAL_BUSY type
retry loops")
+ 682e6b4da5cb ("rtc: opal: Fix OPAL RTC driver OPAL_BUSY loops")
+
+ == Regression Potential ==
+ Low. Limited to powerpc. Fixes a current regression.
+
+ == Test Case ==
+ A test kernel was built with these patches and tested by the original bug
reporter.
+ The bug reporter states the test kernel resolved the bug.
+
+
== Comment: #0 - PAVAMAN SUBRAMANIYAM - 2018-05-23
01:15:30 ==
Install a P9 Open Power Hardware with the latest OP920 Firmware images
provided in the following link:
http://pfd.austin.ibm.com/releasenotes/openpower9/OP920/OP920_1808A/OP920_1808N_RelNote_Main.html
root@witherspoon:~# cat /etc/os-release
ID="openbmc-phosphor"
NAME="Phosphor OpenBMC (Phosphor OpenBMC Project Reference Distro)"
VERSION="ibm-v2.1"
VERSION_ID="ibm-v2.1-438-g0030304-r12-0-g5ee4fb0"
PRETTY_NAME="Phosphor OpenBMC (Phosphor OpenBMC Project Reference Distro)
ibm-v2.1"
BUILD_ID="ibm-v2.1-438-g0030304-r12"
root@witherspoon:~# cat /var/lib/phosphor-software-manager/pnor/ro/VERSION
IBM-witherspoon-ibm-OP9-v2.0-2.14
- op-build-v2.0-11-gb248194-dirty
- buildroot-2018.02.1-6-ga8d1126
- skiboot-v6.0.1
- hostboot-8ab6717d-pfc036fa
- occ-77bb5e6
- linux-4.16.8-openpower2-pb532d68
- petitboot-v1.7.1-p1188545
- machine-xml-7cd20a6
- hostboot-binaries-276bb70
- capp-ucode-p9-dd2-v4
- sbe-a596975
- hcode-b8173e8
+ op-build-v2.0-11-gb248194-dirty
+ buildroot-2018.02.1-6-ga8d1126
+ skiboot-v6.0.1
+ hostboot-8ab6717d-pfc036fa
+ occ-77bb5e6
+ linux-4.16.8-openpower2-pb532d68
+ petitboot-v1.7.1-p1188545
+ machine-xml-7cd20a6
+ hostboot-binaries-276bb70
+ capp-ucode-p9-dd2-v4
+ sbe-a596975
+ hcode-b8173e8
Seeing the following messages in the dmesg logs.
[ 16.377405] ipmi_si: Unable to find any System Interface(s)
[ 17.384118] nf_conntrack version 0.5.0 (65536 buckets, 262144 max)
[ 1372.711730] INFO: rcu_sched self-detected stall on CPU
[ 1372.711787] 32-: (5249 ticks this GP) idle=182/141/0
softirq=1093/1093 fqs=2623
[ 1372.711863] (t=5250 jiffies g=22430 c=22429 q=953)
[ 1372.711921] Task dump for CPU 32:
[ 1372.711922] kworker/32:1R running task0 1123 2
0x0804
[ 1372.711930] Workqueue: events rtc_timer_do_work
[ 1372.711931] Call Trace:
[ 1372.711934] [c03fd2b97350] [c014a8f8]
sched_show_task.part.16+0xd8/0x110 (unreliable)
[ 1372.711939] [c03fd2b973c0] [c01aa8bc]
rcu_dump_cpu_stacks+0xd4/0x138
[ 1372.711942] [c03fd2b97410] [c01a9988]
rcu_check_callbacks+0x8e8/0xb40
[ 1372.711945] [c03fd2b97540] [c01b7c28]
update_process_times+0x48/0x90
[ 1372.711948] [c03fd2b97570] [c01cf974]
tick_sched_handle.isra.5+0x34/0xd0
[ 1372.711950] [c03fd2b975a0] [c01cfa70]
tick_sched_timer+0x60/0xe0
[ 1372.711953] [c03fd2b975e0] [c01b87d4]
__hrtimer_run_queues+0x144/0x370
[ 1372.711956] [c03fd2b97660] [c01b972c]
hrtimer_interrupt+0xfc/0x350
[ 1372.711959] [c03fd2b97730] [c00249f0]
__timer_interrupt+0x90/0x260
[ 1372.711962] [c03fd2b97780] [c0024e08] timer_interrupt+0x98/0xe0
[ 1372.711965] [c03fd2b977b0] [c0009054]
decrementer_common+0x114/0x120
[ 1372.711970] --- interrupt: 901 at opal_get_rtc_time+0x98/0x110
-LR = opal_return+0x14/0x48
+ LR = opal_return+0x14/0x48
[ 1372.711972] [c03fd2b97aa0] [c0a457b8]
opal_get_rtc_time+0x98/0x110 (unreliable)
[ 1372.711975] [c03fd2b97ae0] [c0a3f98c]
__rtc_read_time+0x7c/0x180
[ 1372.711977] [c03fd2b97b60] [c0a41738]
rtc_timer_do_work+0x78/0x250
[ 1372.711980] [c03fd2b97c90] [c0134378]
process_one_work+0x298/0x5a0
[ 1372.711982] [c03fd2b97d20] [c0134718] worker_thread+0x98/0x630
[ 1372.711985] [c03fd2b97dc0] [c013d348] kthread+0x1a8/0x1b0
[ 1372.711988] [c03fd2b97e30] [c000b658]
ret_from_kernel_thread+0x5c/0x84
== Comment: #1 - PAVAMAN SUBRAMANIYAM - 2018-05-23
01:31:06 ==
-
== Comment: #2 - Application Cdeadmin - 2018-05-23
01:33:40 ==
cde00 (cdead...@us.ibm.com) adde
[Bug 1777857] Re: [LTCTest][OPAL][OP920] INFO: rcu_sched self-detected stall on CPU
** Changed in: ubuntu-power-systems Status: Triaged => In Progress -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1777857 Title: [LTCTest][OPAL][OP920] INFO: rcu_sched self-detected stall on CPU To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-power-systems/+bug/1777857/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1777857] Re: [LTCTest][OPAL][OP920] INFO: rcu_sched self-detected stall on CPU
I built a test kernel with commit 682e6b4da5cbe8e9a53f979a58c2a9d7dc997175. Commit 34dd25de9fe3f was also required as a prereq. The test kernel can be downloaded from: http://kernel.ubuntu.com/~jsalisbury/lp1777857 Can you test this kernel and see if it resolves this bug? Note about installing test kernels: • If the test kernel is prior to 4.15(Bionic) you need to install the linux-image and linux-image-extra .deb packages. • If the test kernel is 4.15(Bionic) or newer, you need to install the linux-modules, linux-modules-extra and linux-image-unsigned .deb packages. Thanks in advance! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1777857 Title: [LTCTest][OPAL][OP920] INFO: rcu_sched self-detected stall on CPU To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-power-systems/+bug/1777857/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1777857] Re: [LTCTest][OPAL][OP920] INFO: rcu_sched self-detected stall on CPU
** Changed in: linux (Ubuntu) Status: New => Triaged ** Changed in: linux (Ubuntu) Importance: Undecided => High ** Also affects: linux (Ubuntu Bionic) Importance: Undecided Status: New ** Changed in: linux (Ubuntu Bionic) Status: New => Triaged ** Changed in: linux (Ubuntu Bionic) Importance: Undecided => High ** Changed in: linux (Ubuntu Bionic) Assignee: (unassigned) => Joseph Salisbury (jsalisbury) ** Changed in: linux (Ubuntu) Assignee: Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) => Joseph Salisbury (jsalisbury) ** Changed in: linux (Ubuntu) Status: Triaged => In Progress ** Changed in: linux (Ubuntu Bionic) Status: Triaged => In Progress -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1777857 Title: [LTCTest][OPAL][OP920] INFO: rcu_sched self-detected stall on CPU To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-power-systems/+bug/1777857/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1777857] Re: [LTCTest][OPAL][OP920] INFO: rcu_sched self-detected stall on CPU
** Changed in: ubuntu-power-systems Status: New => Triaged -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1777857 Title: [LTCTest][OPAL][OP920] INFO: rcu_sched self-detected stall on CPU To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-power-systems/+bug/1777857/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1777857] Re: [LTCTest][OPAL][OP920] INFO: rcu_sched self-detected stall on CPU
** Also affects: ubuntu-power-systems Importance: Undecided Status: New ** Changed in: ubuntu-power-systems Assignee: (unassigned) => Canonical Kernel Team (canonical-kernel-team) ** Package changed: kernel-package (Ubuntu) => linux (Ubuntu) ** Changed in: ubuntu-power-systems Importance: Undecided => High ** Tags added: p9 triage-g -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1777857 Title: [LTCTest][OPAL][OP920] INFO: rcu_sched self-detected stall on CPU To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-power-systems/+bug/1777857/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
