[Kernel-packages] [Bug 1777194] syslog-29.txt
--- Comment (attachment only) From hasri...@in.ibm.com 2018-07-20 02:37
EDT---
** Attachment added: "syslog-29.txt"
https://bugs.launchpad.net/bugs/1777194/+attachment/5165491/+files/syslog.txt
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1777194
Title:
Hard LOCKUP observed on stressing Ubuntu 18 04
Status in The Ubuntu-power-systems project:
In Progress
Status in linux package in Ubuntu:
In Progress
Status in linux source package in Bionic:
Fix Committed
Bug description:
== SRU Justification ==
IBM has seen hard lockups when running stress tests against Bionic.
These test sometimes lead to rcu_stalls. IBM found that this bug is
resolved with commits 6bed3237624e and ac61c1156623.
== Fixes ==
6bed3237624e ("powerpc: use NMI IPI for smp_send_stop")
ac61c1156623 ("powerpc: Fix smp_send_stop NMI IPI handling")
== Regression Potential ==
Low. Limited to powerpc.
== Test Case ==
A test kernel was built with these patches and tested by the original bug
reporter.
The bug reporter states the test kernel resolved the bug.
--Problem Description--
Hard LOCKUP on stressing Ubuntu 18 04
---Issue observed---
Hard LOCKUP on stressing Ubuntu 18 04 using Ubuntu 18 04, sometimes leads to
rcu_stalls.
Apr 17 00:00:23 lep8d kernel: [ 4309.786755] Watchdog CPU:3 Hard LOCKUP
Apr 17 00:00:23 lep8d kernel: [ 4309.786759] Modules linked in: algif_rng
salsa20_generic userio camellia_generic cast6_generic cast_common snd_seq
snd_seq_device snd_timer snd soundcore vhost_net serpent_generic tap
twofish_generic twofish_common vhost_vsock vmw_vsock_virtio_transport_common
vhost vsock lrw unix_diag algif_skcipher cuse sctp tgr192 wp512 rmd320 rmd256
rmd160 hci_vhci rmd128 bluetooth ecdh_generic dccp_ipv4 md4 uhid hid algif_hash
dccp af_alg xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4
iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack
nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 xt_tcpudp bridge stp llc
ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter
kvm_hv kvm binfmt_misc uio_pdrv_genirq uio vmx_crypto ibmpowernv
powernv_op_panel ipmi_powernv
Apr 17 00:00:23 lep8d kernel: [ 4309.786899] ipmi_devintf ipmi_msghandler
powernv_rng leds_powernv crct10dif_vpmsum sch_fq_codel ip_tables x_tables
autofs4 ses enclosure scsi_transport_sas btrfs xor zstd_compress raid6_pq uas
usb_storage crc32c_vpmsum tg3 ipr
Apr 17 00:00:23 lep8d kernel: [ 4309.786944] CPU: 3 PID: 28361 Comm:
stress-ng-hrtim Not tainted 4.15.0-15-generic #16-Ubuntu
Apr 17 00:00:23 lep8d kernel: [ 4309.786950] NIP: c0d0c8b8 LR:
c0120dbc CTR: c0024480
Apr 17 00:00:23 lep8d kernel: [ 4309.786956] REGS: c7f7fd80 TRAP:
0900 Not tainted (4.15.0-15-generic)
Apr 17 00:00:23 lep8d kernel: [ 4309.786957] MSR: 90009033
CR: 28000442 XER: 2000
Apr 17 00:00:23 lep8d kernel: [ 4309.786972] CFAR: c0120db8 SOFTE: 0
Apr 17 00:00:23 lep8d kernel: [ 4309.786972] GPR00: c0120dbc
c02d51377ba0 c16eb400 c02d512b0f88
Apr 17 00:00:23 lep8d kernel: [ 4309.786972] GPR04:
0001 01f40668 0001
Apr 17 00:00:23 lep8d kernel: [ 4309.786972] GPR08: 0001
8003
Apr 17 00:00:23 lep8d kernel: [ 4309.786972] GPR12: c0024480
c7a22100 000186a0
Apr 17 00:00:23 lep8d kernel: [ 4309.786972] GPR16: 7fffed3687e0
0abda98e5db8 8005 00040100
Apr 17 00:00:23 lep8d kernel: [ 4309.786972] GPR20:
418004fc 003c 0843
Apr 17 00:00:23 lep8d kernel: [ 4309.786972] GPR24: c02d512b0f88
c02d51377d30 c02d52d47c00
Apr 17 00:00:23 lep8d kernel: [ 4309.786972] GPR28: c02d51377d50
c02d51377d50 c02d52f04500 c02d512b0f88
Apr 17 00:00:23 lep8d kernel: [ 4309.787035] NIP [c0d0c8b8]
_raw_spin_lock+0x38/0xe0
Apr 17 00:00:23 lep8d kernel: [ 4309.787045] LR [c0120dbc]
dequeue_signal+0xcc/0x260
Apr 17 00:00:23 lep8d kernel: [ 4309.787046] Call Trace:
Apr 17 00:00:23 lep8d kernel: [ 4309.787052] [c02d51377bd0]
[c0120dac] dequeue_signal+0xbc/0x260
Apr 17 00:00:23 lep8d kernel: [ 4309.787059] [c02d51377c20]
[c012459c] get_signal+0x13c/0x7a0
Apr 17 00:00:23 lep8d kernel: [ 4309.787066] [c02d51377d10]
[c001dacc] do_signal+0x7c/0x2c0
Apr 17 00:00:23 lep8d kernel: [ 4309.787072] [c02d51377e00]
[c001deb0] do_notify_resume+0xd0/0x100
Apr 17 00:00:23 lep8d kernel: [ 4309.787083] [c02d51377e30]
[c000b7c4] ret_from_except_lite+0x70/0x74
Apr 17 00:00:23 lep8d kernel: [ 4309.787085] Instruction dump:
Apr 17
[Kernel-packages] [Bug 1777194] syslog
Default Comment by Bridge ** Attachment added: "syslog" https://bugs.launchpad.net/bugs/1777194/+attachment/5153138/+files/syslog_p9.txt ** Changed in: ubuntu Assignee: (unassigned) => Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) ** Package changed: ubuntu => linux (Ubuntu) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1777194 Title: Hard LOCKUP observed on stressing Ubuntu 18 04 Status in linux package in Ubuntu: New Bug description: --Problem Description-- Hard LOCKUP on stressing Ubuntu 18 04 ---Issue observed--- Hard LOCKUP on stressing Ubuntu 18 04 using Ubuntu 18 04, sometimes leads to rcu_stalls. Apr 17 00:00:23 lep8d kernel: [ 4309.786755] Watchdog CPU:3 Hard LOCKUP Apr 17 00:00:23 lep8d kernel: [ 4309.786759] Modules linked in: algif_rng salsa20_generic userio camellia_generic cast6_generic cast_common snd_seq snd_seq_device snd_timer snd soundcore vhost_net serpent_generic tap twofish_generic twofish_common vhost_vsock vmw_vsock_virtio_transport_common vhost vsock lrw unix_diag algif_skcipher cuse sctp tgr192 wp512 rmd320 rmd256 rmd160 hci_vhci rmd128 bluetooth ecdh_generic dccp_ipv4 md4 uhid hid algif_hash dccp af_alg xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c ipt_REJECT nf_reject_ipv4 xt_tcpudp bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter kvm_hv kvm binfmt_misc uio_pdrv_genirq uio vmx_crypto ibmpowernv powernv_op_panel ipmi_powernv Apr 17 00:00:23 lep8d kernel: [ 4309.786899] ipmi_devintf ipmi_msghandler powernv_rng leds_powernv crct10dif_vpmsum sch_fq_codel ip_tables x_tables autofs4 ses enclosure scsi_transport_sas btrfs xor zstd_compress raid6_pq uas usb_storage crc32c_vpmsum tg3 ipr Apr 17 00:00:23 lep8d kernel: [ 4309.786944] CPU: 3 PID: 28361 Comm: stress-ng-hrtim Not tainted 4.15.0-15-generic #16-Ubuntu Apr 17 00:00:23 lep8d kernel: [ 4309.786950] NIP: c0d0c8b8 LR: c0120dbc CTR: c0024480 Apr 17 00:00:23 lep8d kernel: [ 4309.786956] REGS: c7f7fd80 TRAP: 0900 Not tainted (4.15.0-15-generic) Apr 17 00:00:23 lep8d kernel: [ 4309.786957] MSR: 90009033 CR: 28000442 XER: 2000 Apr 17 00:00:23 lep8d kernel: [ 4309.786972] CFAR: c0120db8 SOFTE: 0 Apr 17 00:00:23 lep8d kernel: [ 4309.786972] GPR00: c0120dbc c02d51377ba0 c16eb400 c02d512b0f88 Apr 17 00:00:23 lep8d kernel: [ 4309.786972] GPR04: 0001 01f40668 0001 Apr 17 00:00:23 lep8d kernel: [ 4309.786972] GPR08: 0001 8003 Apr 17 00:00:23 lep8d kernel: [ 4309.786972] GPR12: c0024480 c7a22100 000186a0 Apr 17 00:00:23 lep8d kernel: [ 4309.786972] GPR16: 7fffed3687e0 0abda98e5db8 8005 00040100 Apr 17 00:00:23 lep8d kernel: [ 4309.786972] GPR20: 418004fc 003c 0843 Apr 17 00:00:23 lep8d kernel: [ 4309.786972] GPR24: c02d512b0f88 c02d51377d30 c02d52d47c00 Apr 17 00:00:23 lep8d kernel: [ 4309.786972] GPR28: c02d51377d50 c02d51377d50 c02d52f04500 c02d512b0f88 Apr 17 00:00:23 lep8d kernel: [ 4309.787035] NIP [c0d0c8b8] _raw_spin_lock+0x38/0xe0 Apr 17 00:00:23 lep8d kernel: [ 4309.787045] LR [c0120dbc] dequeue_signal+0xcc/0x260 Apr 17 00:00:23 lep8d kernel: [ 4309.787046] Call Trace: Apr 17 00:00:23 lep8d kernel: [ 4309.787052] [c02d51377bd0] [c0120dac] dequeue_signal+0xbc/0x260 Apr 17 00:00:23 lep8d kernel: [ 4309.787059] [c02d51377c20] [c012459c] get_signal+0x13c/0x7a0 Apr 17 00:00:23 lep8d kernel: [ 4309.787066] [c02d51377d10] [c001dacc] do_signal+0x7c/0x2c0 Apr 17 00:00:23 lep8d kernel: [ 4309.787072] [c02d51377e00] [c001deb0] do_notify_resume+0xd0/0x100 Apr 17 00:00:23 lep8d kernel: [ 4309.787083] [c02d51377e30] [c000b7c4] ret_from_except_lite+0x70/0x74 Apr 17 00:00:23 lep8d kernel: [ 4309.787085] Instruction dump: Apr 17 00:00:23 lep8d kernel: [ 4309.787090] 7c0802a6 6000 fbe1fff8 f821ffd1 7c7f1b78 3940 994d028c 814d0008 Apr 17 00:00:23 lep8d kernel: [ 4309.787102] 7d201829 2c09 40c20010 7d40192d <40c2fff0> 7c2004ac 2fa9 409e001c Apr 17 00:00:23 lep8d kernel: [ 4313.015781] kauditd_printk_skb: 13 callbacks suppressed ---uname output--- # uname -a Linux lep8d 4.15.0-15-generic #16-Ubuntu SMP Wed Apr 4 13:57:51 UTC 2018 ppc64le ppc64le ppc64le GNU/Linux Machine Type = Power 8 BML/Tuleta Additional Info- Hard LOCKUP is also seen on garri BML. syslog is attached. Reproducible :
