This is on node running 4.9.11 TIPC. 9 nodes in cluster, 7 of which are running the same 4.9.11 TIPC (on x86-64), 2 running an old 1.7 TIPC (on PPC).
It keeps cycling through these same logs every few seconds. [118768.064830] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [swapper/3:0] [118768.069831] NMI watchdog: BUG: soft lockup - CPU#6 stuck for 22s! [sysmon:31634] [118768.069855] Modules linked in: nf_log_ipv4 nf_log_common xt_LOG iptable_mangle iptable_raw sctp libcrc32c e1000e tipc udp_tunnel ip6_udp_tunnel 8021q garp iTCO_wdt ipmiq_drv(O) sio_mmc(O) xt_physdev br_netfilter event_drv(O) bridge stp llc lockd nf_conntrack_ipv4 nf_defrag_ipv4 grace ip6t_REJECT nf_reject_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack pt_timer_info(O) ip6table_filter ip6_tables ddi(O) iTCO_vendor_support usb_storage igb pcspkr ixgbe i2c_i801 intel_ips i2c_algo_bit lpc_ich i2c_core mfd_core ioatdma ptp dca pps_core mdio tpm_tis tpm sunrpc [last unloaded: iTCO_wdt] [118768.069856] CPU: 6 PID: 31634 Comm: sysmon Tainted: G O L 4.4.0 #24 [118768.069857] Hardware name: PT AMC124/Base Board Product Name, BIOS LGNAJFIP.PTI.0012.P15 01/15/2014 [118768.069859] task: ffff8802d99551c0 ti: ffff8802d9ec4000 task.ti: ffff8802d9ec4000 [118768.069862] RIP: 0010:[<ffffffff810c34c2>] [<ffffffff810c34c2>] queued_spin_lock_slowpath+0x42/0x160 [118768.069863] RSP: 0018:ffff8802d9ec7c98 EFLAGS: 00000202 [118768.069864] RAX: 0000000000000101 RBX: ffffffff81ce6300 RCX: 0000000000000001 [118768.069865] RDX: 0000000000000101 RSI: 0000000000000001 RDI: ffff88034fbe3338 [118768.069866] RBP: ffff8802d9ec7c98 R08: 0000000000000101 R09: 0000000000000000 [118768.069867] R10: 00007fffed1ef1bc R11: 0000000000000000 R12: ffff88034fbe3338 [118768.069868] R13: ffff8802d9ec7d94 R14: 00000000000036b2 R15: 0000000000000000 [118768.069869] FS: 00007fe87cc36740(0000) GS:ffff88035fcc0000(0000) knlGS:0000000000000000 [118768.069871] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [118768.069872] CR2: 0000000001e6b000 CR3: 000000033d62a000 CR4: 00000000000006e0 [118768.069872] Stack: [118768.069874] ffff8802d9ec7ca8 ffffffff816e087c ffff8802d9ec7d08 ffffffffa024a0f6 [118768.069876] 00000000d9ec7d48 ffffffff81007de1 ffff88034fbe3300 ffff88035128c000 [118768.069878] ffff88035fcc3fc0 ffff8802d9ec7de8 ffff880036557800 ffff88009ae21680 [118768.069878] Call Trace: [118768.069881] [<ffffffff816e087c>] _raw_spin_lock_bh+0x2c/0x40 [118768.069886] [<ffffffffa024a0f6>] tipc_nametbl_translate+0x96/0x1e0 [tipc] [118768.069888] [<ffffffff81007de1>] ? dump_trace+0xc1/0x2e0 [118768.069893] [<ffffffffa0251e99>] __tipc_sendmsg+0x259/0x3e0 [tipc] [118768.069895] [<ffffffff8101483f>] ? save_stack_trace+0x2f/0x50 [118768.069897] [<ffffffff811d3d9f>] ? __save_stack_trace+0x2f/0x40 [118768.069899] [<ffffffff811d4af9>] ? create_object+0x1f9/0x2a0 [118768.069904] [<ffffffffa02521d1>] tipc_connect+0x141/0x190 [tipc] [118768.069906] [<ffffffff815bfaa3>] SYSC_connect+0xa3/0xc0 [118768.069909] [<ffffffff811195cc>] ? __audit_syscall_exit+0x1fc/0x260 [118768.069910] [<ffffffff81002186>] ? do_audit_syscall_entry+0x66/0x70 [118768.069912] [<ffffffff81002798>] ? syscall_trace_enter_phase1+0xf8/0x120 [118768.069914] [<ffffffff8107b138>] ? syscall_slow_exit_work+0x43/0xae [118768.069917] [<ffffffff815bff1e>] SyS_connect+0xe/0x10 [118768.069918] [<ffffffff816e0c97>] entry_SYSCALL_64_fastpath+0x12/0x6a [118768.069939] Code: b9 01 00 00 00 eb 02 89 c6 f7 c6 00 ff ff ff 75 41 83 fe 01 89 ca 89 f0 41 0f 44 d0 f0 0f b1 17 39 f0 75 e3 83 fa 01 75 04 eb 0d <f3> 90 8b 07 84 c0 75 f8 66 c7 07 01 00 5d c3 8b 37 81 fe 00 01 [118768.365578] Modules linked in: nf_log_ipv4 nf_log_common xt_LOG iptable_mangle iptable_raw sctp libcrc32c e1000e tipc udp_tunnel ip6_udp_tunnel 8021q garp iTCO_wdt ipmiq_drv(O) sio_mmc(O) xt_physdev br_netfilter event_drv(O) bridge stp llc lockd nf_conntrack_ipv4 nf_defrag_ipv4 grace ip6t_REJECT nf_reject_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack pt_timer_info(O) ip6table_filter ip6_tables ddi(O) iTCO_vendor_support usb_storage igb pcspkr ixgbe i2c_i801 intel_ips i2c_algo_bit lpc_ich i2c_core mfd_core ioatdma ptp dca pps_core mdio tpm_tis tpm sunrpc [last unloaded: iTCO_wdt] [118768.418992] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G O L 4.4.0 #24 [118768.426196] Hardware name: PT AMC124/Base Board Product Name, BIOS LGNAJFIP.PTI.0012.P15 01/15/2014 [118768.435306] task: ffff880351088000 ti: ffff880351090000 task.ti: ffff880351090000 [118768.442864] RIP: 0010:[<ffffffff810c3575>] [<ffffffff810c3575>] queued_spin_lock_slowpath+0xf5/0x160 [118768.452172] RSP: 0018:ffff88035fc63c68 EFLAGS: 00000246 [118768.457568] RAX: 0000000000000000 RBX: ffff88035128c000 RCX: ffff88035fc75fc0 [118768.464780] RDX: ffff88035fc15fc0 RSI: 0000000000100000 RDI: ffff88035128d140 [118768.471991] RBP: ffff88035fc63c68 R08: 0000000000000101 R09: ffff88034e8ce0c0 [118768.479193] R10: 0000000000000001 R11: ffff88034e764640 R12: ffffffff81ce6300 [118768.486399] R13: ffff88035128d140 R14: 000000000100100d R15: 0000000000000000 [118768.493611] FS: 0000000000000000(0000) GS:ffff88035fc60000(0000) knlGS:0000000000000000 [118768.501778] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [118768.507602] CR2: 00007f8f8602e360 CR3: 0000000001c0a000 CR4: 00000000000006e0 [118768.514808] Stack: [118768.516911] ffff88035fc63c78 ffffffff816e087c ffff88035fc63cc8 ffffffffa024a3c7 [118768.524443] ffff880300000000 000000020100100d 000000000000af3e 0000000000000080 [118768.531976] ffffffff81ce6300 000000000100100d 0000000000000000 0000000000000000 [118768.539508] Call Trace: [118768.542036] <IRQ> [118768.544049] [<ffffffff816e087c>] _raw_spin_lock_bh+0x2c/0x40 [118768.550069] [<ffffffffa024a3c7>] tipc_nametbl_withdraw+0x57/0x130 [tipc] [118768.556932] [<ffffffffa024d923>] tipc_node_write_unlock+0xb3/0x110 [tipc] [118768.563885] [<ffffffffa024e112>] tipc_node_link_down+0x92/0x120 [tipc] [118768.570578] [<ffffffffa024e318>] tipc_node_timeout+0x108/0x110 [tipc] [118768.577185] [<ffffffffa024e210>] ? tipc_node_calculate_timer+0x70/0x70 [tipc] [118768.584484] [<ffffffffa024e210>] ? tipc_node_calculate_timer+0x70/0x70 [tipc] [118768.591781] [<ffffffff810e1644>] call_timer_fn+0x44/0x110 [118768.597349] [<ffffffffa024e210>] ? tipc_node_calculate_timer+0x70/0x70 [tipc] [118768.604646] [<ffffffff810e2c4c>] run_timer_softirq+0x22c/0x280 [118768.610647] [<ffffffff81083d88>] __do_softirq+0xc8/0x260 [118768.616127] [<ffffffff81084123>] irq_exit+0x83/0xb0 [118768.621168] [<ffffffff816e3265>] do_IRQ+0x65/0xf0 [118768.626041] [<ffffffff816e173f>] common_interrupt+0x7f/0x7f [118768.631780] <EOI> [118768.633791] [<ffffffff815938ed>] ? cpuidle_enter_state+0xad/0x200 [118768.640248] [<ffffffff815938d1>] ? cpuidle_enter_state+0x91/0x200 [118768.646502] [<ffffffff81593a77>] cpuidle_enter+0x17/0x20 [118768.651980] [<ffffffff810bcdc7>] call_cpuidle+0x37/0x60 [118768.657367] [<ffffffff81593a53>] ? cpuidle_select+0x13/0x20 [118768.663108] [<ffffffff810bd001>] cpu_startup_entry+0x211/0x2d0 [118768.669107] [<ffffffff8103b213>] start_secondary+0x103/0x130 [118768.674935] Code: 12 48 c1 ea 0c 83 e8 01 83 e2 30 48 98 48 81 c2 c0 5f 01 00 48 03 14 c5 40 b4 d1 81 48 89 0a 8b 41 08 85 c0 75 0d f3 90 8b 41 08 <85> c0 74 f7 eb 02 f3 90 8b 17 66 85 d2 75 f7 39 f2 75 0f 89 d0 etc. . . . ------------------------------------------------------------------------------ Announcing the Oxford Dictionaries API! The API offers world-renowned dictionary content that is easy and intuitive to access. Sign up for an account today to start using our lexical data to power your apps and projects. Get started today and enter our developer competition. http://sdm.link/oxford _______________________________________________ tipc-discussion mailing list tipc-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/tipc-discussion
