Public bug reported:
Summary:
Kernel panic occurs after multiple TCP connection creations/closures to the
localhost.
The bug was found using STAF RPC calls, but is easily reproducible with SSH.
The bug doesn't appear on an identical virtual machine booting from the disk.
The bug is not reproducible on a similarly-prepared Ubuntu 16.04 machine.
The bug is reproducible using an older 4.13.0-16-generic kernel
Reproducible on multiple hardware types.
Unable to create a kernel memory dump due to makedumpfile errors.
apport-bug save attached.
NFSRoot boot options:
vmlinuz initrd=initrd.img boot=nfs root=/dev/nfs
nfsroot=190.0.0.254:/diskless/host/u1616/Ubuntu/17.10 intel_iommu=on
net.ifnames=0 biosdevname=0 apparmor=0 ip=:::::eth0:dhcp
blacklist=i40e,ixgbe,fm10k crashkernel=384M-:768M rw
Software:
OS: Ubuntu 17.10
Kernel: 4.13.0-17-generic x86_64
Reproduction steps:
1. Boot a system from a nfsroot
2. Configure password-less localhost ssh access
3. Run a loop: `while true; do ssh localhost 'uname -a'; done`
4. Wait for system to crash
Trace:
4,1151,52372730,-;general protection fault: 0000 [#1] SMP
4,1152,52372771,-;Modules linked in: arc4 md4 rpcsec_gss_krb5 nls_utf8
auth_rpcgss cifs nfsv4 ccm ipmi_ssif intel_rapl sb_edac x86_pkg_temp_thermal
intel_powerclamp coretemp intel_cstate mei_me input_leds joydev intel_rapl_perf
mei kvm_intel lpc_ich ioatdma kvm irqbypass ipmi_si ipmi_devintf
ipmi_msghandler shpchp acpi_pad acpi_power_meter mac_hid ib_iser rdma_cm iw_cm
ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables
x_tables autofs4 nfsv3 nfs_acl nfs lockd grace sunrpc fscache raid10 raid456
async_raid6_recov async_memcpy async_pq async_xor async_tx xor hid_generic
usbhid hid raid6_pq libcrc32c raid1 raid0 multipath linear uas usb_storage
crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcbc ast ttm aesni_intel igb
drm_kms_helper aes_x86_64 crypto_simd syscopyarea glue_helper
4,1153,52373251,c; sysfillrect dca cryptd sysimgblt i2c_algo_bit fb_sys_fops
ahci ptp drm libahci pps_core wmi
4,1154,52373322,-;CPU: 11 PID: 1848 Comm: STAFProc Not tainted
4.13.0-17-generic #20-Ubuntu
4,1155,52373371,-;Hardware name: Supermicro Super Server/X10SRD-F, BIOS 2.0
12/17/2015
4,1156,52373418,-;task: ffff9d09267f5d00 task.stack: ffffafddc3a70000
4,1157,52373461,-;RIP: 0010:kfree+0x53/0x160
4,1158,52373486,-;RSP: 0018:ffff9d092ecc3bc8 EFLAGS: 00010207
4,1159,52373521,-;RAX: 0000000000000000 RBX: 241c894900000001 RCX:
0000000000000004
4,1160,52373566,-;RDX: 000032d49081cc08 RSI: 0000000000010080 RDI:
000062fac0000000
4,1161,52373611,-;RBP: ffff9d092ecc3be0 R08: 000000000001f4c0 R09:
ffffffff943bb839
4,1162,52373656,-;R10: 00904c7891000000 R11: 0000000000000000 R12:
ffff9d09267ef000
4,1163,52373701,-;R13: ffffffff93fa155e R14: ffff9d09267ef000 R15:
ffff9d09267ef000
4,1164,52373746,-;FS: 00007f3a53313700(0000) GS:ffff9d092ecc0000(0000)
knlGS:0000000000000000
4,1165,52373797,-;CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
4,1166,52373834,-;CR2: 00007fd5c9ffa780 CR3: 00000004666d7000 CR4:
00000000003406e0
4,1167,52373878,-;DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
4,1168,52373923,-;DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
0000000000000400
4,1169,52373968,-;Call Trace:
4,1170,52373987,-; <IRQ>
4,1171,52374009,-; security_sk_free+0x3e/0x50
4,1172,52374042,-; __sk_destruct+0x108/0x190
4,1173,52374070,-; sk_destruct+0x20/0x30
4,1174,52374095,-; __sk_free+0x82/0xa0
4,1175,52374120,-; sk_free+0x19/0x20
4,1176,52374144,-; sock_put+0x14/0x20
4,1177,52374168,-; tcp_v4_rcv+0x94d/0x9d0
4,1178,52374195,-; ip_local_deliver_finish+0x5c/0x1f0
4,1179,52374226,-; ip_local_deliver+0x6f/0xe0
4,1180,52374254,-; ip_rcv_finish+0x120/0x410
4,1181,52374281,-; ip_rcv+0x28c/0x3a0
4,1182,52374305,-; ? update_load_avg+0x46d/0x590
4,1183,52374335,-; __netif_receive_skb_core+0x39a/0xaa0
4,1184,52374369,-; __netif_receive_skb+0x18/0x60
4,1185,52374398,-; ? __netif_receive_skb+0x18/0x60
4,1186,52374428,-; process_backlog+0x89/0x140
4,1187,52374457,-; net_rx_action+0x13b/0x380
4,1188,52374485,-; __do_softirq+0xde/0x2a5
4,1189,52375837,-; do_softirq_own_stack+0x1c/0x30
4,1190,52377188,-; </IRQ>
4,1191,52378538,-; do_softirq.part.17+0x4e/0x50
4,1192,52379869,-; __local_bh_enable_ip+0x5a/0x60
4,1193,52381174,-; ip_finish_output2+0x172/0x3a0
4,1194,52382442,-; ip_finish_output+0x190/0x250
4,1195,52383658,-; ? ip_finish_output+0x190/0x250
4,1196,52384833,-; ip_output+0x70/0xe0
4,1197,52385959,-; ? lock_timer_base+0x81/0xa0
4,1198,52387043,-; ip_local_out+0x35/0x40
4,1199,52388078,-; ip_queue_xmit+0x160/0x3e0
4,1200,52389068,-; ? __alloc_skb+0x87/0x1e0
4,1201,52390019,-; tcp_transmit_skb+0x538/0x9e0
4,1202,52390944,-; tcp_send_ack.part.35+0xbd/0x130
4,1203,52391849,-; tcp_send_ack+0x16/0x20
4,1204,52392729,-; tcp_cleanup_rbuf+0x67/0x100
4,1205,52393600,-; tcp_recvmsg+0x572/0xb60
4,1206,52394464,-; inet_recvmsg+0x4b/0xc0
4,1207,52395307,-; sock_recvmsg+0x3d/0x50
4,1208,52396135,-; sock_read_iter+0x90/0xe0
4,1209,52396955,-; new_sync_read+0xde/0x130
4,1210,52397766,-; __vfs_read+0x26/0x40
4,1211,52398568,-; vfs_read+0x8e/0x130
4,1212,52399347,-; SyS_read+0x55/0xc0
4,1213,52400099,-; entry_SYSCALL_64_fastpath+0x1e/0xa9
4,1214,52400845,-;RIP: 0033:0x7f3a58da0d5d
4,1215,52401583,-;RSP: 002b:00007f3a53310e70 EFLAGS: 00000246 ORIG_RAX:
0000000000000000
4,1216,52402370,-;RAX: ffffffffffffffda RBX: 00007f3a44000078 RCX:
00007f3a58da0d5d
4,1217,52403153,-;RDX: 0000000000000020 RSI: 00007f3a4400b5f8 RDI:
000000000000000f
4,1218,52403952,-;RBP: 0000000000001010 R08: 0000000000000000 R09:
0000000000000000
4,1219,52404752,-;R10: 00007f3a593e4d20 R11: 0000000000000246 R12:
00007f3a44000020
4,1220,52405553,-;R13: 0000000000001000 R14: 00007f3a44000078 R15:
0000000000000001
4,1221,52406363,-;Code: 00 80 49 01 da 0f 82 1c 01 00 00 48 c7 c7 00 00 00 80
48 2b 3d 7f e6 c1 00 49 01 fa 49 c1 ea 0c 49 c1 e2 06 4c 03 15 5d e6 c1 00 <49>
8b 42 20 48 8d 50 ff a8 01 4c 0f 45 d2 49 8b 52 20 48 8d 42
1,1222,52408171,-;RIP: kfree+0x53/0x160 RSP: ffff9d092ecc3bc8
4,1223,52409129,-;---[ end trace f7b53e7f81a1cbda ]---
0,1224,52414686,-;Kernel panic - not syncing: Fatal exception in interrupt
** Affects: linux (Ubuntu)
Importance: Undecided
Status: New
** Attachment added: "apport-bug save"
https://bugs.launchpad.net/bugs/1734327/+attachment/5014060/+files/bug.crash
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1734327
Title:
Kernel panic on a nfsroot system
Status in linux package in Ubuntu:
New
Bug description:
Summary:
Kernel panic occurs after multiple TCP connection creations/closures to the
localhost.
The bug was found using STAF RPC calls, but is easily reproducible with SSH.
The bug doesn't appear on an identical virtual machine booting from the disk.
The bug is not reproducible on a similarly-prepared Ubuntu 16.04 machine.
The bug is reproducible using an older 4.13.0-16-generic kernel
Reproducible on multiple hardware types.
Unable to create a kernel memory dump due to makedumpfile errors.
apport-bug save attached.
NFSRoot boot options:
vmlinuz initrd=initrd.img boot=nfs root=/dev/nfs
nfsroot=190.0.0.254:/diskless/host/u1616/Ubuntu/17.10 intel_iommu=on
net.ifnames=0 biosdevname=0 apparmor=0 ip=:::::eth0:dhcp
blacklist=i40e,ixgbe,fm10k crashkernel=384M-:768M rw
Software:
OS: Ubuntu 17.10
Kernel: 4.13.0-17-generic x86_64
Reproduction steps:
1. Boot a system from a nfsroot
2. Configure password-less localhost ssh access
3. Run a loop: `while true; do ssh localhost 'uname -a'; done`
4. Wait for system to crash
Trace:
4,1151,52372730,-;general protection fault: 0000 [#1] SMP
4,1152,52372771,-;Modules linked in: arc4 md4 rpcsec_gss_krb5 nls_utf8
auth_rpcgss cifs nfsv4 ccm ipmi_ssif intel_rapl sb_edac x86_pkg_temp_thermal
intel_powerclamp coretemp intel_cstate mei_me input_leds joydev intel_rapl_perf
mei kvm_intel lpc_ich ioatdma kvm irqbypass ipmi_si ipmi_devintf
ipmi_msghandler shpchp acpi_pad acpi_power_meter mac_hid ib_iser rdma_cm iw_cm
ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables
x_tables autofs4 nfsv3 nfs_acl nfs lockd grace sunrpc fscache raid10 raid456
async_raid6_recov async_memcpy async_pq async_xor async_tx xor hid_generic
usbhid hid raid6_pq libcrc32c raid1 raid0 multipath linear uas usb_storage
crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcbc ast ttm aesni_intel igb
drm_kms_helper aes_x86_64 crypto_simd syscopyarea glue_helper
4,1153,52373251,c; sysfillrect dca cryptd sysimgblt i2c_algo_bit fb_sys_fops
ahci ptp drm libahci pps_core wmi
4,1154,52373322,-;CPU: 11 PID: 1848 Comm: STAFProc Not tainted
4.13.0-17-generic #20-Ubuntu
4,1155,52373371,-;Hardware name: Supermicro Super Server/X10SRD-F, BIOS 2.0
12/17/2015
4,1156,52373418,-;task: ffff9d09267f5d00 task.stack: ffffafddc3a70000
4,1157,52373461,-;RIP: 0010:kfree+0x53/0x160
4,1158,52373486,-;RSP: 0018:ffff9d092ecc3bc8 EFLAGS: 00010207
4,1159,52373521,-;RAX: 0000000000000000 RBX: 241c894900000001 RCX:
0000000000000004
4,1160,52373566,-;RDX: 000032d49081cc08 RSI: 0000000000010080 RDI:
000062fac0000000
4,1161,52373611,-;RBP: ffff9d092ecc3be0 R08: 000000000001f4c0 R09:
ffffffff943bb839
4,1162,52373656,-;R10: 00904c7891000000 R11: 0000000000000000 R12:
ffff9d09267ef000
4,1163,52373701,-;R13: ffffffff93fa155e R14: ffff9d09267ef000 R15:
ffff9d09267ef000
4,1164,52373746,-;FS: 00007f3a53313700(0000) GS:ffff9d092ecc0000(0000)
knlGS:0000000000000000
4,1165,52373797,-;CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
4,1166,52373834,-;CR2: 00007fd5c9ffa780 CR3: 00000004666d7000 CR4:
00000000003406e0
4,1167,52373878,-;DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
4,1168,52373923,-;DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
0000000000000400
4,1169,52373968,-;Call Trace:
4,1170,52373987,-; <IRQ>
4,1171,52374009,-; security_sk_free+0x3e/0x50
4,1172,52374042,-; __sk_destruct+0x108/0x190
4,1173,52374070,-; sk_destruct+0x20/0x30
4,1174,52374095,-; __sk_free+0x82/0xa0
4,1175,52374120,-; sk_free+0x19/0x20
4,1176,52374144,-; sock_put+0x14/0x20
4,1177,52374168,-; tcp_v4_rcv+0x94d/0x9d0
4,1178,52374195,-; ip_local_deliver_finish+0x5c/0x1f0
4,1179,52374226,-; ip_local_deliver+0x6f/0xe0
4,1180,52374254,-; ip_rcv_finish+0x120/0x410
4,1181,52374281,-; ip_rcv+0x28c/0x3a0
4,1182,52374305,-; ? update_load_avg+0x46d/0x590
4,1183,52374335,-; __netif_receive_skb_core+0x39a/0xaa0
4,1184,52374369,-; __netif_receive_skb+0x18/0x60
4,1185,52374398,-; ? __netif_receive_skb+0x18/0x60
4,1186,52374428,-; process_backlog+0x89/0x140
4,1187,52374457,-; net_rx_action+0x13b/0x380
4,1188,52374485,-; __do_softirq+0xde/0x2a5
4,1189,52375837,-; do_softirq_own_stack+0x1c/0x30
4,1190,52377188,-; </IRQ>
4,1191,52378538,-; do_softirq.part.17+0x4e/0x50
4,1192,52379869,-; __local_bh_enable_ip+0x5a/0x60
4,1193,52381174,-; ip_finish_output2+0x172/0x3a0
4,1194,52382442,-; ip_finish_output+0x190/0x250
4,1195,52383658,-; ? ip_finish_output+0x190/0x250
4,1196,52384833,-; ip_output+0x70/0xe0
4,1197,52385959,-; ? lock_timer_base+0x81/0xa0
4,1198,52387043,-; ip_local_out+0x35/0x40
4,1199,52388078,-; ip_queue_xmit+0x160/0x3e0
4,1200,52389068,-; ? __alloc_skb+0x87/0x1e0
4,1201,52390019,-; tcp_transmit_skb+0x538/0x9e0
4,1202,52390944,-; tcp_send_ack.part.35+0xbd/0x130
4,1203,52391849,-; tcp_send_ack+0x16/0x20
4,1204,52392729,-; tcp_cleanup_rbuf+0x67/0x100
4,1205,52393600,-; tcp_recvmsg+0x572/0xb60
4,1206,52394464,-; inet_recvmsg+0x4b/0xc0
4,1207,52395307,-; sock_recvmsg+0x3d/0x50
4,1208,52396135,-; sock_read_iter+0x90/0xe0
4,1209,52396955,-; new_sync_read+0xde/0x130
4,1210,52397766,-; __vfs_read+0x26/0x40
4,1211,52398568,-; vfs_read+0x8e/0x130
4,1212,52399347,-; SyS_read+0x55/0xc0
4,1213,52400099,-; entry_SYSCALL_64_fastpath+0x1e/0xa9
4,1214,52400845,-;RIP: 0033:0x7f3a58da0d5d
4,1215,52401583,-;RSP: 002b:00007f3a53310e70 EFLAGS: 00000246 ORIG_RAX:
0000000000000000
4,1216,52402370,-;RAX: ffffffffffffffda RBX: 00007f3a44000078 RCX:
00007f3a58da0d5d
4,1217,52403153,-;RDX: 0000000000000020 RSI: 00007f3a4400b5f8 RDI:
000000000000000f
4,1218,52403952,-;RBP: 0000000000001010 R08: 0000000000000000 R09:
0000000000000000
4,1219,52404752,-;R10: 00007f3a593e4d20 R11: 0000000000000246 R12:
00007f3a44000020
4,1220,52405553,-;R13: 0000000000001000 R14: 00007f3a44000078 R15:
0000000000000001
4,1221,52406363,-;Code: 00 80 49 01 da 0f 82 1c 01 00 00 48 c7 c7 00 00 00 80
48 2b 3d 7f e6 c1 00 49 01 fa 49 c1 ea 0c 49 c1 e2 06 4c 03 15 5d e6 c1 00 <49>
8b 42 20 48 8d 50 ff a8 01 4c 0f 45 d2 49 8b 52 20 48 8d 42
1,1222,52408171,-;RIP: kfree+0x53/0x160 RSP: ffff9d092ecc3bc8
4,1223,52409129,-;---[ end trace f7b53e7f81a1cbda ]---
0,1224,52414686,-;Kernel panic - not syncing: Fatal exception in interrupt
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1734327/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
