Public bug reported: After running offload enabled LNST ipv6 vxlan ovs test (recipes/ovs_offload/1_virt_ovs_vxlan_ipv6.xml) with the setup that it creates multiple times till it crashes. The test itself and other LNST tests pass, it's the shutdown phase that causes this. There are different stack traces that usually relate to some kind of allocation (or ext4, inode), see one below.
scenario : 1. Install lnst tests git clone https://github.com/jpirko/lnst.git && cd lnst && ./setup.py install 2. prepare OVS offload enable setup (2 machines) connected Back to Back 3. enable 2 VM's on the mlnx5 Physical Function on each machine 4. setup lnst on vm and HV (run lnst-slave) 5. run IPv VXLAN lnst test in loop for example #lnst-ctl -d --pools=talat run recipes/ovs_offload/1_virt_ovs_vxlan_ipv6.xml Call trace kernel: [76406.381439] Oops: 0000 [#1] SMP kernel: [76406.419297] Modules linked in: act_mirred act_gact act_tunnel_key cls_flower sch_ingress vport_vxlan vxlan ip6_udp_tunnel udp_tunnel vfio_pci vfio_iommu_type1 vfio_virqfd vfio mlx5_ib ib_core nfsv3 nfs fscache xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat xt_conntrack ipt_REJECT nf_reject_ipv4 xt_tcpudp ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter bridge stp llc openvswitch nf_conntrack_ipv6 nf_nat_ipv6 nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_defrag_ipv6 nf_nat nf_conntrack intel_rapl sb_edac edac_core x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcbc aesni_intel aes_x86_64 crypto_simd glue_helper cryptd ipmi_ssif intel_cstate ipmi_si input_leds joydev ipmi_devintf kernel: [76406.981750] mei_me dcdbas intel_rapl_perf shpchp mei ipmi_msghandler lpc_ich mac_hid acpi_power_meter configfs nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables x_tables autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear mlx4_en hid_generic tg3 mlx5_core usbhid mlx4_core ahci ptp mxm_wmi hid libahci megaraid_sas devlink pps_core fjes wmi kernel: [76407.335099] CPU: 25 PID: 5253 Comm: ip Not tainted 4.10.0-19-generic #21-Ubuntu kernel: [76407.446475] Hardware name: Dell Inc. PowerEdge R730/072T6D, BIOS 2.3.4 11/08/2016 kernel: [76407.558645] task: ffff9a2b76f89680 task.stack: ffffbda6c76a8000 kernel: [76407.618666] RIP: 0010:rb_erase+0x194/0x350 kernel: [76407.676596] RSP: 0018:ffffbda6c76ab4f0 EFLAGS: 00010046 kernel: [76407.735460] RAX: ffff9a2c2cc30bc0 RBX: ffff9a2c53372d18 RCX: 0000000000000000 kernel: [76407.797100] RDX: 0000000000000000 RSI: ffff9a2c53372d20 RDI: ffff9a2c2cc30a40 kernel: [76407.858831] RBP: ffffbda6c76ab4f0 R08: 0000000000000000 R09: 000000018040002e kernel: [76407.921323] R10: ffff9a2c2cc30b40 R11: 00000000000f9e00 R12: ffff9a2c2cc30a40 kernel: [76407.984793] R13: ffff9a2c53372d18 R14: 0000000000000046 R15: ffff9a2c5536b800 kernel: [76408.048453] FS: 00007f3d96082d80(0000) GS:ffff9a2c5f300000(0000) knlGS:0000000000000000 kernel: [76408.166912] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 kernel: [76408.227997] CR2: 0000000000000000 CR3: 00000010181b5000 CR4: 00000000003426e0 kernel: [76408.290488] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 kernel: [76408.351513] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 kernel: [76408.410855] Call Trace: kernel: [76408.463227] private_free_iova+0x37/0x80 kernel: [76408.516647] iova_magazine_free_pfns+0x41/0x80 kernel: [76408.569805] free_iova_fast+0xd4/0x210 kernel: [76408.620830] flush_unmaps_timeout+0xc5/0x1c0 kernel: [76408.671569] intel_unmap+0x15a/0x210 kernel: [76408.720173] intel_unmap_page+0xe/0x10 kernel: [76408.767571] mlx5e_page_release+0xae/0x110 [mlx5_core] kernel: [76408.816057] mlx5e_destroy_rq+0xd8/0x130 [mlx5_core] kernel: [76408.863341] mlx5e_close_channel+0xd1/0x1d0 [mlx5_core] kernel: [76408.910105] mlx5e_close_channels+0xd9/0x110 [mlx5_core] kernel: [76408.955876] mlx5e_close_locked+0x60/0x80 [mlx5_core] kernel: [76409.000267] mlx5e_close+0x33/0x50 [mlx5_core] kernel: [76409.042831] __dev_close_many+0x99/0x100 kernel: [76409.083537] __dev_close+0x45/0x70 kernel: [76409.122035] __dev_change_flags+0x9d/0x160 kernel: [76409.160600] dev_change_flags+0x29/0x60 kernel: [76409.198250] do_setlink+0x338/0xd20 kernel: [76409.235159] ? nla_parse+0x31/0x110 kernel: [76409.270899] rtnl_newlink+0x5c6/0x860 kernel: [76409.306535] ? security_capable+0x20/0x60 kernel: [76409.341989] ? ns_capable_common+0x68/0x80 kernel: [76409.377215] ? ns_capable+0x13/0x20 kernel: [76409.411543] rtnetlink_rcv_msg+0xe6/0x210 kernel: [76409.445368] ? __kmalloc_node_track_caller+0x1de/0x2a0 kernel: [76409.480369] ? __alloc_skb+0x87/0x1e0 kernel: [76409.512880] ? rtnl_newlink+0x860/0x860 kernel: [76409.544854] netlink_rcv_skb+0xa4/0xc0 kernel: [76409.576628] rtnetlink_rcv+0x28/0x30 kernel: [76409.608717] netlink_unicast+0x18c/0x220 kernel: [76409.640348] netlink_sendmsg+0x2f7/0x3b0 kernel: [76409.671522] ? aa_sock_msg_perm+0x61/0x150 kernel: [76409.702695] sock_sendmsg+0x38/0x50 kernel: [76409.733084] ___sys_sendmsg+0x2c2/0x2d0 kernel: [76409.763826] ? mem_cgroup_commit_charge+0x7e/0x510 kernel: [76409.796237] ? lru_cache_add_active_or_unevictable+0x36/0xb0 kernel: [76409.830626] ? handle_mm_fault+0xf9b/0x1360 kernel: [76409.863080] ? __dentry_kill+0x110/0x160 kernel: [76409.894984] __sys_sendmsg+0x54/0x90 kernel: [76409.926068] SyS_sendmsg+0x12/0x20 kernel: [76409.956873] entry_SYSCALL_64_fastpath+0x1e/0xad kernel: [76409.989499] RIP: 0033:0x7f3d95799237 kernel: [76410.020795] RSP: 002b:00007ffd54522db8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e kernel: [76410.087104] RAX: ffffffffffffffda RBX: 00007ffd5452aec0 RCX: 00007f3d95799237 kernel: [76410.127078] RDX: 0000000000000000 RSI: 00007ffd54522e00 RDI: 0000000000000003 kernel: [76410.167215] RBP: 00007ffd54522e00 R08: 0000000000000001 R09: fefefeff77686d74 kernel: [76410.207897] R10: 00007ffd5452c7c0 R11: 0000000000000246 R12: 00007ffd54522e40 kernel: [76410.248850] R13: 00005612b473a020 R14: 00007ffd5452aec0 R15: 0000000000000000 kernel: [76410.290185] Code: 10 f6 c2 01 0f 84 d3 00 00 00 48 83 e2 fc 0f 84 1e ff ff ff 48 89 c1 48 89 d0 48 8b 50 08 48 39 ca 0f 85 71 ff ff ff 48 8b 50 10 <f6> 02 01 75 3a 48 8b 7a 08 48 89 c1 48 83 c9 01 48 89 78 10 48 kernel: [76410.416172] RIP: rb_erase+0x194/0x350 RSP: ffffbda6c76ab4f0 kernel: [76410.458747] CR2: 0000000000000000 kernel: [76410.498095] ---[ end trace 8d9a539d70087300 ]--- other traces kernel: general protection fault: 0000 [#1] SMP kernel: Modules linked in: act_mirred act_gact act_tunnel_key cls_flower sch_ingress vport_vxlan vxlan ip6_udp_tunnel udp_tunnel vfio_pci vfio_iommu_type1 vfio_virqfd vfio mlx5_ib ib_core nfsv3 rpcsec_gss_kr kernel: lpc_ich dcdbas intel_rapl_perf ipmi_devintf shpchp ipmi_msghandler mac_hid acpi_power_meter nfsd configfs auth_rpcgss nfs_acl lockd grace sunrpc ip_tables x_tables autofs4 btrfs raid10 raid456 async kernel: CPU: 19 PID: 1902 Comm: ovs-vswitchd Not tainted 4.10.8+ #13 kernel: Hardware name: Dell Inc. PowerEdge R730/072T6D, BIOS 2.3.4 11/08/2016 kernel: task: ffff8d28f135ad00 task.stack: ffffadb848858000 kernel: RIP: 0010:kmem_cache_alloc_trace+0x7b/0x1c0 kernel: RSP: 0018:ffffadb84885bda0 EFLAGS: 00010286 kernel: RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00000000000028da kernel: RDX: 00000000000028d9 RSI: 00000000014000c0 RDI: 000000000001c5c0 kernel: RBP: ffffadb84885bde0 R08: ffff8d311f25c5c0 R09: ffff8d291f407980 kernel: R10: ffff006400000000 R11: ffff8d28f56c5280 R12: 00000000014000c0 kernel: R13: ffffffff9218255b R14: 00007ffce91a4c70 R15: ffff8d291f407980 kernel: FS: 00007fda84415940(0000) GS:ffff8d311f240000(0000) knlGS:0000000000000000 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 kernel: CR2: 0000564f3e7bc224 CR3: 0000000830bfc000 CR4: 00000000003426e0 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 kernel: Call Trace: kernel: ? kmem_cache_alloc+0xd7/0x1b0 kernel: sock_alloc_inode+0x3b/0xc0 kernel: alloc_inode+0x1d/0x90 kernel: new_inode_pseudo+0x11/0x60 kernel: sock_alloc+0x1c/0x80 kernel: SYSC_accept4+0x71/0x210 kernel: ? ____fput+0xe/0x10 kernel: ? task_work_run+0x83/0xa0 kernel: SyS_accept+0x10/0x20 kernel: entry_SYSCALL_64_fastpath+0x1e/0xad kernel: RIP: 0033:0x7fda8394f8ed kernel: RSP: 002b:00007ffce91a4c50 EFLAGS: 00000293 ORIG_RAX: 000000000000002b kernel: RAX: ffffffffffffffda RBX: 000055e38935c6a0 RCX: 00007fda8394f8ed kernel: RDX: 00007ffce91a4c6c RSI: 00007ffce91a4c70 RDI: 0000000000000069 kernel: RBP: 000055e3893800d0 R08: 0000000000000000 R09: 0000000000000001 kernel: R10: 000000000003516c R11: 0000000000000293 R12: 00007ffce91a4da0 kernel: R13: 00007ffce91a4c70 R14: 00007ffce91a4d40 R15: 000055e38937dc40 kernel: Code: 08 65 4c 03 05 f7 1d 3f 6e 49 83 78 10 00 4d 8b 10 0f 84 f0 00 00 00 4d 85 d2 0f 84 e7 00 00 00 49 63 41 20 48 8d 4a 01 49 8b 39 <49> 8b 1c 02 4c 89 d0 65 48 0f c7 0f 0f 94 c0 84 c0 74 bb 49 63 kernel: general protection fault: 0000 [#1] SMP kernel: Modules linked in: act_tunnel_key act_gact act_mirred cls_flower mlx5_ib mlx5_core mst_pciconf(OE) mst_pci(OE) ib_umad nfsv3 nfs fscache vfio_pci vfio_iommu_type1 vfio_virqfd vfio netconsole ib_core kernel: stp llc ipmi_si intel_cstate joydev input_leds mei_me ipmi_devintf mei lpc_ich intel_rapl_perf shpchp dcdbas ipmi_msghandler mac_hid acpi_power_meter nfsd auth_rpcgss nfs_acl lockd grace sunrpc conf kernel: CPU: 37 PID: 12495 Comm: modprobe Tainted: G OE 4.10.6+ #8 kernel: Hardware name: Dell Inc. PowerEdge R730/072T6D, BIOS 2.3.4 11/08/2016 kernel: task: ffffa0b485b60000 task.stack: ffffb84c89260000 kernel: RIP: 0010:__kmalloc+0xbc/0x200 kernel: RSP: 0018:ffffb84c89263be0 EFLAGS: 00010286 kernel: RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000018061 kernel: RDX: 0000000000018060 RSI: 0000000000000000 RDI: 000000000001c5c0 kernel: RBP: ffffb84c89263c18 R08: ffffa0b49f49c5c0 R09: ffffa0ac9f407980 kernel: R10: ffff006400000000 R11: 000000006f6e736f R12: 00000000014080c0 kernel: R13: 0000000000000040 R14: ffffffff8c6d0f3e R15: ffffa0ac9f407980 kernel: FS: 00007f4ef6650700(0000) GS:ffffa0b49f480000(0000) knlGS:0000000000000000 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 kernel: CR2: 000055cdb53d4228 CR3: 000000103ace8000 CR4: 00000000003426e0 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 kernel: Call Trace: kernel: ? ext4fs_dirhash+0xc2/0x2b0 kernel: ext4_htree_store_dirent+0x3e/0x120 kernel: htree_dirblock_to_tree+0xf3/0x290 kernel: ? dput+0x34/0x250 kernel: ext4_htree_fill_tree+0xb5/0x320 kernel: ? kmem_cache_alloc_trace+0xdb/0x1c0 kernel: ext4_readdir+0x701/0xa20 kernel: ? lru_cache_add_active_or_unevictable+0x36/0xb0 kernel: iterate_dir+0x172/0x1a0 kernel: SyS_getdents+0x99/0x120 kernel: ? fillonedir+0x100/0x100 kernel: entry_SYSCALL_64_fastpath+0x1e/0xad kernel: RIP: 0033:0x7f4ef614331b kernel: RSP: 002b:00007fffd33e0b60 EFLAGS: 00000206 ORIG_RAX: 000000000000004e kernel: RAX: ffffffffffffffda RBX: 00007f4ef643cb58 RCX: 00007f4ef614331b kernel: RDX: 0000000000008000 RSI: 000055cdb53cc220 RDI: 0000000000000000 kernel: RBP: 00007f4ef643cb00 R08: 00007f4ef643cbb8 R09: 0000000000000000 kernel: R10: 000055cdb53cc1f0 R11: 0000000000000206 R12: 00007f4ef643cb58 kernel: R13: 0000000000008040 R14: 00007f4ef643cb58 R15: 000000000000270f kernel: Code: 08 65 4c 03 05 c6 08 9f 73 49 83 78 10 00 4d 8b 10 0f 84 d5 00 00 00 4d 85 d2 0f 84 cc 00 00 00 49 63 41 20 48 8d 4a 01 49 8b 39 <49> 8b 1c 02 4c 89 d0 65 48 0f c7 0f 0f 94 c0 84 c0 74 bb 49 63 kernel: RIP: __kmalloc+0xbc/0x200 RSP: ffffb84c89263be0 ** Affects: linux (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1682418 Title: [zesty] mlx5 OVS vxlan ipv6 LNST test cause Oops To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1682418/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
