Re: [vpp-dev] Issue with DPDK 19.08 / i40e driver
Hi, I'm pushing another patch to enlarge the stack to 18 since the stack size is still not enough for some complex cli process, e.g. ‘test flow’... Best Regards, Sun, Chenmin From: vpp-dev@lists.fd.io [mailto:vpp-dev@lists.fd.io] On Behalf Of Aloys Augustin (aloaugus) via Lists.Fd.Io Sent: Tuesday, September 17, 2019 4:16 PM To: Sun, Chenmin Cc: vpp-dev@lists.fd.io Subject: Re: [vpp-dev] Issue with DPDK 19.08 / i40e driver Hi Chenmin, Thanks a lot for digging into this. I found an old issue, VPP-133: https://jira.fd.io/browse/VPP-133, that looks very similar to this bug. Increasing the stack size allocated to CLI processes resolves the issue, so it looks like the inlining increased DPDK’s stack usage. I have pushed a patch to gerrit that fixes this bug: https://gerrit.fd.io/r/c/vpp/+/22074 . Best, Aloÿs On 16 Sep 2019, at 08:12, Sun, Chenmin mailto:chenmin@intel.com>> wrote: Hi, I've bisected and found this issue is imported by DPDK commit: commit 1f4d55be438b428bed74f2e3dc49cfd6efc3e6fd Author: Maxime Coquelin mailto:maxime.coque...@redhat.com>> Date: Wed May 29 15:04:20 2019 +0200 eal/x86: force inlining of all memcpy and mov helpers Some helpers in the header file are forced inlined other are only inlined, this patch forces inline for all. It will avoid it to be embedded as functions when called multiple times in the same object file. For example, when we added packed ring support in vhost-user library, rte_memcpy_generic got no more inlined. Signed-off-by: Maxime Coquelin mailto:maxime.coque...@redhat.com>> Acked-by: Bruce Richardson mailto:bruce.richard...@intel.com>> The segment fault disappeared after I reverted this commit. I also tried with testpmd, which is a DPDK app integrated in DPDK code, and didn't see this issue. That DPDK commit just changed the function declaration from inlie to __rte_always_inline, all seems ok but why brings such a strange issue to VPP(debug mode only). - Best Regards, Sun, Chenmin From: Yang, Zhiyong Sent: Monday, September 16, 2019 2:08 PM To: Sun, Chenmin mailto:chenmin@intel.com>> Subject: FW: [vpp-dev] Issue with DPDK 19.08 / i40e driver From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> mailto:vpp-dev@lists.fd.io>> On Behalf Of Damjan Marion via Lists.Fd.Io Sent: Thursday, September 12, 2019 9:00 PM To: Aloys Augustin (aloaugus) mailto:aloau...@cisco.com>> Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> Subject: Re: [vpp-dev] Issue with DPDK 19.08 / i40e driver Florin hit the same issue few weeks ago, don't use debug version of DPDK and it will work OK. it is unlikely vpp issue, as it happens only when dpdk is compiled with -g0. It looks like stack corruption in dpdk. If somebody have cycles, probably bisecting the dpdk between last two releases will give some clue what is wrong. On 12 Sep 2019, at 13:55, Aloys Augustin (aloaugus) via Lists.Fd.Io mailto:aloaugus=cisco@lists.fd.io>> wrote: Hi Damjan, I encountered the same issue on a different host with the same hardware, here is a stack trace: DBGvpp# sh int Name IdxState MTU (L3/IP4/IP6/MPLS) Counter Count FortyGigabitEthernet5e/0/01 down 9000/0/0/0 local00 down 0/0/0/0 DBGvpp# set int state FortyGigabitEthernet5e/0/0 up Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault. 0x7fffb07b9f07 in i40e_asq_send_command (hw=0x7fc1fff9d340, desc=0x7fffb9880cc0, buff=0x7fc2002c3b80, buff_size=16, cmd_details=0x0) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/base/i40e_adminq.c:933 933 wr32(hw, hw->aq.asq.tail, hw->aq.asq.next_to_use); (gdb) bt #0 0x7fffb07b9f07 in i40e_asq_send_command (hw=0x7fc1fff9d340, desc=0x7fffb9880cc0, buff=0x7fc2002c3b80, buff_size=16, cmd_details=0x0) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/base/i40e_adminq.c:933 #1 0x7fffb0812e1c in i40e_aq_remove_macvlan (hw=0x7fc1fff9d340, seid=390, mv_list=0x7fc2002c3b80, count=1, cmd_details=0x0) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/base/i40e_common.c:3121 #2 0x7fffb0992a10 in i40e_remove_macvlan_filters (vsi=0x7fc1fff1aac0, filter=0x7fc1fff1c280, total=1) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/i40e_ethdev.c:6881 #3 0x7fffb09bd4b8 in i40e_vsi_delete_mac (vsi=0x7fc1fff1aac0, addr=0x7fffb98856c0) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/i40e_ethdev.c:7319 #4 0x7fffb0a2172b in i40e_set_default_mac_addr (dev=0x7fffb3069d00 , mac_addr=0x7fc1fff1a500) at /home/aloaugus/vpp/build-root/build-
Re: [vpp-dev] Issue with DPDK 19.08 / i40e driver
Hi Aloÿs Thanks for fixing this issue so fast. ☺ - Best Regards, Sun, Chenmin From: vpp-dev@lists.fd.io [mailto:vpp-dev@lists.fd.io] On Behalf Of Aloys Augustin (aloaugus) via Lists.Fd.Io Sent: Tuesday, September 17, 2019 4:16 PM To: Sun, Chenmin Cc: vpp-dev@lists.fd.io Subject: Re: [vpp-dev] Issue with DPDK 19.08 / i40e driver Hi Chenmin, Thanks a lot for digging into this. I found an old issue, VPP-133: https://jira.fd.io/browse/VPP-133, that looks very similar to this bug. Increasing the stack size allocated to CLI processes resolves the issue, so it looks like the inlining increased DPDK’s stack usage. I have pushed a patch to gerrit that fixes this bug: https://gerrit.fd.io/r/c/vpp/+/22074 . Best, Aloÿs On 16 Sep 2019, at 08:12, Sun, Chenmin mailto:chenmin@intel.com>> wrote: Hi, I've bisected and found this issue is imported by DPDK commit: commit 1f4d55be438b428bed74f2e3dc49cfd6efc3e6fd Author: Maxime Coquelin mailto:maxime.coque...@redhat.com>> Date: Wed May 29 15:04:20 2019 +0200 eal/x86: force inlining of all memcpy and mov helpers Some helpers in the header file are forced inlined other are only inlined, this patch forces inline for all. It will avoid it to be embedded as functions when called multiple times in the same object file. For example, when we added packed ring support in vhost-user library, rte_memcpy_generic got no more inlined. Signed-off-by: Maxime Coquelin mailto:maxime.coque...@redhat.com>> Acked-by: Bruce Richardson mailto:bruce.richard...@intel.com>> The segment fault disappeared after I reverted this commit. I also tried with testpmd, which is a DPDK app integrated in DPDK code, and didn't see this issue. That DPDK commit just changed the function declaration from inlie to __rte_always_inline, all seems ok but why brings such a strange issue to VPP(debug mode only). - Best Regards, Sun, Chenmin From: Yang, Zhiyong Sent: Monday, September 16, 2019 2:08 PM To: Sun, Chenmin mailto:chenmin@intel.com>> Subject: FW: [vpp-dev] Issue with DPDK 19.08 / i40e driver From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> mailto:vpp-dev@lists.fd.io>> On Behalf Of Damjan Marion via Lists.Fd.Io Sent: Thursday, September 12, 2019 9:00 PM To: Aloys Augustin (aloaugus) mailto:aloau...@cisco.com>> Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> Subject: Re: [vpp-dev] Issue with DPDK 19.08 / i40e driver Florin hit the same issue few weeks ago, don't use debug version of DPDK and it will work OK. it is unlikely vpp issue, as it happens only when dpdk is compiled with -g0. It looks like stack corruption in dpdk. If somebody have cycles, probably bisecting the dpdk between last two releases will give some clue what is wrong. On 12 Sep 2019, at 13:55, Aloys Augustin (aloaugus) via Lists.Fd.Io mailto:aloaugus=cisco@lists.fd.io>> wrote: Hi Damjan, I encountered the same issue on a different host with the same hardware, here is a stack trace: DBGvpp# sh int Name IdxState MTU (L3/IP4/IP6/MPLS) Counter Count FortyGigabitEthernet5e/0/01 down 9000/0/0/0 local00 down 0/0/0/0 DBGvpp# set int state FortyGigabitEthernet5e/0/0 up Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault. 0x7fffb07b9f07 in i40e_asq_send_command (hw=0x7fc1fff9d340, desc=0x7fffb9880cc0, buff=0x7fc2002c3b80, buff_size=16, cmd_details=0x0) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/base/i40e_adminq.c:933 933 wr32(hw, hw->aq.asq.tail, hw->aq.asq.next_to_use); (gdb) bt #0 0x7fffb07b9f07 in i40e_asq_send_command (hw=0x7fc1fff9d340, desc=0x7fffb9880cc0, buff=0x7fc2002c3b80, buff_size=16, cmd_details=0x0) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/base/i40e_adminq.c:933 #1 0x7fffb0812e1c in i40e_aq_remove_macvlan (hw=0x7fc1fff9d340, seid=390, mv_list=0x7fc2002c3b80, count=1, cmd_details=0x0) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/base/i40e_common.c:3121 #2 0x7fffb0992a10 in i40e_remove_macvlan_filters (vsi=0x7fc1fff1aac0, filter=0x7fc1fff1c280, total=1) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/i40e_ethdev.c:6881 #3 0x7fffb09bd4b8 in i40e_vsi_delete_mac (vsi=0x7fc1fff1aac0, addr=0x7fffb98856c0) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/i40e_ethdev.c:7319 #4 0x7fffb0a2172b in i40e_set_default_mac_addr (dev=0x7fffb3069d00 , mac_addr=0x7fc1fff1a500) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/i40e_ethdev.c
Re: [vpp-dev] Issue with DPDK 19.08 / i40e driver
Hi Chenmin, Thanks a lot for digging into this. I found an old issue, VPP-133: https://jira.fd.io/browse/VPP-133 <https://jira.fd.io/browse/VPP-133>, that looks very similar to this bug. Increasing the stack size allocated to CLI processes resolves the issue, so it looks like the inlining increased DPDK’s stack usage. I have pushed a patch to gerrit that fixes this bug: https://gerrit.fd.io/r/c/vpp/+/22074 <https://gerrit.fd.io/r/c/vpp/+/22074> . Best, Aloÿs > On 16 Sep 2019, at 08:12, Sun, Chenmin wrote: > > Hi, <> > > I've bisected and found this issue is imported by DPDK commit: > > commit 1f4d55be438b428bed74f2e3dc49cfd6efc3e6fd > Author: Maxime Coquelin <mailto:maxime.coque...@redhat.com>> > Date: Wed May 29 15:04:20 2019 +0200 > > eal/x86: force inlining of all memcpy and mov helpers > > Some helpers in the header file are forced inlined other are > only inlined, this patch forces inline for all. > > It will avoid it to be embedded as functions when called multiple > times in the same object file. For example, when we added packed > ring support in vhost-user library, rte_memcpy_generic got no > more inlined. > > Signed-off-by: Maxime Coquelin <mailto:maxime.coque...@redhat.com>> > Acked-by: Bruce Richardson <mailto:bruce.richard...@intel.com>> > > The segment fault disappeared after I reverted this commit. > I also tried with testpmd, which is a DPDK app integrated in DPDK code, and > didn't see this issue. > > That DPDK commit just changed the function declaration from inlie to > __rte_always_inline, all seems ok but why brings such a strange issue to > VPP(debug mode only). > > - > Best Regards, > Sun, Chenmin > > From: Yang, Zhiyong > Sent: Monday, September 16, 2019 2:08 PM > To: Sun, Chenmin > Subject: FW: [vpp-dev] Issue with DPDK 19.08 / i40e driver > > > > <>From: vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io> > mailto:vpp-dev@lists.fd.io>> On Behalf Of Damjan Marion > via Lists.Fd.Io > Sent: Thursday, September 12, 2019 9:00 PM > To: Aloys Augustin (aloaugus) mailto:aloau...@cisco.com>> > Cc: vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io> > Subject: Re: [vpp-dev] Issue with DPDK 19.08 / i40e driver > > > Florin hit the same issue few weeks ago, don't use debug version of DPDK and > it will work OK. > it is unlikely vpp issue, as it happens only when dpdk is compiled with -g0. > It looks like stack corruption in dpdk. If somebody have cycles, probably > bisecting the dpdk > between last two releases will give some clue what is wrong. > > > On 12 Sep 2019, at 13:55, Aloys Augustin (aloaugus) via Lists.Fd.Io > mailto:aloaugus=cisco@lists.fd.io>> > wrote: > > Hi Damjan, > > I encountered the same issue on a different host with the same hardware, here > is a stack trace: > > DBGvpp# sh int > Name IdxState MTU (L3/IP4/IP6/MPLS) > Counter Count > FortyGigabitEthernet5e/0/01 down 9000/0/0/0 > local00 down 0/0/0/0 > DBGvpp# set int state FortyGigabitEthernet5e/0/0 up > > Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault. > 0x7fffb07b9f07 in i40e_asq_send_command (hw=0x7fc1fff9d340, > desc=0x7fffb9880cc0, buff=0x7fc2002c3b80, buff_size=16, cmd_details=0x0) > at > /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/base/i40e_adminq.c:933 > 933 wr32(hw, hw->aq.asq.tail, > hw->aq.asq.next_to_use); > (gdb) bt > #0 0x7fffb07b9f07 in i40e_asq_send_command (hw=0x7fc1fff9d340, > desc=0x7fffb9880cc0, buff=0x7fc2002c3b80, buff_size=16, cmd_details=0x0) > at > /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/base/i40e_adminq.c:933 > #1 0x7fffb0812e1c in i40e_aq_remove_macvlan (hw=0x7fc1fff9d340, > seid=390, mv_list=0x7fc2002c3b80, count=1, cmd_details=0x0) > at > /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/base/i40e_common.c:3121 > #2 0x7fffb0992a10 in i40e_remove_macvlan_filters (vsi=0x7fc1fff1aac0, > filter=0x7fc1fff1c280, total=1) at > /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/i40e_ethdev.c:6881 > #3 0x7fffb09bd4b8 in i40e_vsi_delete_mac (vsi=0x7fc1fff1aac0, > addr=0x7fffb98856c0) at > /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/i40e_ethdev.c:7319 > #4 0x7
Re: [vpp-dev] Issue with DPDK 19.08 / i40e driver
Hi, I've bisected and found this issue is imported by DPDK commit: commit 1f4d55be438b428bed74f2e3dc49cfd6efc3e6fd Author: Maxime Coquelin Date: Wed May 29 15:04:20 2019 +0200 eal/x86: force inlining of all memcpy and mov helpers Some helpers in the header file are forced inlined other are only inlined, this patch forces inline for all. It will avoid it to be embedded as functions when called multiple times in the same object file. For example, when we added packed ring support in vhost-user library, rte_memcpy_generic got no more inlined. Signed-off-by: Maxime Coquelin Acked-by: Bruce Richardson The segment fault disappeared after I reverted this commit. I also tried with testpmd, which is a DPDK app integrated in DPDK code, and didn't see this issue. That DPDK commit just changed the function declaration from inlie to __rte_always_inline, all seems ok but why brings such a strange issue to VPP(debug mode only). - Best Regards, Sun, Chenmin From: Yang, Zhiyong Sent: Monday, September 16, 2019 2:08 PM To: Sun, Chenmin Subject: FW: [vpp-dev] Issue with DPDK 19.08 / i40e driver From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> mailto:vpp-dev@lists.fd.io>> On Behalf Of Damjan Marion via Lists.Fd.Io Sent: Thursday, September 12, 2019 9:00 PM To: Aloys Augustin (aloaugus) mailto:aloau...@cisco.com>> Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> Subject: Re: [vpp-dev] Issue with DPDK 19.08 / i40e driver Florin hit the same issue few weeks ago, don't use debug version of DPDK and it will work OK. it is unlikely vpp issue, as it happens only when dpdk is compiled with -g0. It looks like stack corruption in dpdk. If somebody have cycles, probably bisecting the dpdk between last two releases will give some clue what is wrong. On 12 Sep 2019, at 13:55, Aloys Augustin (aloaugus) via Lists.Fd.Io mailto:aloaugus=cisco@lists.fd.io>> wrote: Hi Damjan, I encountered the same issue on a different host with the same hardware, here is a stack trace: DBGvpp# sh int Name IdxState MTU (L3/IP4/IP6/MPLS) Counter Count FortyGigabitEthernet5e/0/01 down 9000/0/0/0 local00 down 0/0/0/0 DBGvpp# set int state FortyGigabitEthernet5e/0/0 up Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault. 0x7fffb07b9f07 in i40e_asq_send_command (hw=0x7fc1fff9d340, desc=0x7fffb9880cc0, buff=0x7fc2002c3b80, buff_size=16, cmd_details=0x0) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/base/i40e_adminq.c:933 933 wr32(hw, hw->aq.asq.tail, hw->aq.asq.next_to_use); (gdb) bt #0 0x7fffb07b9f07 in i40e_asq_send_command (hw=0x7fc1fff9d340, desc=0x7fffb9880cc0, buff=0x7fc2002c3b80, buff_size=16, cmd_details=0x0) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/base/i40e_adminq.c:933 #1 0x7fffb0812e1c in i40e_aq_remove_macvlan (hw=0x7fc1fff9d340, seid=390, mv_list=0x7fc2002c3b80, count=1, cmd_details=0x0) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/base/i40e_common.c:3121 #2 0x7fffb0992a10 in i40e_remove_macvlan_filters (vsi=0x7fc1fff1aac0, filter=0x7fc1fff1c280, total=1) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/i40e_ethdev.c:6881 #3 0x7fffb09bd4b8 in i40e_vsi_delete_mac (vsi=0x7fc1fff1aac0, addr=0x7fffb98856c0) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/i40e_ethdev.c:7319 #4 0x7fffb0a2172b in i40e_set_default_mac_addr (dev=0x7fffb3069d00 , mac_addr=0x7fc1fff1a500) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/i40e_ethdev.c:11962 #5 0x7fffaffb3491 in rte_eth_dev_mac_restore (dev=0x7fffb3069d00 , dev_info=0x7fffb9885770) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/lib/librte_ethdev/rte_ethdev.c:1357 #6 0x7fffaffb35d5 in rte_eth_dev_config_restore (dev=0x7fffb3069d00 , dev_info=0x7fffb9885770, port_id=0) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/lib/librte_ethdev/rte_ethdev.c:1388 #7 0x7fffaffb37e9 in rte_eth_dev_start (port_id=0) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/lib/librte_ethdev/rte_ethdev.c:1435 #8 0x7fffb2732604 in dpdk_device_start (xd=0x7fffb9842740) at /home/aloaugus/vpp/src/plugins/dpdk/device/common.c:168 #9 0x7fffb274b25d in dpdk_interface_admin_up_down (vnm=0x77b64520 , hw_if_index=1, flags=1) at /home/aloaugus/vpp/src/plugins/dpdk/device/device.c:483 #10 0x76c71bb4 in vnet_sw_interface_set_flags_helper (vnm=0x77b64520 , sw_if_index=1, flags
Re: [vpp-dev] Issue with DPDK 19.08 / i40e driver
Florin hit the same issue few weeks ago, don't use debug version of DPDK and it will work OK. it is unlikely vpp issue, as it happens only when dpdk is compiled with -g0. It looks like stack corruption in dpdk. If somebody have cycles, probably bisecting the dpdk between last two releases will give some clue what is wrong. > On 12 Sep 2019, at 13:55, Aloys Augustin (aloaugus) via Lists.Fd.Io > wrote: > > Hi Damjan, > > I encountered the same issue on a different host with the same hardware, here > is a stack trace: > > DBGvpp# sh int > Name IdxState MTU (L3/IP4/IP6/MPLS) > Counter Count > FortyGigabitEthernet5e/0/01 down 9000/0/0/0 > local00 down 0/0/0/0 > DBGvpp# set int state FortyGigabitEthernet5e/0/0 up > > Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault. > 0x7fffb07b9f07 in i40e_asq_send_command (hw=0x7fc1fff9d340, > desc=0x7fffb9880cc0, buff=0x7fc2002c3b80, buff_size=16, cmd_details=0x0) > at > /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/base/i40e_adminq.c:933 > 933 wr32(hw, hw->aq.asq.tail, hw->aq.asq.next_to_use); > (gdb) bt > #0 0x7fffb07b9f07 in i40e_asq_send_command (hw=0x7fc1fff9d340, > desc=0x7fffb9880cc0, buff=0x7fc2002c3b80, buff_size=16, cmd_details=0x0) > at > /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/base/i40e_adminq.c:933 > #1 0x7fffb0812e1c in i40e_aq_remove_macvlan (hw=0x7fc1fff9d340, > seid=390, mv_list=0x7fc2002c3b80, count=1, cmd_details=0x0) > at > /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/base/i40e_common.c:3121 > #2 0x7fffb0992a10 in i40e_remove_macvlan_filters (vsi=0x7fc1fff1aac0, > filter=0x7fc1fff1c280, total=1) at > /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/i40e_ethdev.c:6881 > #3 0x7fffb09bd4b8 in i40e_vsi_delete_mac (vsi=0x7fc1fff1aac0, > addr=0x7fffb98856c0) at > /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/i40e_ethdev.c:7319 > #4 0x7fffb0a2172b in i40e_set_default_mac_addr (dev=0x7fffb3069d00 > , mac_addr=0x7fc1fff1a500) at > /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/i40e_ethdev.c:11962 > #5 0x7fffaffb3491 in rte_eth_dev_mac_restore (dev=0x7fffb3069d00 > , dev_info=0x7fffb9885770) at > /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/lib/librte_ethdev/rte_ethdev.c:1357 > #6 0x7fffaffb35d5 in rte_eth_dev_config_restore (dev=0x7fffb3069d00 > , dev_info=0x7fffb9885770, port_id=0) > at > /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/lib/librte_ethdev/rte_ethdev.c:1388 > #7 0x7fffaffb37e9 in rte_eth_dev_start (port_id=0) at > /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/lib/librte_ethdev/rte_ethdev.c:1435 > #8 0x7fffb2732604 in dpdk_device_start (xd=0x7fffb9842740) at > /home/aloaugus/vpp/src/plugins/dpdk/device/common.c:168 > #9 0x7fffb274b25d in dpdk_interface_admin_up_down (vnm=0x77b64520 > , hw_if_index=1, flags=1) at > /home/aloaugus/vpp/src/plugins/dpdk/device/device.c:483 > #10 0x76c71bb4 in vnet_sw_interface_set_flags_helper > (vnm=0x77b64520 , sw_if_index=1, > flags=VNET_SW_INTERFACE_FLAG_ADMIN_UP, helper_flags=(unknown: 0)) at > /home/aloaugus/vpp/src/vnet/interface.c:455 > #11 0x76c71cde in vnet_sw_interface_set_flags (vnm=0x77b64520 > , sw_if_index=1, flags=VNET_SW_INTERFACE_FLAG_ADMIN_UP) at > /home/aloaugus/vpp/src/vnet/interface.c:504 > #12 0x76c8d3b1 in set_state (vm=0x766b5dc0 , > input=0x7fffb9885f00, cmd=0x7fffb9534e40) at > /home/aloaugus/vpp/src/vnet/interface_cli.c:902 > #13 0x763c7085 in vlib_cli_dispatch_sub_commands (vm=0x766b5dc0 > , cm=0x766b5fd0 , > input=0x7fffb9885f00, parent_command_index=66) at > /home/aloaugus/vpp/src/vlib/cli.c:645 > #14 0x763c6f1a in vlib_cli_dispatch_sub_commands (vm=0x766b5dc0 > , cm=0x766b5fd0 , > input=0x7fffb9885f00, parent_command_index=36) at > /home/aloaugus/vpp/src/vlib/cli.c:606 > #15 0x763c6f1a in vlib_cli_dispatch_sub_commands (vm=0x766b5dc0 > , cm=0x766b5fd0 , > input=0x7fffb9885f00, parent_command_index=0) at > /home/aloaugus/vpp/src/vlib/cli.c:606 > #16 0x763c74b0 in vlib_cli_input (vm=0x766b5dc0 > , input=0x7fffb9885f00, function=0x7646d6de > , function_arg=0) at > /home/aloaugus/vpp/src/vlib/cli.c:746 > #17 0x76473514 in unix_cli_process_input (cm=0x766b67a0 > , cli_file_index=0) at > /home/aloaugus/vpp/src/vlib/unix/cli.c:2525 > #18 0x76474086 in unix_cli_process (vm=0x766b5dc0 > , rt=0x7fffb9875000, f=0x0) at > /home/aloaugus/vpp/src/vlib/unix/cli.c:2641 > #19
Re: [vpp-dev] Issue with DPDK 19.08 / i40e driver
Hi Damjan, I encountered the same issue on a different host with the same hardware, here is a stack trace: DBGvpp# sh int Name IdxState MTU (L3/IP4/IP6/MPLS) Counter Count FortyGigabitEthernet5e/0/01 down 9000/0/0/0 local00 down 0/0/0/0 DBGvpp# set int state FortyGigabitEthernet5e/0/0 up Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault. 0x7fffb07b9f07 in i40e_asq_send_command (hw=0x7fc1fff9d340, desc=0x7fffb9880cc0, buff=0x7fc2002c3b80, buff_size=16, cmd_details=0x0) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/base/i40e_adminq.c:933 933 wr32(hw, hw->aq.asq.tail, hw->aq.asq.next_to_use); (gdb) bt #0 0x7fffb07b9f07 in i40e_asq_send_command (hw=0x7fc1fff9d340, desc=0x7fffb9880cc0, buff=0x7fc2002c3b80, buff_size=16, cmd_details=0x0) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/base/i40e_adminq.c:933 #1 0x7fffb0812e1c in i40e_aq_remove_macvlan (hw=0x7fc1fff9d340, seid=390, mv_list=0x7fc2002c3b80, count=1, cmd_details=0x0) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/base/i40e_common.c:3121 #2 0x7fffb0992a10 in i40e_remove_macvlan_filters (vsi=0x7fc1fff1aac0, filter=0x7fc1fff1c280, total=1) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/i40e_ethdev.c:6881 #3 0x7fffb09bd4b8 in i40e_vsi_delete_mac (vsi=0x7fc1fff1aac0, addr=0x7fffb98856c0) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/i40e_ethdev.c:7319 #4 0x7fffb0a2172b in i40e_set_default_mac_addr (dev=0x7fffb3069d00 , mac_addr=0x7fc1fff1a500) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/drivers/net/i40e/i40e_ethdev.c:11962 #5 0x7fffaffb3491 in rte_eth_dev_mac_restore (dev=0x7fffb3069d00 , dev_info=0x7fffb9885770) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/lib/librte_ethdev/rte_ethdev.c:1357 #6 0x7fffaffb35d5 in rte_eth_dev_config_restore (dev=0x7fffb3069d00 , dev_info=0x7fffb9885770, port_id=0) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/lib/librte_ethdev/rte_ethdev.c:1388 #7 0x7fffaffb37e9 in rte_eth_dev_start (port_id=0) at /home/aloaugus/vpp/build-root/build-vpp_debug-native/external/dpdk-19.08/lib/librte_ethdev/rte_ethdev.c:1435 #8 0x7fffb2732604 in dpdk_device_start (xd=0x7fffb9842740) at /home/aloaugus/vpp/src/plugins/dpdk/device/common.c:168 #9 0x7fffb274b25d in dpdk_interface_admin_up_down (vnm=0x77b64520 , hw_if_index=1, flags=1) at /home/aloaugus/vpp/src/plugins/dpdk/device/device.c:483 #10 0x76c71bb4 in vnet_sw_interface_set_flags_helper (vnm=0x77b64520 , sw_if_index=1, flags=VNET_SW_INTERFACE_FLAG_ADMIN_UP, helper_flags=(unknown: 0)) at /home/aloaugus/vpp/src/vnet/interface.c:455 #11 0x76c71cde in vnet_sw_interface_set_flags (vnm=0x77b64520 , sw_if_index=1, flags=VNET_SW_INTERFACE_FLAG_ADMIN_UP) at /home/aloaugus/vpp/src/vnet/interface.c:504 #12 0x76c8d3b1 in set_state (vm=0x766b5dc0 , input=0x7fffb9885f00, cmd=0x7fffb9534e40) at /home/aloaugus/vpp/src/vnet/interface_cli.c:902 #13 0x763c7085 in vlib_cli_dispatch_sub_commands (vm=0x766b5dc0 , cm=0x766b5fd0 , input=0x7fffb9885f00, parent_command_index=66) at /home/aloaugus/vpp/src/vlib/cli.c:645 #14 0x763c6f1a in vlib_cli_dispatch_sub_commands (vm=0x766b5dc0 , cm=0x766b5fd0 , input=0x7fffb9885f00, parent_command_index=36) at /home/aloaugus/vpp/src/vlib/cli.c:606 #15 0x763c6f1a in vlib_cli_dispatch_sub_commands (vm=0x766b5dc0 , cm=0x766b5fd0 , input=0x7fffb9885f00, parent_command_index=0) at /home/aloaugus/vpp/src/vlib/cli.c:606 #16 0x763c74b0 in vlib_cli_input (vm=0x766b5dc0 , input=0x7fffb9885f00, function=0x7646d6de , function_arg=0) at /home/aloaugus/vpp/src/vlib/cli.c:746 #17 0x76473514 in unix_cli_process_input (cm=0x766b67a0 , cli_file_index=0) at /home/aloaugus/vpp/src/vlib/unix/cli.c:2525 #18 0x76474086 in unix_cli_process (vm=0x766b5dc0 , rt=0x7fffb9875000, f=0x0) at /home/aloaugus/vpp/src/vlib/unix/cli.c:2641 #19 0x764148e4 in vlib_process_bootstrap (_a=140736284985712) at /home/aloaugus/vpp/src/vlib/main.c:1468 #20 0x75eb7a48 in clib_calljmp () from /home/aloaugus/vpp/build-root/install-vpp_debug-native/vpp/lib/libvppinfra.so.20.01 #21 0x7fffb8460940 in ?? () #22 0x764149ec in vlib_process_startup (vm=0x766b5dc0 , p=0x259, f=0x0) at /home/aloaugus/vpp/src/vlib/main.c:1490 Backtrace stopped: previous frame inner to this frame (corrupt stack?) Reverting commit b6103105f99e0c7f210a9596f205a1efd21b626f (update of dpdk to 19.08) fixes the issue.
Re: [vpp-dev] Issue with DPDK 19.08 / i40e driver
> On 11 Sep 2019, at 16:43, Mathias Raoul wrote: > > Hello, > > I have an issue with VPP and i40e driver, when I try to switch the interface > to up, the program stop with a segmentation fault. My configuration details > are below. > > It might be a compatibility issue, because the DPDK documentation recommend > using the firmware v7 for i40E with DPDK v19.08. But the firmware is not yet > available for the Cisco XL710 card. > vpp stop in this file : dpdk-19.08/drivers/net/i40/base/i40e_adminq.c:933 > When I change dpdk version to 19.05 the bug disappear. > > DBGvpp# show int > Name IdxState MTU (L3/IP4/IP6/MPLS) > Counter Count > FortyGigabitEthernet5e/0/01 down 9000/0/0/0 > local00 down 0/0/0/0 > DBGvpp# set interface state FortyGigabitEthernet5e/0/0 up > vl_msg_api_trace_save:252: Message table length 44998 > > > Configuration: > VPP : last commit on master : 1146ff4bcd336d8efc19405f1d83914e6115a01f > > show version verbose > Version: v20.01-rc0~171-g1146ff4bc > Compiled by: root > Compile host: 524b94e75c4d > Compile date: Wed Sep 11 12:42:53 UTC 2019 > Compile location: /home/mraoul/dev/vpp > Compiler: GCC 7.4.0 > Current PID: 19052 > > OS : Ubuntu 18.04.2 LTS > > Network card : Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ > (rev 02)\ There is no enough information in your email to make any conclusion. Can you try to run VPP under gdb and grab traceback, preferably with debug image? -- Damjan -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#13964): https://lists.fd.io/g/vpp-dev/message/13964 Mute This Topic: https://lists.fd.io/mt/34104216/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
