[vpp-dev] #vpp #dpdk Flow creation ends up with message: "test flow: flow error: "not supported""

[anton.belousov via lists.fd.io]

Hi,
I have an aws instance m5n.xlarge, where I'm trying to setup vpp with multiple 
flows to utilize RSS. VPP version is 20.01.
I'm using igb_uio.ko driver from http://dpdk.org/git/dpdk-kmods with 
wc_activate=1 option. I'm using following configuration:

*unix {*
*nodaemon*
*log /var/log/vpp/vpp.log*
*full-coredump*
*cli-listen /run/vpp/cli-vpp.sock*
*}*
*cpu {*
*workers 3*
*scheduler-policy fifo*
*}*
*api-trace {*
*on*
*}*
*dpdk {*
*dev default {*
*num-rx-queues 3*
*num-tx-queues 3*
*num-rx-desc 1024*
*num-tx-desc 1024*
*}*
*dev :00:06.0*
*dev :00:07.0*
*uio-driver igb_uio*
*}

* Command show hardware-interfaces returns following output:
*vpp# show hardware-interfaces*
*Name    Idx   Link  Hardware*
*VirtualFunctionEthernet0/6/0   1 up   VirtualFunctionEthernet0/6/0*
*Link speed: unknown*
*Ethernet address 06:7b:e2:10:2d:08*
*AWS ENA VF*
*carrier up full duplex mtu 1558*
*flags: admin-up pmd maybe-multiseg rx-ip4-cksum*
*Devargs:*
*rx: queues 3 (max 8), desc 1024 (min 128 max 8192 align 1)*
*tx: queues 3 (max 8), desc 1024 (min 128 max 1024 align 1)*
*pci: device 1d0f:ec20 subsystem 1d0f:ec20 address :00:06.00 numa 0*
*max rx packet len: 9216*
*promiscuous: unicast off all-multicast off*
*vlan offload: strip off filter off qinq off*
*rx offload avail:  ipv4-cksum udp-cksum tcp-cksum jumbo-frame*
*rx offload active: ipv4-cksum jumbo-frame*
*tx offload avail:  ipv4-cksum udp-cksum tcp-cksum*
*tx offload active: none*
*rss avail: ipv4-frag ipv4-tcp ipv4-udp ipv4-other ipv4 ipv6-tcp-ex*
*ipv6-udp-ex ipv6-frag ipv6-tcp ipv6-udp ipv6-other*
*ipv6-ex ipv6*
*rss active:    none*
*tx burst function: (nil)*
*rx burst function: (nil)*

*tx frames ok   7*
*tx bytes ok  294*
*rx frames ok   7*
*rx bytes ok  294*
*extended stats:*
*rx good packets  7*
*tx good packets  7*
*rx good bytes  294*
*tx good bytes  294*
*rx q0packets 7*
*rx q0bytes 294*
*tx q0packets 7*
*tx q0bytes 294*
*dev start    1*
*rx q0 cnt    7*
*rx q0 bytes    294*
*tx q0 cnt    7*
*tx q0 bytes    294*
*tx q0 tx poll    7*
*tx q0 doorbells  7*
*tx q0 available desc  1021*
*tx q1 available desc  1023*
*tx q2 available desc  1023*
*VirtualFunctionEthernet0/7/0   2 up   VirtualFunctionEthernet0/7/0*
*Link speed: unknown*
*Ethernet address 06:46:0a:ba:20:94*
*AWS ENA VF*
*carrier up full duplex mtu 1558*
*flags: admin-up pmd maybe-multiseg rx-ip4-cksum*
*Devargs:*
*rx: queues 3 (max 8), desc 1024 (min 128 max 8192 align 1)*
*tx: queues 3 (max 8), desc 1024 (min 128 max 1024 align 1)*
*pci: device 1d0f:ec20 subsystem 1d0f:ec20 address :00:07.00 numa 0*
*max rx packet len: 9216*
*promiscuous: unicast off all-multicast off*
*vlan offload: strip off filter off qinq off*
*rx offload avail:  ipv4-cksum udp-cksum tcp-cksum jumbo-frame*
*rx offload active: ipv4-cksum jumbo-frame*
*tx offload avail:  ipv4-cksum udp-cksum tcp-cksum*
*tx offload active: none*
*rss avail: ipv4-frag ipv4-tcp ipv4-udp ipv4-other ipv4 ipv6-tcp-ex*
*ipv6-udp-ex ipv6-frag ipv6-tcp ipv6-udp ipv6-other*
*ipv6-ex ipv6*
*rss active:    none*
*tx burst function: (nil)*
*rx burst function: (nil)*

*tx frames ok   6*
*tx bytes ok  252*
*rx frames ok   6*
*rx bytes ok  252*
*extended stats:*
*rx good packets  6

* Then I'm creating some flow with following command:
test flow add src-ip 10.0.10.103 dst-ip 10.0.10.100 proto udp src-port 2152 
redirect-to-queue 1 The flow is created, but when I try to enable it error 
occurs: *
* test flow enable index 0 VirtualFunctionEthernet0/6/0
test flow: flow error: "not supported" From vpp source code I can see that it 
is line 112 of flow.c file
if ( dev_class -> flow_ops_function == 0 )
return VNET_FLOW_ERROR_NOT_SUPPORTED;

Does it mean that NIC isn't supporting flows or RSS? Maybe I have 
misconfiguration somewhere? If you need some additional info please ask and I 
will provide it.
Any advice would be appreciated. Thank you in advance!

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent 

Re: [vpp-dev] How to enable Mellanox compilation in VPP 21.01

[Mohammed Hawari]

Hi Julius,

Thanks for finding that issue on master, could you verify that this patch fixes 
it? https://gerrit.fd.io/r/c/vpp/+/32042 

Best regards,

Mohammed

> On 19 Apr 2021, at 09:08, Július Milan  wrote:
> 
> Hi Mohammed
> 
> Thanks a lot, I confirm that it worked well on branch stable/2101.
> 
> However on master (ae34872077c956aa34ee816b55ccb5c5f6ab40a1), I got the 
> following error:
> $ make run
> /home/pantheontech/dev/vpp_master/build-root/install-vpp_debug-native/vpp/bin/vpp:
>  symbol lookup error: 
> /home/pantheontech/dev/vpp_master/build-root/install-vpp_debug-native/vpp/lib/vpp_plugins/dpdk_plugin.so:
>  undefined symbol: ibv_fork_init
> 
> Following the same procedure as for stable/2101, compiling from the clean 
> workspace, having the only changes:
> $ git diff
> diff --git a/build/external/packages/dpdk.mk  
> b/build/external/packages/dpdk.mk 
> index c7eb0fc3f..f1663a1aa 100644
> --- a/build/external/packages/dpdk.mk 
> +++ b/build/external/packages/dpdk.mk 
> @@ -14,9 +14,9 @@
>  DPDK_PKTMBUF_HEADROOM?= 128
>  DPDK_USE_LIBBSD  ?= n
>  DPDK_DEBUG   ?= n
> -DPDK_MLX4_PMD?= n
> -DPDK_MLX5_PMD?= n
> -DPDK_MLX5_COMMON_PMD ?= n
> +DPDK_MLX4_PMD:= y
> +DPDK_MLX5_PMD:= y
> +DPDK_MLX5_COMMON_PMD := y
>  DPDK_TAP_PMD ?= n
>  DPDK_FAILSAFE_PMD?= n
>  DPDK_MACHINE ?= default
> 
> Best regards
> Julius
> 
> On Fri, 16 Apr 2021 at 17:27, Mohammed Hawari  > wrote:
> Hi Julius,
> 
> You don’t need any patch for Ubuntu it should work out of the box on 21.01… 
> This patch is CentOS specific because the default libdir on CentOS is ‘lib64' 
> (while it is ‘lib' on Ubuntu/Debian). On Ubuntu, you only need to edit 
> dpdk.mk  to enable MLX4_PMD, MLX5_PMD and MLX5_COMMON_PMD. 
> Internally it works like this:
> - rdma-core (and the libibverbs *static* library) is built and installed in 
> some internal staging directory (build-root/install-vpp-native/external/lib/).
> - DPDK is configured to be statically linked against this libibverb statis 
> library, and is also statically built.
> - then the VPP dpdk plugin is statically linked against DPDK.
> 
> The patch fixes the first step, as on CentOS, cmake will install the 
> rdma-core libraries in build-root/install-vpp-native/external/lib64 while the 
> DPDK build system will look for them in 
> build-root/install-vpp-native/external/lib. Note that this patch is not 
> “specific” to CentOS, but is *required* for CentOS support (and is merged in 
> master).
> 
> Best regards,
> Mohammed
> 
>> On 16 Apr 2021, at 16:55, Július Milan > > wrote:
>> 
>> Hi Mohammed
>> 
>> I am trying to create the patch similar to yours: 
>> https://gerrit.fd.io/r/c/vpp/+/31876 
>> But being able to run on Ubuntu.
>> Can you point me a bit? I would like to understand how your patch is CentOS 
>> specific.
>> Probably it has something to do with linking of libibverbs.so into 
>> dpdk_plugin, which I was not able to achieve yet.
>> Or am I completely wrong?
>> 
>> Best regards
>> Julius
>> 
>> On Fri, 16 Apr 2021 at 09:49, Mohammed Hawari > > wrote:
>> 
>> Hi Chetan,
>> 
>>> On 16 Apr 2021, at 07:04, chetan bhasin >> > wrote:
>>> 
>>> Hi Mohammed ,
>>> 
>>> After applying the patch you have mentioned , I am no longer facing the 
>>> compilation issue.
>>> 
>>> Thanks for your support. 
>>> 
>>> With Static RDMA linking , do we need to do anything extra while bringing 
>>> our application on the server having Mellanox Nic ?
>> 
>> No we don’t, it should work out of the box!
>>> 
>>> Thanks & Regards,
>>> Chetan
>> 
>> Regards
>> 
>> Mohammed
>>> 
>>> On Wed, Apr 14, 2021 at 8:33 PM chetan bhasin via lists.fd.io 
>>>  >> > wrote:
>>> Thanks a lot Mohammed!
>>> 
>>> Let me try with the rdma-core and patch mentioned by you and 
>>> 
>>> Thanks,
>>> Chetan
>>> 
>>> On Wed, Apr 14, 2021 at 6:20 PM Mohammed Hawari >> > wrote:
>>> Hi Chetan
>>> 
>>> You are building DPDK with Mellanox support relying on dlopen and the glue 
>>> library which is fine. The default approach implemented in the VPP build 
>>> system consists of statically linking DPDK with the the libraries provided 
>>> by rdma-core (as it’s also built by the VPP build system). The default 
>>> approach should work fine, provided this patch 
>>> https://gerrit.fd.io/r/c/vpp/+/31876  
>>> is applied (specific to CentOS). But what you should work just fine. Now 
>>> the reason why  you need to comment dpdk_depends is kind of mysterious here…
>>> 
>>> Regards,
>>> 
>>> Mohammed
 O