Hello Matt,
Can you *please *confirm which DPDK and VPP versions you have that are
working in Azure?
I am hoping to get a base build with no errors and the ability to set MTU
size...
*Thank you in advance for your help!!!*
--
John M.
On Mon, May 2, 2022 at 1:42 PM Matthew Smith <mgsm...@netgate.com> wrote:
>
> Sorry, there was a typo. You should use DPDK_MLX5_COMMON_PMD=y instead
> of DPDK_MLX_COMMON_PMD=y.
>
> -Matt
>
>
> On Mon, May 2, 2022 at 11:28 AM Johnny Martinez <john.marti...@gmail.com>
> wrote:
>
>> Hello Matt!
>>
>> *Thank you very much for the reply!*
>>
>> Our end goal is to be using Alma Linux but we went ahead and tried it in
>> a fresh Ubuntu 20.04 installation with the same result.
>>
>> It looks like it is unable to compile the *mlx5 drivers*.
>>
>> *I’ve attached the complete build log for your review.*
>>
>>
>>
>> *Build was started with:*
>>
>> *make DPDK_MLX4_PMD=y DPDK_MLX5_PMD=y DPDK_MLX_COMMON_PMD=y pkg-deb | tee
>> /root/vpp-build.log*
>>
>>
>>
>> As an extra precaution, the vpp.mk content is:
>>
>>
>>
>> *MACHINE=$(shell uname -m)*
>>
>> *vpp_arch = native*
>>
>> *vpp_root_packages = vpp*
>>
>> *vpp_debug_TAG_BUILD_TYPE = debug*
>>
>> *vpp_TAG_BUILD_TYPE = release*
>>
>> *vpp_clang_TAG_BUILD_TYPE = release*
>>
>> *vpp_gcov_TAG_BUILD_TYPE = gcov*
>>
>> *vpp_coverity_TAG_BUILD_TYPE = coverity*
>>
>> *vpp_dpdk_inc_dir = /usr/include/dpdk/*
>>
>> *vpp_dpdk_lib_dir = /usr/lib64/*
>>
>> *vpp_uses_dpdk_mlx4_pmd = yes*
>>
>> *DPDK_MLX4_PMD=y*
>>
>> *DPDK_MLX5_PMD=y*
>>
>> *DPDK_MLX_COMMON_PMD=y*
>>
>>
>>
>> The build completes, but there’s an issue with the MLX5 driver. As seen
>> in the build log, the *MLX4 is built, but MLX5 is not*. I’ve tried
>> searching online, but there’s reference to *glue* with regard to the
>> MLX5, but nothing actionable I could find:
>>
>>
>>
>> *Part 1: Shows that the library “mlx4” and “mlx5” are included as part of
>> “make” (means our build options were correctly interperated):*
>>
>>
>>
>> *cmake3 --build
>> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/build-rdma-core
>> -- libccan.a libibverbs.a librdma_util.a libmlx5.a libmlx4.a >
>> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/rdma-core.build.log*
>>
>> *sed 's/^Libs.private:.*/Libs.private: -lmlx4 -lmlx5 -libverbs
>> -lrdma_util -lccan -lpthread/' -i
>> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/build-rdma-core/lib/pkgconfig/libibverbs.pc
>> >>
>> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/rdma-core.build.log*
>>
>>
>> *Part 2: Shows that MLX5 is missing “common_mlx5” (which I can’t find a
>> way to install…?) so is disabled. MLX4 however, is built:*
>>
>>
>>
>> *Using 'PKG_CONFIG_PATH' from environment with value:
>> '/root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/install-vpp-native/external/lib/pkgconfig'*
>>
>> *Run-time dependency libmlx4 found: YES 1.0.39.1*
>>
>> *Using 'PKG_CONFIG_PATH' from environment with value:
>> '/root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/install-vpp-native/external/lib/pkgconfig'*
>>
>> *Run-time dependency libibverbs found: YES 1.14.39.1*
>>
>> *Compiler for C supports arguments -std=c11: YES*
>>
>> *Compiler for C supports arguments -Wno-strict-prototypes
>> -Wstrict-prototypes: YES*
>>
>> *Compiler for C supports arguments -D_BSD_SOURCE: YES*
>>
>> *Compiler for C supports arguments -D_DEFAULT_SOURCE: YES*
>>
>> *Compiler for C supports arguments -D_XOPEN_SOURCE=600: YES*
>>
>> *Header <infiniband/mlx4dv.h> has symbol
>> "MLX4DV_SET_CTX_ATTR_BUF_ALLOCATORS" with dependencies libmlx4, libibverbs:
>> YES*
>>
>> *Header <infiniband/mlx4dv.h> has symbol "MLX4DV_QP_MASK_UAR_MMAP_OFFSET"
>> with dependencies libmlx4, libibverbs: YES*
>>
>> *Checking whether type "struct mlx4_wqe_lso_seg" has member
>> "mss_hdr_size" with dependencies libmlx4, libibverbs: YES*
>>
>> *Configuring mlx4_autoconf.h using configuration*
>>
>> *Compiler for C supports arguments -std=c11: YES*
>>
>> *Compiler for C supports arguments -Wno-strict-prototypes
>> -Wstrict-prototypes: YES*
>>
>> *Compiler for C supports arguments -D_BSD_SOURCE: YES*
>>
>> *Compiler for C supports arguments -D_DEFAULT_SOURCE: YES*
>>
>> *Compiler for C supports arguments -D_XOPEN_SOURCE=600: YES*
>>
>> *Message: Disabling mlx5 [drivers/net/mlx5]: missing internal dependency
>> "common_mlx5"*
>>
>> *Run-time dependency libmusdk found: NO (tried pkgconfig)*
>>
>> *Run-time dependency libmusdk found: NO (tried pkgconfig)*
>>
>> *Run-time dependency netcope-common found: NO (tried pkgconfig)*
>>
>>
>>
>> *Part 3: Shows multiple missing “internal dependency” issues:*
>>
>>
>>
>> *Message: Disabling dpaa [drivers/dma/dpaa]: missing internal dependency
>> "bus_dpaa"*
>>
>> *Message: Disabling dpaa [drivers/net/dpaa]: missing internal dependency
>> "mempool_dpaa"*
>>
>> *Message: Disabling kni [drivers/net/kni]: missing internal dependency
>> "kni"*
>>
>> *Message: Disabling mlx5 [drivers/net/mlx5]: missing internal dependency
>> "common_mlx5"*
>>
>> *Message: Disabling caam_jr [drivers/crypto/caam_jr]: missing internal
>> dependency "bus_dpaa"*
>>
>> *Message: Disabling mlx5 [drivers/crypto/mlx5]: missing internal
>> dependency "common_mlx5"*
>>
>> *Message: Disabling mlx5 [drivers/compress/mlx5]: missing internal
>> dependency "common_mlx5"*
>>
>> *Message: Disabling mlx5 [drivers/regex/mlx5]: missing internal
>> dependency "common_mlx5"*
>>
>> *Message: Disabling mlx5 [drivers/vdpa/mlx5]: missing internal dependency
>> "common_mlx5"*
>>
>>
>> *I am not able to find a way to enable MLX5.*
>>
>> --
>> John M.
>>
>>
>> On Fri, Apr 29, 2022 at 4:31 PM Matthew Smith <mgsm...@netgate.com>
>> wrote:
>>
>>>
>>> Hi John,
>>>
>>> Are you using a stock build of VPP from packagecloud or did you build
>>> your own? VPP's build of DPDK does not enable the Mellanox (mlx4, mlx5)
>>> PMDs by default. If you didn't already, you would need to build VPP with
>>> the environment variables DPDK_MLX4_PMD, DPDK_MLX5_PMD, and
>>> DPDK_MLX_COMMON_PMD set to y. You could change the default values from n to
>>> y in build/external/packages/dpdk.mk or you could run something like
>>> 'make DPDK_MLX4_PMD=y DPDK_MLX5_PMD=y DPDK_MLX_COMMON_PMD=y pkg-deb'. That
>>> presumes that you're running on ubuntu or some other debian-based distro,
>>> if you're running some other OS maybe substituting pkg-snap or pkg-rpm for
>>> the pkg-deb in that command would work for you.
>>>
>>> -Matt
>>>
>>>
>>> On Fri, Apr 29, 2022 at 2:40 PM Johnny Martinez <john.marti...@gmail.com>
>>> wrote:
>>>
>>>> Johnny Martinez <john.marti...@gmail.com>
>>>> 9:57 AM (5 hours ago)
>>>> to Benoit
>>>> *@Ben *- thank you *VERY *much for the detailed response!
>>>>
>>>> I had *partial *success but almost there - now I am seeing some DPDK
>>>> errors and other behavior outlined below.
>>>>
>>>> On my test device I have 3 interfaces:
>>>>
>>>> - Management interface (kernel interface, not managed by VPP)
>>>> - vpp-wan interface
>>>> - vpp-lan interface
>>>>
>>>> I configureD the interfaces with the following:
>>>>
>>>> set logging class linux-cp rate-limit 1000 level warn syslog-level
>>>> notice
>>>>
>>>>
>>>> lcp default netns dataplane
>>>>
>>>> lcp lcp-sync on
>>>>
>>>> lcp lcp-auto-subint on
>>>>
>>>>
>>>> # vpp-wan
>>>>
>>>> vpp# lcp create NetVSC1 host-if vpp-wan
>>>>
>>>> vpp# set int state NetVSC1 up
>>>>
>>>> vpp# set int ip address NetVSC1 10.0.2.4/24
>>>>
>>>> vpp# ip route add 0.0.0.0/0 via 10.0.2.1
>>>>
>>>>
>>>> # vpp-lan
>>>>
>>>> vpp# lcp create NetVSC0 host-if vpp-lan
>>>>
>>>> vpp# set int state NetVSC0 up
>>>>
>>>> vpp# set int ip address NetVSC0 10.0.1.4/24
>>>>
>>>>
>>>> I am now able to see the two interfaces in vpp *HOWEVER *I am not able
>>>> to set the MTU size on either interface to 1500.
>>>>
>>>> vpp# set int mtu 1500 NetVSC0
>>>> set interface mtu: Unsupported (dpdk driver doesn't support MTU change)
>>>> vpp# set int mtu 1500 NetVSC1
>>>> set interface mtu: Unsupported (dpdk driver doesn't support MTU change)
>>>>
>>>>
>>>> I see the following in the log files using '*journalctl | grep dpdk*'
>>>> - we get two seperate DPDK errors. *One seems to be interface specific
>>>> and the other when we try to set the MTU size*.
>>>>
>>>> The *orange *error is every second - whereas the MTU error is only
>>>> when we try to set the MTU size.
>>>>
>>>> *Apr 29 12:55:15 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add():
>>>> RNDIS reports VF but device not found, retrying*
>>>>
>>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach():
>>>> Couldn't find port for VF*
>>>>
>>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add():
>>>> RNDIS reports VF but device not found, retrying*
>>>>
>>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach():
>>>> Couldn't find port for VF*
>>>>
>>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add():
>>>> RNDIS reports VF but device not found, retrying*
>>>>
>>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: [0]
>>>> rte_eth_dev_set_mtu failed (mtu 1496, rv -95)*
>>>>
>>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: set interface mtu:
>>>> Unsupported (dpdk driver doesn't support MTU change*
>>>>
>>>> *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach():
>>>> Couldn't find port for VF*
>>>>
>>>> *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add():
>>>> RNDIS reports VF but device not found, retrying*
>>>>
>>>> *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach():
>>>> Couldn't find port for VF*
>>>>
>>>> *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add():
>>>> RNDIS reports VF but device not found, retrying*
>>>>
>>>>
>>>> Anyone have any recommendation on how to resolve this?
>>>>
>>>> *I have also tried setting dev XXX and vdev commands in the dpdk{}
>>>> block, but this causes the virtual interfaces not to be found (as you have
>>>> previously mentioned).*
>>>>
>>>> *Thank you all in advance for your assistance!*
>>>>
>>>> --
>>>> John M
>>>>
>>>>
>>>> On Fri, Apr 29, 2022 at 2:40 PM Matthew Smith <mgsm...@netgate.com>
>>>> wrote:
>>>>
>>>>> Hi Ben,
>>>>>
>>>>> Sure, I'll work on an update to the doc.
>>>>>
>>>>> -Matt
>>>>>
>>>>>
>>>>> On Fri, Apr 29, 2022 at 3:48 AM Benoit Ganne (bganne) <
>>>>> bga...@cisco.com> wrote:
>>>>>
>>>>>> Hi Matt, John,
>>>>>>
>>>>>> This is a great summary!
>>>>>>
>>>>>> >> The only reference I could find for VPP in Azure was in this link
>>>>>> >> but it is very outdated:
>>>>>> >> https://fd.io/docs/vpp/v2101/usecases/vppinazure.html
>>>>>>
>>>>>> > Yes, that doc is way outdated. I advise against following the
>>>>>> instructions
>>>>>> > in it.
>>>>>>
>>>>>> Matt or John, would you mind proposing a patch for that? Even a quick
>>>>>> & dirty "here are the steps you need to follow" would be great.
>>>>>>
>>>>>> Best
>>>>>> ben
>>>>>>
>>>>>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21417): https://lists.fd.io/g/vpp-dev/message/21417
Mute This Topic: https://lists.fd.io/mt/90759128/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
