Hello Matt, Thank you again for the response.
So we were able to compile but now are unable to start VPP. We are seeing the following errors: I rebuilt, and got: May 02 20:20:24 demo001 vpp[2161]: vpp[2161]: perfmon: skipping source 'intel-uncore' - intel_uncore_init: no uncore units found May 02 20:20:24 demo001 vpp[2161]: perfmon: skipping source 'intel-uncore' - intel_uncore_init: no uncore units found May 02 20:20:24 demo001 vpp[2161]: vpp[2161]: tls_init_ca_chain:617: Could not initialize TLS CA certificates May 02 20:20:24 demo001 vpp[2161]: vpp[2161]: tls_mbedtls_init:652: failed to initialize TLS CA chain May 02 20:20:24 demo001 vpp[2161]: tls_init_ca_chain:617: Could not initialize TLS CA certificates May 02 20:20:24 demo001 vpp[2161]: tls_mbedtls_init:652: failed to initialize TLS CA chain May 02 20:20:24 demo001 vpp[2161]: vpp[2161]: tls_init_ca_chain:1083: Could not initialize TLS CA certificates May 02 20:20:24 demo001 vpp[2161]: vpp[2161]: tls_openssl_init:1206: failed to initialize TLS CA chain May 02 20:20:24 demo001 vpp[2161]: tls_init_ca_chain:1083: Could not initialize TLS CA certificates May 02 20:20:24 demo001 vpp[2161]: tls_openssl_init:1206: failed to initialize TLS CA chain May 02 20:20:24 demo001 vpp[2161]: vpp[2161]: vlib_vmbus_bind_to_uio: Skipping VMBUS device 000d3a8a-a89d-000d-3a8a-a89d000d3a8a as host interface eth0 is up May 02 20:20:24 demo001 vpp[2161]: vlib_vmbus_bind_to_uio: Skipping VMBUS device 000d3a8a-a89d-000d-3a8a-a89d000d3a8a as host interface eth0 is up May 02 20:20:24 demo001 vnet[2161]: dpdk: Interface Ethernet0/2/0 error -22: Unknown error -22 May 02 20:20:24 demo001 vnet[2161]: dpdk: Interface Ethernet0/2/0 error -22: Unknown error -22 May 02 20:20:24 demo001 vnet[2161]: dpdk: [0] setup failed Errors: rte_eth_tx_queue_setup[port:0, errno:-22]: Unknown error -22 rte_eth_rx_queue_setup[port:0, errno:-22]: Unknown error -22 May 02 20:20:25 demo001 vnet[2161]: received signal SIGSEGV, PC 0x7f4ba079c12c, faulting address 0xdd7 May 02 20:20:25 demo001 vnet[2161]: #0 0x00007f4ba12a69d7 0x7f4ba12a69d7 May 02 20:20:25 demo001 vnet[2161]: #1 0x00007f4ba0be0c20 0x7f4ba0be0c20 May 02 20:20:25 demo001 vnet[2161]: #2 0x00007f4ba079c12c 0x7f4ba079c12c May 02 20:20:25 demo001 vnet[2161]: #3 0x00007f4ba079dbfe va_format + 0xee May 02 20:20:25 demo001 vnet[2161]: #4 0x00007f4ba079fb04 format + 0x84 May 02 20:20:25 demo001 vnet[2161]: #5 0x00007f4ba079b88c 0x7f4ba079b88c May 02 20:20:25 demo001 vnet[2161]: #6 0x00007f4ba079dbfe va_format + 0xee May 02 20:20:25 demo001 vnet[2161]: #7 0x00007f4ba124a9f8 vlib_log + 0x2a8 May 02 20:20:25 demo001 vnet[2161]: #8 0x00007f4b5b160305 0x7f4b5b160305 May 02 20:20:25 demo001 vnet[2161]: #9 0x00007f4b5b170382 0x7f4b5b170382 May 02 20:20:25 demo001 vnet[2161]: #10 0x00007f4b5b1750a6 0x7f4b5b1750a6 May 02 20:20:25 demo001 vnet[2161]: #11 0x00007f4ba124bac6 0x7f4ba124bac6 May 02 20:20:25 demo001 vnet[2161]: #12 0x00007f4ba07b3e10 0x7f4ba07b3e10 May 02 20:20:25 demo001 vpp[2161]: Failed to save post-mortem API trace to /tmp/api_post_mortem.2161 May 02 20:20:25 demo001 systemd[1]: vpp.service: Main process exited, code=killed, status=6/ABRT May 02 20:20:25 demo001 systemd[1]: vpp.service: Failed with result 'signal'. -- Subject: Unit failed This time, vpp doesn’t even start up due to the above errors. *Is there a specific kernel version you used?* -- John M. On Mon, May 2, 2022 at 1:42 PM Matthew Smith <mgsm...@netgate.com> wrote: > > Sorry, there was a typo. You should use DPDK_MLX5_COMMON_PMD=y instead > of DPDK_MLX_COMMON_PMD=y. > > -Matt > > > On Mon, May 2, 2022 at 11:28 AM Johnny Martinez <john.marti...@gmail.com> > wrote: > >> Hello Matt! >> >> *Thank you very much for the reply!* >> >> Our end goal is to be using Alma Linux but we went ahead and tried it in >> a fresh Ubuntu 20.04 installation with the same result. >> >> It looks like it is unable to compile the *mlx5 drivers*. >> >> *I’ve attached the complete build log for your review.* >> >> >> >> *Build was started with:* >> >> *make DPDK_MLX4_PMD=y DPDK_MLX5_PMD=y DPDK_MLX_COMMON_PMD=y pkg-deb | tee >> /root/vpp-build.log* >> >> >> >> As an extra precaution, the vpp.mk content is: >> >> >> >> *MACHINE=$(shell uname -m)* >> >> *vpp_arch = native* >> >> *vpp_root_packages = vpp* >> >> *vpp_debug_TAG_BUILD_TYPE = debug* >> >> *vpp_TAG_BUILD_TYPE = release* >> >> *vpp_clang_TAG_BUILD_TYPE = release* >> >> *vpp_gcov_TAG_BUILD_TYPE = gcov* >> >> *vpp_coverity_TAG_BUILD_TYPE = coverity* >> >> *vpp_dpdk_inc_dir = /usr/include/dpdk/* >> >> *vpp_dpdk_lib_dir = /usr/lib64/* >> >> *vpp_uses_dpdk_mlx4_pmd = yes* >> >> *DPDK_MLX4_PMD=y* >> >> *DPDK_MLX5_PMD=y* >> >> *DPDK_MLX_COMMON_PMD=y* >> >> >> >> The build completes, but there’s an issue with the MLX5 driver. As seen >> in the build log, the *MLX4 is built, but MLX5 is not*. I’ve tried >> searching online, but there’s reference to *glue* with regard to the >> MLX5, but nothing actionable I could find: >> >> >> >> *Part 1: Shows that the library “mlx4” and “mlx5” are included as part of >> “make” (means our build options were correctly interperated):* >> >> >> >> *cmake3 --build >> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/build-rdma-core >> -- libccan.a libibverbs.a librdma_util.a libmlx5.a libmlx4.a > >> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/rdma-core.build.log* >> >> *sed 's/^Libs.private:.*/Libs.private: -lmlx4 -lmlx5 -libverbs >> -lrdma_util -lccan -lpthread/' -i >> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/build-rdma-core/lib/pkgconfig/libibverbs.pc >> >> >> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/rdma-core.build.log* >> >> >> *Part 2: Shows that MLX5 is missing “common_mlx5” (which I can’t find a >> way to install…?) so is disabled. MLX4 however, is built:* >> >> >> >> *Using 'PKG_CONFIG_PATH' from environment with value: >> '/root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/install-vpp-native/external/lib/pkgconfig'* >> >> *Run-time dependency libmlx4 found: YES 1.0.39.1* >> >> *Using 'PKG_CONFIG_PATH' from environment with value: >> '/root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/install-vpp-native/external/lib/pkgconfig'* >> >> *Run-time dependency libibverbs found: YES 1.14.39.1* >> >> *Compiler for C supports arguments -std=c11: YES* >> >> *Compiler for C supports arguments -Wno-strict-prototypes >> -Wstrict-prototypes: YES* >> >> *Compiler for C supports arguments -D_BSD_SOURCE: YES* >> >> *Compiler for C supports arguments -D_DEFAULT_SOURCE: YES* >> >> *Compiler for C supports arguments -D_XOPEN_SOURCE=600: YES* >> >> *Header <infiniband/mlx4dv.h> has symbol >> "MLX4DV_SET_CTX_ATTR_BUF_ALLOCATORS" with dependencies libmlx4, libibverbs: >> YES* >> >> *Header <infiniband/mlx4dv.h> has symbol "MLX4DV_QP_MASK_UAR_MMAP_OFFSET" >> with dependencies libmlx4, libibverbs: YES* >> >> *Checking whether type "struct mlx4_wqe_lso_seg" has member >> "mss_hdr_size" with dependencies libmlx4, libibverbs: YES* >> >> *Configuring mlx4_autoconf.h using configuration* >> >> *Compiler for C supports arguments -std=c11: YES* >> >> *Compiler for C supports arguments -Wno-strict-prototypes >> -Wstrict-prototypes: YES* >> >> *Compiler for C supports arguments -D_BSD_SOURCE: YES* >> >> *Compiler for C supports arguments -D_DEFAULT_SOURCE: YES* >> >> *Compiler for C supports arguments -D_XOPEN_SOURCE=600: YES* >> >> *Message: Disabling mlx5 [drivers/net/mlx5]: missing internal dependency >> "common_mlx5"* >> >> *Run-time dependency libmusdk found: NO (tried pkgconfig)* >> >> *Run-time dependency libmusdk found: NO (tried pkgconfig)* >> >> *Run-time dependency netcope-common found: NO (tried pkgconfig)* >> >> >> >> *Part 3: Shows multiple missing “internal dependency” issues:* >> >> >> >> *Message: Disabling dpaa [drivers/dma/dpaa]: missing internal dependency >> "bus_dpaa"* >> >> *Message: Disabling dpaa [drivers/net/dpaa]: missing internal dependency >> "mempool_dpaa"* >> >> *Message: Disabling kni [drivers/net/kni]: missing internal dependency >> "kni"* >> >> *Message: Disabling mlx5 [drivers/net/mlx5]: missing internal dependency >> "common_mlx5"* >> >> *Message: Disabling caam_jr [drivers/crypto/caam_jr]: missing internal >> dependency "bus_dpaa"* >> >> *Message: Disabling mlx5 [drivers/crypto/mlx5]: missing internal >> dependency "common_mlx5"* >> >> *Message: Disabling mlx5 [drivers/compress/mlx5]: missing internal >> dependency "common_mlx5"* >> >> *Message: Disabling mlx5 [drivers/regex/mlx5]: missing internal >> dependency "common_mlx5"* >> >> *Message: Disabling mlx5 [drivers/vdpa/mlx5]: missing internal dependency >> "common_mlx5"* >> >> >> *I am not able to find a way to enable MLX5.* >> >> -- >> John M. >> >> >> On Fri, Apr 29, 2022 at 4:31 PM Matthew Smith <mgsm...@netgate.com> >> wrote: >> >>> >>> Hi John, >>> >>> Are you using a stock build of VPP from packagecloud or did you build >>> your own? VPP's build of DPDK does not enable the Mellanox (mlx4, mlx5) >>> PMDs by default. If you didn't already, you would need to build VPP with >>> the environment variables DPDK_MLX4_PMD, DPDK_MLX5_PMD, and >>> DPDK_MLX_COMMON_PMD set to y. You could change the default values from n to >>> y in build/external/packages/dpdk.mk or you could run something like >>> 'make DPDK_MLX4_PMD=y DPDK_MLX5_PMD=y DPDK_MLX_COMMON_PMD=y pkg-deb'. That >>> presumes that you're running on ubuntu or some other debian-based distro, >>> if you're running some other OS maybe substituting pkg-snap or pkg-rpm for >>> the pkg-deb in that command would work for you. >>> >>> -Matt >>> >>> >>> On Fri, Apr 29, 2022 at 2:40 PM Johnny Martinez <john.marti...@gmail.com> >>> wrote: >>> >>>> Johnny Martinez <john.marti...@gmail.com> >>>> 9:57 AM (5 hours ago) >>>> to Benoit >>>> *@Ben *- thank you *VERY *much for the detailed response! >>>> >>>> I had *partial *success but almost there - now I am seeing some DPDK >>>> errors and other behavior outlined below. >>>> >>>> On my test device I have 3 interfaces: >>>> >>>> - Management interface (kernel interface, not managed by VPP) >>>> - vpp-wan interface >>>> - vpp-lan interface >>>> >>>> I configureD the interfaces with the following: >>>> >>>> set logging class linux-cp rate-limit 1000 level warn syslog-level >>>> notice >>>> >>>> >>>> lcp default netns dataplane >>>> >>>> lcp lcp-sync on >>>> >>>> lcp lcp-auto-subint on >>>> >>>> >>>> # vpp-wan >>>> >>>> vpp# lcp create NetVSC1 host-if vpp-wan >>>> >>>> vpp# set int state NetVSC1 up >>>> >>>> vpp# set int ip address NetVSC1 10.0.2.4/24 >>>> >>>> vpp# ip route add 0.0.0.0/0 via 10.0.2.1 >>>> >>>> >>>> # vpp-lan >>>> >>>> vpp# lcp create NetVSC0 host-if vpp-lan >>>> >>>> vpp# set int state NetVSC0 up >>>> >>>> vpp# set int ip address NetVSC0 10.0.1.4/24 >>>> >>>> >>>> I am now able to see the two interfaces in vpp *HOWEVER *I am not able >>>> to set the MTU size on either interface to 1500. >>>> >>>> vpp# set int mtu 1500 NetVSC0 >>>> set interface mtu: Unsupported (dpdk driver doesn't support MTU change) >>>> vpp# set int mtu 1500 NetVSC1 >>>> set interface mtu: Unsupported (dpdk driver doesn't support MTU change) >>>> >>>> >>>> I see the following in the log files using '*journalctl | grep dpdk*' >>>> - we get two seperate DPDK errors. *One seems to be interface specific >>>> and the other when we try to set the MTU size*. >>>> >>>> The *orange *error is every second - whereas the MTU error is only >>>> when we try to set the MTU size. >>>> >>>> *Apr 29 12:55:15 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add(): >>>> RNDIS reports VF but device not found, retrying* >>>> >>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach(): >>>> Couldn't find port for VF* >>>> >>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add(): >>>> RNDIS reports VF but device not found, retrying* >>>> >>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach(): >>>> Couldn't find port for VF* >>>> >>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add(): >>>> RNDIS reports VF but device not found, retrying* >>>> >>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: [0] >>>> rte_eth_dev_set_mtu failed (mtu 1496, rv -95)* >>>> >>>> *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: set interface mtu: >>>> Unsupported (dpdk driver doesn't support MTU change* >>>> >>>> *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach(): >>>> Couldn't find port for VF* >>>> >>>> *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add(): >>>> RNDIS reports VF but device not found, retrying* >>>> >>>> *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach(): >>>> Couldn't find port for VF* >>>> >>>> *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add(): >>>> RNDIS reports VF but device not found, retrying* >>>> >>>> >>>> Anyone have any recommendation on how to resolve this? >>>> >>>> *I have also tried setting dev XXX and vdev commands in the dpdk{} >>>> block, but this causes the virtual interfaces not to be found (as you have >>>> previously mentioned).* >>>> >>>> *Thank you all in advance for your assistance!* >>>> >>>> -- >>>> John M >>>> >>>> >>>> On Fri, Apr 29, 2022 at 2:40 PM Matthew Smith <mgsm...@netgate.com> >>>> wrote: >>>> >>>>> Hi Ben, >>>>> >>>>> Sure, I'll work on an update to the doc. >>>>> >>>>> -Matt >>>>> >>>>> >>>>> On Fri, Apr 29, 2022 at 3:48 AM Benoit Ganne (bganne) < >>>>> bga...@cisco.com> wrote: >>>>> >>>>>> Hi Matt, John, >>>>>> >>>>>> This is a great summary! >>>>>> >>>>>> >> The only reference I could find for VPP in Azure was in this link >>>>>> >> but it is very outdated: >>>>>> >> https://fd.io/docs/vpp/v2101/usecases/vppinazure.html >>>>>> >>>>>> > Yes, that doc is way outdated. I advise against following the >>>>>> instructions >>>>>> > in it. >>>>>> >>>>>> Matt or John, would you mind proposing a patch for that? Even a quick >>>>>> & dirty "here are the steps you need to follow" would be great. >>>>>> >>>>>> Best >>>>>> ben >>>>>> >>>>>
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#21330): https://lists.fd.io/g/vpp-dev/message/21330 Mute This Topic: https://lists.fd.io/mt/90759128/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-