Re: [vpp-dev] Query on VPP in Aure
Hi John, I have had some problems on Azure with my build lately similar to the MTU errors you mentioned. The build that has problems is based on the commit b0f0f8c8d ("memif: fix memif_process_desc indexing") from the upstream master branch (so..master branch from 1 or 2 months ago using DPDK 21.11, not a stable release version of VPP). Last week I applied these recent patches to my build: 739a342b4 dpdk: fix overflow in mtu arithmetic b6bf5c9c4 dpdk: clear the RTE_MEMPOOL_F_NON_IO With those patches applied, the problems may have gone away. Netgate's QA team is still trying to verify if they can reproduce the issue with the patches applied. It was intermittent to begin with, so its not straightforward to declare that the issue is resolved. The build in our previous release, which did not have any issues with setting MTUs on Azure netvsc interfaces, was based on commit 895def45c ("avf: fix RSS hash key") from the upstream master branch. The DPDK version used there was 21.08. -Matt On Wed, May 18, 2022 at 12:20 PM Johnny Martinez wrote: > Hello Matt, > > Can you *please *confirm which DPDK and VPP versions you have that are > working in Azure? > > I am hoping to get a base build with no errors and the ability to set MTU > size... > > *Thank you in advance for your help!!!* > > -- > John M. > > > On Mon, May 2, 2022 at 1:42 PM Matthew Smith wrote: > >> >> Sorry, there was a typo. You should use DPDK_MLX5_COMMON_PMD=y instead >> of DPDK_MLX_COMMON_PMD=y. >> >> -Matt >> >> >> On Mon, May 2, 2022 at 11:28 AM Johnny Martinez >> wrote: >> >>> Hello Matt! >>> >>> *Thank you very much for the reply!* >>> >>> Our end goal is to be using Alma Linux but we went ahead and tried it in >>> a fresh Ubuntu 20.04 installation with the same result. >>> >>> It looks like it is unable to compile the *mlx5 drivers*. >>> >>> *I’ve attached the complete build log for your review.* >>> >>> >>> >>> *Build was started with:* >>> >>> *make DPDK_MLX4_PMD=y DPDK_MLX5_PMD=y DPDK_MLX_COMMON_PMD=y pkg-deb | >>> tee /root/vpp-build.log* >>> >>> >>> >>> As an extra precaution, the vpp.mk content is: >>> >>> >>> >>> *MACHINE=$(shell uname -m)* >>> >>> *vpp_arch = native* >>> >>> *vpp_root_packages = vpp* >>> >>> *vpp_debug_TAG_BUILD_TYPE = debug* >>> >>> *vpp_TAG_BUILD_TYPE = release* >>> >>> *vpp_clang_TAG_BUILD_TYPE = release* >>> >>> *vpp_gcov_TAG_BUILD_TYPE = gcov* >>> >>> *vpp_coverity_TAG_BUILD_TYPE = coverity* >>> >>> *vpp_dpdk_inc_dir = /usr/include/dpdk/* >>> >>> *vpp_dpdk_lib_dir = /usr/lib64/* >>> >>> *vpp_uses_dpdk_mlx4_pmd = yes* >>> >>> *DPDK_MLX4_PMD=y* >>> >>> *DPDK_MLX5_PMD=y* >>> >>> *DPDK_MLX_COMMON_PMD=y* >>> >>> >>> >>> The build completes, but there’s an issue with the MLX5 driver. As seen >>> in the build log, the *MLX4 is built, but MLX5 is not*. I’ve tried >>> searching online, but there’s reference to *glue* with regard to the >>> MLX5, but nothing actionable I could find: >>> >>> >>> >>> *Part 1: Shows that the library “mlx4” and “mlx5” are included as part >>> of “make” (means our build options were correctly interperated):* >>> >>> >>> >>> *cmake3 --build >>> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/build-rdma-core >>> -- libccan.a libibverbs.a librdma_util.a libmlx5.a libmlx4.a > >>> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/rdma-core.build.log* >>> >>> *sed 's/^Libs.private:.*/Libs.private: -lmlx4 -lmlx5 -libverbs >>> -lrdma_util -lccan -lpthread/' -i >>> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/build-rdma-core/lib/pkgconfig/libibverbs.pc >>> >> >>> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/rdma-core.build.log* >>> >>> >>> *Part 2: Shows that MLX5 is missing “common_mlx5” (which I can’t find a >>> way to install…?) so is disabled. MLX4 however, is built:* >>> >>> >>> >>> *Using 'PKG_CONFIG_PATH' from environment with value: >>> '/root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/install-vpp-native/external/lib/pkgconfig'* >>> >>> *Run-time dependency libmlx4 found: YES 1.0.39.1* >>> >>> *Using 'PKG_CONFIG_PATH' from environment with value: >>> '/root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/install-vpp-native/external/lib/pkgconfig'* >>> >>> *Run-time dependency libibverbs found: YES 1.14.39.1* >>> >>> *Compiler for C supports arguments -std=c11: YES* >>> >>> *Compiler for C supports arguments -Wno-strict-prototypes >>> -Wstrict-prototypes: YES* >>> >>> *Compiler for C supports arguments -D_BSD_SOURCE: YES* >>> >>> *Compiler for C supports arguments -D_DEFAULT_SOURCE: YES* >>> >>> *Compiler for C supports arguments -D_XOPEN_SOURCE=600: YES* >>> >>> *Header has symbol >>> "MLX4DV_SET_CTX_ATTR_BUF_ALLOCATORS" with dependencies libmlx4, libibverbs: >>> YES* >>> >>> *Header has symbol >>> "MLX4DV_QP_MASK_UAR_MMAP_OFFSET" with dependencies libmlx4, libibverbs: YES* >>> >>> *Checking whether type "struct mlx4_wqe_lso_seg" has
Re: [vpp-dev] Query on VPP in Aure
Hello Matt, Can you *please *confirm which DPDK and VPP versions you have that are working in Azure? I am hoping to get a base build with no errors and the ability to set MTU size... *Thank you in advance for your help!!!* -- John M. On Mon, May 2, 2022 at 1:42 PM Matthew Smith wrote: > > Sorry, there was a typo. You should use DPDK_MLX5_COMMON_PMD=y instead > of DPDK_MLX_COMMON_PMD=y. > > -Matt > > > On Mon, May 2, 2022 at 11:28 AM Johnny Martinez > wrote: > >> Hello Matt! >> >> *Thank you very much for the reply!* >> >> Our end goal is to be using Alma Linux but we went ahead and tried it in >> a fresh Ubuntu 20.04 installation with the same result. >> >> It looks like it is unable to compile the *mlx5 drivers*. >> >> *I’ve attached the complete build log for your review.* >> >> >> >> *Build was started with:* >> >> *make DPDK_MLX4_PMD=y DPDK_MLX5_PMD=y DPDK_MLX_COMMON_PMD=y pkg-deb | tee >> /root/vpp-build.log* >> >> >> >> As an extra precaution, the vpp.mk content is: >> >> >> >> *MACHINE=$(shell uname -m)* >> >> *vpp_arch = native* >> >> *vpp_root_packages = vpp* >> >> *vpp_debug_TAG_BUILD_TYPE = debug* >> >> *vpp_TAG_BUILD_TYPE = release* >> >> *vpp_clang_TAG_BUILD_TYPE = release* >> >> *vpp_gcov_TAG_BUILD_TYPE = gcov* >> >> *vpp_coverity_TAG_BUILD_TYPE = coverity* >> >> *vpp_dpdk_inc_dir = /usr/include/dpdk/* >> >> *vpp_dpdk_lib_dir = /usr/lib64/* >> >> *vpp_uses_dpdk_mlx4_pmd = yes* >> >> *DPDK_MLX4_PMD=y* >> >> *DPDK_MLX5_PMD=y* >> >> *DPDK_MLX_COMMON_PMD=y* >> >> >> >> The build completes, but there’s an issue with the MLX5 driver. As seen >> in the build log, the *MLX4 is built, but MLX5 is not*. I’ve tried >> searching online, but there’s reference to *glue* with regard to the >> MLX5, but nothing actionable I could find: >> >> >> >> *Part 1: Shows that the library “mlx4” and “mlx5” are included as part of >> “make” (means our build options were correctly interperated):* >> >> >> >> *cmake3 --build >> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/build-rdma-core >> -- libccan.a libibverbs.a librdma_util.a libmlx5.a libmlx4.a > >> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/rdma-core.build.log* >> >> *sed 's/^Libs.private:.*/Libs.private: -lmlx4 -lmlx5 -libverbs >> -lrdma_util -lccan -lpthread/' -i >> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/build-rdma-core/lib/pkgconfig/libibverbs.pc >> >> >> /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/rdma-core.build.log* >> >> >> *Part 2: Shows that MLX5 is missing “common_mlx5” (which I can’t find a >> way to install…?) so is disabled. MLX4 however, is built:* >> >> >> >> *Using 'PKG_CONFIG_PATH' from environment with value: >> '/root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/install-vpp-native/external/lib/pkgconfig'* >> >> *Run-time dependency libmlx4 found: YES 1.0.39.1* >> >> *Using 'PKG_CONFIG_PATH' from environment with value: >> '/root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/install-vpp-native/external/lib/pkgconfig'* >> >> *Run-time dependency libibverbs found: YES 1.14.39.1* >> >> *Compiler for C supports arguments -std=c11: YES* >> >> *Compiler for C supports arguments -Wno-strict-prototypes >> -Wstrict-prototypes: YES* >> >> *Compiler for C supports arguments -D_BSD_SOURCE: YES* >> >> *Compiler for C supports arguments -D_DEFAULT_SOURCE: YES* >> >> *Compiler for C supports arguments -D_XOPEN_SOURCE=600: YES* >> >> *Header has symbol >> "MLX4DV_SET_CTX_ATTR_BUF_ALLOCATORS" with dependencies libmlx4, libibverbs: >> YES* >> >> *Header has symbol "MLX4DV_QP_MASK_UAR_MMAP_OFFSET" >> with dependencies libmlx4, libibverbs: YES* >> >> *Checking whether type "struct mlx4_wqe_lso_seg" has member >> "mss_hdr_size" with dependencies libmlx4, libibverbs: YES* >> >> *Configuring mlx4_autoconf.h using configuration* >> >> *Compiler for C supports arguments -std=c11: YES* >> >> *Compiler for C supports arguments -Wno-strict-prototypes >> -Wstrict-prototypes: YES* >> >> *Compiler for C supports arguments -D_BSD_SOURCE: YES* >> >> *Compiler for C supports arguments -D_DEFAULT_SOURCE: YES* >> >> *Compiler for C supports arguments -D_XOPEN_SOURCE=600: YES* >> >> *Message: Disabling mlx5 [drivers/net/mlx5]: missing internal dependency >> "common_mlx5"* >> >> *Run-time dependency libmusdk found: NO (tried pkgconfig)* >> >> *Run-time dependency libmusdk found: NO (tried pkgconfig)* >> >> *Run-time dependency netcope-common found: NO (tried pkgconfig)* >> >> >> >> *Part 3: Shows multiple missing “internal dependency” issues:* >> >> >> >> *Message: Disabling dpaa [drivers/dma/dpaa]: missing internal dependency >> "bus_dpaa"* >> >> *Message: Disabling dpaa [drivers/net/dpaa]: missing internal dependency >> "mempool_dpaa"* >> >> *Message: Disabling kni [drivers/net/kni]: missing internal dependency >> "kni"* >> >> *Message: Disabling mlx5 [drivers/net/mlx5]: missing internal dependency >>
Re: [vpp-dev] Query on VPP in Aure
Hello Matt, *Here is some more detail that may be useful:* *Loaded modules (loaded by doing `* *modprobe uio_hv_generic`):* *[root@demo001 vpp]# lsmod | grep uio* *uio_pci_generic16384 0* *uio_hv_generic 16384 12* *hv_vmbus 143360 9 hv_balloon,hv_utils,uio_hv_generic,hv_netvsc,hid_hyperv,hv_storvsc,hyperv_keyboard,pci_hyperv,hyperv_fb* *uio20480 27 uio_hv_generic,cnic,uio_pci_generic* *PCI Devices:* *[root@demo001 vpp]# lspci | grep Ethernet* *2ab4:00:02.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] (rev 80)* *8213:00:02.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] (rev 80)* *e2ca:00:02.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] (rev 80)* *Hardware:* *[root@demo001 vpp]# lshw -class network -businfo* *Bus info Device Class Description* ** *pci@2ab4:00:02.0 eth5networkMT27710 Family [ConnectX-4 Lx Virtual Function]* *pci@8213:00:02.0 eth3networkMT27710 Family [ConnectX-4 Lx Virtual Function]* *pci@e2ca:00:02.0 eth4networkMT27710 Family [ConnectX-4 Lx Virtual Function]* * eth0networkEthernet interface* *Kernel: (We’ve tried with 4.18, 4.20, 5.11 and 5.16)* *NOTE: kernel 4.20 is end of life* *[root@demo001 vpp]# uname -a* *Linux demo001 5.16.13-1.el8.elrepo.x86_64 #1 SMP PREEMPT Tue Mar 8 08:33:30 EST 2022 x86_64 x86_64 x86_64 GNU/Linux* *Interface states:* *[root@demo001 ~]# ip addr | grep "state"* *1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000* *2: eth0: mtu 1500 qdisc mq state UP group default qlen 1000* *5: eth3: mtu 1500 qdisc mq state DOWN group default qlen 1000* *6: eth4: mtu 1500 qdisc mq master eth0 state DOWN group default qlen 1000* *7: eth5: mtu 1500 qdisc mq state DOWN group default qlen 1000* *WITHOUT `dev` config options in dpdk{} section of startup:* *May 03 11:19:54 demo001 vpp[2154]: vpp[2154]: vlib_vmbus_bind_to_uio: Skipping VMBUS device 000d3a8a-a89d-000d-3a8a-a89d000d3a8a as host interface eth0 is up* *May 03 11:19:54 demo001 vpp[2154]: vlib_vmbus_bind_to_uio: Skipping VMBUS device 000d3a8a-a89d-000d-3a8a-a89d000d3a8a as host interface eth0 is up* *May 03 11:19:54 demo001 vpp[2154]: vpp[2154]: dpdk: EAL init args: --in-memory --no-telemetry --file-prefix vpp -b 000d3a8a-a89d-000d-3a8a-a89d000d3a8a* *May 03 11:19:54 demo001 vpp[2154]: dpdk: EAL init args: --in-memory --no-telemetry --file-prefix vpp -b 000d3a8a-a89d-000d-3a8a-a89d000d3a8a* *May 03 11:19:55 demo001 kernel: mlx5_core 2ab4:00:02.0 eth5: Link up* *May 03 11:19:55 demo001 NetworkManager[1047]: [1651576795.1466] device (eth5): carrier: link connected* *May 03 11:19:55 demo001 kernel: mlx5_core 8213:00:02.0 eth3: Link up* *May 03 11:19:55 demo001 NetworkManager[1047]: [1651576795.3444] device (eth3): carrier: link connected* *May 03 11:19:55 demo001 kernel: mlx5_core e2ca:00:02.0 eth4: Link up* *May 03 11:19:55 demo001 kernel: hv_netvsc 000d3a8a-a89d-000d-3a8a-a89d000d3a8a eth0: Data path switched to VF: eth4* *May 03 11:19:55 demo001 NetworkManager[1047]: [1651576795.5913] device (eth4): carrier: link connected* *May 03 11:19:55 demo001 vnet[2154]: dpdk: Interface error -22: Unknown error -22* *May 03 11:19:55 demo001 vnet[2154]: dpdk: Interface error -22: Unknown error -22* *May 03 11:19:55 demo001 vnet[2154]: dpdk: [2] setup failed Errors:* *rte_eth_tx_queue_setup[port:2, errno:-22]: Unknown error -22* * rte_eth_rx_queue_setup[port:2, errno:-22]: Unknown error -22* *May 03 11:19:55 demo001 vnet[2154]: received signal SIGSEGV, PC 0x7fcdaab1512c, faulting address 0x0* *May 03 11:19:55 demo001 vnet[2154]: #0 0x7fcdab61f9d7 0x7fcdab61f9d7* *May 03 11:19:55 demo001 vnet[2154]: #1 0x7fcdaaf59c20 0x7fcdaaf59c20* *May 03 11:19:55 demo001 vnet[2154]: #2 0x7fcdaab1512c 0x7fcdaab1512c* *May 03 11:19:55 demo001 vnet[2154]: #3 0x7fcdaab16bfe va_format + 0xee* *May 03 11:19:55 demo001 vnet[2154]: #4 0x7fcdaab18b04 format + 0x84* *May 03 11:19:55 demo001 vnet[2154]: #5 0x7fcdaab1488c 0x7fcdaab1488c* *May 03 11:19:55 demo001 vnet[2154]: #6 0x7fcdaab16bfe va_format + 0xee* *May 03 11:19:55 demo001 vnet[2154]: #7 0x7fcdab5c39f8 vlib_log + 0x2a8* *May 03 11:19:55 demo001 vnet[2154]: #8 0x7fcd45389305 0x7fcd45389305* *May 03 11:19:55 demo001 vnet[2154]: #9 0x7fcd45399382 0x7fcd45399382* *May 03 11:19:55 demo001 vnet[2154]: #10 0x7fcd4539e0a6 0x7fcd4539e0a6* *May 03 11:19:55 demo001 vnet[2154]: #11 0x7fcdab5c4ac6 0x7fcdab5c4ac6* *May 03 11:19:55 demo001 vnet[2154]: #12 0x7fcdaab2ce10 0x7fcdaab2ce10* *May 03 11:19:55 demo001 vpp[2154]: Fail
Re: [vpp-dev] Query on VPP in Aure
Hello Matt, Thank you again for the response. So we were able to compile but now are unable to start VPP. We are seeing the following errors: I rebuilt, and got: May 02 20:20:24 demo001 vpp[2161]: vpp[2161]: perfmon: skipping source 'intel-uncore' - intel_uncore_init: no uncore units found May 02 20:20:24 demo001 vpp[2161]: perfmon: skipping source 'intel-uncore' - intel_uncore_init: no uncore units found May 02 20:20:24 demo001 vpp[2161]: vpp[2161]: tls_init_ca_chain:617: Could not initialize TLS CA certificates May 02 20:20:24 demo001 vpp[2161]: vpp[2161]: tls_mbedtls_init:652: failed to initialize TLS CA chain May 02 20:20:24 demo001 vpp[2161]: tls_init_ca_chain:617: Could not initialize TLS CA certificates May 02 20:20:24 demo001 vpp[2161]: tls_mbedtls_init:652: failed to initialize TLS CA chain May 02 20:20:24 demo001 vpp[2161]: vpp[2161]: tls_init_ca_chain:1083: Could not initialize TLS CA certificates May 02 20:20:24 demo001 vpp[2161]: vpp[2161]: tls_openssl_init:1206: failed to initialize TLS CA chain May 02 20:20:24 demo001 vpp[2161]: tls_init_ca_chain:1083: Could not initialize TLS CA certificates May 02 20:20:24 demo001 vpp[2161]: tls_openssl_init:1206: failed to initialize TLS CA chain May 02 20:20:24 demo001 vpp[2161]: vpp[2161]: vlib_vmbus_bind_to_uio: Skipping VMBUS device 000d3a8a-a89d-000d-3a8a-a89d000d3a8a as host interface eth0 is up May 02 20:20:24 demo001 vpp[2161]: vlib_vmbus_bind_to_uio: Skipping VMBUS device 000d3a8a-a89d-000d-3a8a-a89d000d3a8a as host interface eth0 is up May 02 20:20:24 demo001 vnet[2161]: dpdk: Interface Ethernet0/2/0 error -22: Unknown error -22 May 02 20:20:24 demo001 vnet[2161]: dpdk: Interface Ethernet0/2/0 error -22: Unknown error -22 May 02 20:20:24 demo001 vnet[2161]: dpdk: [0] setup failed Errors: rte_eth_tx_queue_setup[port:0, errno:-22]: Unknown error -22 rte_eth_rx_queue_setup[port:0, errno:-22]: Unknown error -22 May 02 20:20:25 demo001 vnet[2161]: received signal SIGSEGV, PC 0x7f4ba079c12c, faulting address 0xdd7 May 02 20:20:25 demo001 vnet[2161]: #0 0x7f4ba12a69d7 0x7f4ba12a69d7 May 02 20:20:25 demo001 vnet[2161]: #1 0x7f4ba0be0c20 0x7f4ba0be0c20 May 02 20:20:25 demo001 vnet[2161]: #2 0x7f4ba079c12c 0x7f4ba079c12c May 02 20:20:25 demo001 vnet[2161]: #3 0x7f4ba079dbfe va_format + 0xee May 02 20:20:25 demo001 vnet[2161]: #4 0x7f4ba079fb04 format + 0x84 May 02 20:20:25 demo001 vnet[2161]: #5 0x7f4ba079b88c 0x7f4ba079b88c May 02 20:20:25 demo001 vnet[2161]: #6 0x7f4ba079dbfe va_format + 0xee May 02 20:20:25 demo001 vnet[2161]: #7 0x7f4ba124a9f8 vlib_log + 0x2a8 May 02 20:20:25 demo001 vnet[2161]: #8 0x7f4b5b160305 0x7f4b5b160305 May 02 20:20:25 demo001 vnet[2161]: #9 0x7f4b5b170382 0x7f4b5b170382 May 02 20:20:25 demo001 vnet[2161]: #10 0x7f4b5b1750a6 0x7f4b5b1750a6 May 02 20:20:25 demo001 vnet[2161]: #11 0x7f4ba124bac6 0x7f4ba124bac6 May 02 20:20:25 demo001 vnet[2161]: #12 0x7f4ba07b3e10 0x7f4ba07b3e10 May 02 20:20:25 demo001 vpp[2161]: Failed to save post-mortem API trace to /tmp/api_post_mortem.2161 May 02 20:20:25 demo001 systemd[1]: vpp.service: Main process exited, code=killed, status=6/ABRT May 02 20:20:25 demo001 systemd[1]: vpp.service: Failed with result 'signal'. -- Subject: Unit failed This time, vpp doesn’t even start up due to the above errors. *Is there a specific kernel version you used?* -- John M. On Mon, May 2, 2022 at 1:42 PM Matthew Smith wrote: > > Sorry, there was a typo. You should use DPDK_MLX5_COMMON_PMD=y instead > of DPDK_MLX_COMMON_PMD=y. > > -Matt > > > On Mon, May 2, 2022 at 11:28 AM Johnny Martinez > wrote: > >> Hello Matt! >> >> *Thank you very much for the reply!* >> >> Our end goal is to be using Alma Linux but we went ahead and tried it in >> a fresh Ubuntu 20.04 installation with the same result. >> >> It looks like it is unable to compile the *mlx5 drivers*. >> >> *I’ve attached the complete build log for your review.* >> >> >> >> *Build was started with:* >> >> *make DPDK_MLX4_PMD=y DPDK_MLX5_PMD=y DPDK_MLX_COMMON_PMD=y pkg-deb | tee >> /root/vpp-build.log* >> >> >> >> As an extra precaution, the vpp.mk content is: >> >> >> >> *MACHINE=$(shell uname -m)* >> >> *vpp_arch = native* >> >> *vpp_root_packages = vpp* >> >> *vpp_debug_TAG_BUILD_TYPE = debug* >> >> *vpp_TAG_BUILD_TYPE = release* >> >> *vpp_clang_TAG_BUILD_TYPE = release* >> >> *vpp_gcov_TAG_BUILD_TYPE = gcov* >> >> *vpp_coverity_TAG_BUILD_TYPE = coverity* >> >> *vpp_dpdk_inc_dir = /usr/include/dpdk/* >> >> *vpp_dpdk_lib_dir = /usr/lib64/* >> >> *vpp_uses_dpdk_mlx4_pmd = yes* >> >> *DPDK_MLX4_PMD=y* >> >> *DPDK_MLX5_PMD=y* >> >> *DPDK_MLX_COMMON_PMD=y* >> >> >> >> The build completes, but there’s an issue with the MLX5 driver. As seen >> in the build log, the *MLX4 is built, but MLX5 is not*. I’ve tried >> searching online, but there’s reference to *glue* with regard
Re: [vpp-dev] Query on VPP in Aure
Sorry, there was a typo. You should use DPDK_MLX5_COMMON_PMD=y instead of DPDK_MLX_COMMON_PMD=y. -Matt On Mon, May 2, 2022 at 11:28 AM Johnny Martinez wrote: > Hello Matt! > > *Thank you very much for the reply!* > > Our end goal is to be using Alma Linux but we went ahead and tried it in a > fresh Ubuntu 20.04 installation with the same result. > > It looks like it is unable to compile the *mlx5 drivers*. > > *I’ve attached the complete build log for your review.* > > > > *Build was started with:* > > *make DPDK_MLX4_PMD=y DPDK_MLX5_PMD=y DPDK_MLX_COMMON_PMD=y pkg-deb | tee > /root/vpp-build.log* > > > > As an extra precaution, the vpp.mk content is: > > > > *MACHINE=$(shell uname -m)* > > *vpp_arch = native* > > *vpp_root_packages = vpp* > > *vpp_debug_TAG_BUILD_TYPE = debug* > > *vpp_TAG_BUILD_TYPE = release* > > *vpp_clang_TAG_BUILD_TYPE = release* > > *vpp_gcov_TAG_BUILD_TYPE = gcov* > > *vpp_coverity_TAG_BUILD_TYPE = coverity* > > *vpp_dpdk_inc_dir = /usr/include/dpdk/* > > *vpp_dpdk_lib_dir = /usr/lib64/* > > *vpp_uses_dpdk_mlx4_pmd = yes* > > *DPDK_MLX4_PMD=y* > > *DPDK_MLX5_PMD=y* > > *DPDK_MLX_COMMON_PMD=y* > > > > The build completes, but there’s an issue with the MLX5 driver. As seen in > the build log, the *MLX4 is built, but MLX5 is not*. I’ve tried searching > online, but there’s reference to *glue* with regard to the MLX5, but > nothing actionable I could find: > > > > *Part 1: Shows that the library “mlx4” and “mlx5” are included as part of > “make” (means our build options were correctly interperated):* > > > > *cmake3 --build > /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/build-rdma-core > -- libccan.a libibverbs.a librdma_util.a libmlx5.a libmlx4.a > > /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/rdma-core.build.log* > > *sed 's/^Libs.private:.*/Libs.private: -lmlx4 -lmlx5 -libverbs -lrdma_util > -lccan -lpthread/' -i > /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/build-rdma-core/lib/pkgconfig/libibverbs.pc > >> > /root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/build-vpp-native/external/rdma-core.build.log* > > > *Part 2: Shows that MLX5 is missing “common_mlx5” (which I can’t find a > way to install…?) so is disabled. MLX4 however, is built:* > > > > *Using 'PKG_CONFIG_PATH' from environment with value: > '/root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/install-vpp-native/external/lib/pkgconfig'* > > *Run-time dependency libmlx4 found: YES 1.0.39.1* > > *Using 'PKG_CONFIG_PATH' from environment with value: > '/root/azure/vpp/build-root/rpmbuild/vpp-22.06/build-root/install-vpp-native/external/lib/pkgconfig'* > > *Run-time dependency libibverbs found: YES 1.14.39.1* > > *Compiler for C supports arguments -std=c11: YES* > > *Compiler for C supports arguments -Wno-strict-prototypes > -Wstrict-prototypes: YES* > > *Compiler for C supports arguments -D_BSD_SOURCE: YES* > > *Compiler for C supports arguments -D_DEFAULT_SOURCE: YES* > > *Compiler for C supports arguments -D_XOPEN_SOURCE=600: YES* > > *Header has symbol > "MLX4DV_SET_CTX_ATTR_BUF_ALLOCATORS" with dependencies libmlx4, libibverbs: > YES* > > *Header has symbol "MLX4DV_QP_MASK_UAR_MMAP_OFFSET" > with dependencies libmlx4, libibverbs: YES* > > *Checking whether type "struct mlx4_wqe_lso_seg" has member "mss_hdr_size" > with dependencies libmlx4, libibverbs: YES* > > *Configuring mlx4_autoconf.h using configuration* > > *Compiler for C supports arguments -std=c11: YES* > > *Compiler for C supports arguments -Wno-strict-prototypes > -Wstrict-prototypes: YES* > > *Compiler for C supports arguments -D_BSD_SOURCE: YES* > > *Compiler for C supports arguments -D_DEFAULT_SOURCE: YES* > > *Compiler for C supports arguments -D_XOPEN_SOURCE=600: YES* > > *Message: Disabling mlx5 [drivers/net/mlx5]: missing internal dependency > "common_mlx5"* > > *Run-time dependency libmusdk found: NO (tried pkgconfig)* > > *Run-time dependency libmusdk found: NO (tried pkgconfig)* > > *Run-time dependency netcope-common found: NO (tried pkgconfig)* > > > > *Part 3: Shows multiple missing “internal dependency” issues:* > > > > *Message: Disabling dpaa [drivers/dma/dpaa]: missing internal dependency > "bus_dpaa"* > > *Message: Disabling dpaa [drivers/net/dpaa]: missing internal dependency > "mempool_dpaa"* > > *Message: Disabling kni [drivers/net/kni]: missing internal dependency > "kni"* > > *Message: Disabling mlx5 [drivers/net/mlx5]: missing internal dependency > "common_mlx5"* > > *Message: Disabling caam_jr [drivers/crypto/caam_jr]: missing internal > dependency "bus_dpaa"* > > *Message: Disabling mlx5 [drivers/crypto/mlx5]: missing internal > dependency "common_mlx5"* > > *Message: Disabling mlx5 [drivers/compress/mlx5]: missing internal > dependency "common_mlx5"* > > *Message: Disabling mlx5 [drivers/regex/mlx5]: missing internal dependency > "common_mlx5"* > > *Message: Disabling mlx5 [drivers/vdpa/mlx5]: missin
Re: [vpp-dev] Query on VPP in Aure
[Edited Message Follows] Hi Matt, I run two vpp instance (different prefix) in same hyper-v vm, the first vpp instance prefix is vs, second is vc. I have eth1 and eth2, the config is below: I start the vs and vc instance one by one, after vc instance started, the first instance vs stuck in dpdk wait function. Does one hyper-v only support one vpp instance ? ``` [root@localhost net-base]# ls /sys/class/net/eth1/device -all lrwxrwxrwx 1 root root 0 Apr 29 21:14 /sys/class/net/eth1/device -> ../../../3362e322-f26f-4568-ba5c-d8ff6b734d0c [root@localhost net-base]# ls /sys/class/net/eth2/device -all lrwxrwxrwx 1 root root 0 Apr 29 21:14 /sys/class/net/eth2/device -> ../../../1c66c020-246f-4948-8d16-dd7e6c892f0d [root@localhost net-base]# modprobe uio_hv_generic [root@localhost net-base]# ifconfig eth1 down [root@localhost net-base]# ifconfig eth2 down [root@localhost net-base]# ifconfig eth1 eth1: flags=4098 mtu 1500 ether 00:15:5d:5c:68:0e txqueuelen 1000 (Ethernet) RX packets 459960 bytes 47915079 (45.6 MiB) RX errors 0 dropped 7730 overruns 0 frame 0 TX packets 7838 bytes 1283736 (1.2 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 [root@localhost net-base]# ifconfig eth2 eth2: flags=4098 mtu 1500 ether 00:15:5d:5c:68:0f txqueuelen 1000 (Ethernet) RX packets 31316 bytes 3085337 (2.9 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 7658 bytes 1270728 (1.2 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 [root@localhost net-base]# ``` When i start the second vpp, the first vpp instance vs stuck in dpdk `vmbus_txbr_write` function Below is the first vpp insance vs stack : ``` vs$ sh hard Name Idx Link Hardware eth0 1 up eth0 Link speed: 1 Gbps RX Queues: queue thread mode 0 vs_wk_0 (1) polling 1 vs_wk_0 (1) polling 2 vs_wk_0 (1) polling 3 vs_wk_0 (1) polling 4 vs_wk_0 (1) polling 5 vs_wk_0 (1) polling 6 vs_wk_0 (1) polling 7 vs_wk_0 (1) polling Ethernet address 00:15:5d:5c:68:0e Microsoft Hyper-V Netvsc carrier up full duplex max-frame-size 0 flags: pmd-init-fail rx: queues 8 (max 64), desc 1024 (min 0 max 65535 align 1) tx: queues 2 (max 64), desc 1024 (min 1 max 4096 align 1) max rx packet len: 2048 promiscuous: unicast off all-multicast off vlan offload: strip off filter off qinq off rx offload avail: vlan-strip ipv4-cksum udp-cksum tcp-cksum rss-hash rx offload active: none tx offload avail: vlan-insert ipv4-cksum udp-cksum tcp-cksum tcp-tso multi-segs tx offload active: none rss avail: ipv4-tcp ipv4-udp ipv4 ipv6-tcp ipv6 rss active: ipv4-tcp ipv4 tx burst function: (not available) rx burst function: (not available) Errors: rte_eth_rx_queue_setup[port:0, errno:-12]: Unknown error -12 local0 0 down local0 Link speed: unknown local vs$ Program received signal SIGINT, Interrupt. rte_atomic32_cmpset (dst=0x7ffa8000, exp=29392, src=29480) at ../src-dpdk/lib/eal/x86/include/rte_atomic.h:189 189 { Missing separate debuginfos, use: debuginfo-install libgcc-4.8.5-44.el7.x86_64 libstdc++-4.8.5-44.el7.x86_64 libuuid-2.23.2-65.el7_9.1.x86_64 mbedtls-2.7.17-1.el7.x86_64 pkcs11-helper-1.11-3.el7.x86_64 (gdb) bt #0 rte_atomic32_cmpset (dst=0x7ffa8000, exp=29392, src=29480) at ../src-dpdk/lib/eal/x86/include/rte_atomic.h:189 #1 0x7fff687689c7 in vmbus_txbr_write (tbr=0x7fe63ffb11f0, iov=0x7fff64421b90, iovlen=4, need_sig=0x7fff64421bdf) at ../src-dpdk/drivers/bus/vmbus/vmbus_bufring.c:162 #2 0x7fff6876906e in rte_vmbus_chan_send_sglist (chan=0x7fe63ffb11c0, sg=0x7fff64421c80, sglen=1, data=0x7fff64421c90, dlen=40, xactid=0, need_sig=0x0) at ../src-dpdk/drivers/bus/vmbus/vmbus_channel.c:175 #3 0x7fff6afe2e11 in hn_nvs_send_sglist (chan=0x7fe63ffb11c0, sg=0x7fff64421c80, sglen=1, nvs_msg=0x7fff64421c90, nvs_msglen=40, sndc=0, need_sig=0x0) at ../src-dpdk/drivers/net/netvsc/hn_nvs.h:236 #4 0x7fff6afe2fff in hn_nvs_send_rndis_ctrl (chan=0x7fe63ffb11c0, req=0x7fe640cd7000, reqlen=28) at ../src-dpdk/drivers/net/netvsc/hn_rndis.c:285 #5 0x7fff6afe33bb in hn_rndis_exec1 (hv=0x7fe63ffb1380, req=0x7fe640cd7000, reqlen=28, comp=0x7fe640cd6000, comp_len=28) at ../src-dpdk/drivers/net/netvsc/hn_rndis.c:396 #6 0x7fff6afe3554 in hn_rndis_execute (hv=0x7fe63ffb1380, rid=191, req=0x7fe640cd7000, reqlen=28, comp=0x7fe640cd6000, comp_len=28, comp_type=2147483652) at ../src-dpdk/drivers/net/netvsc/hn_rndis.c:440 #7 0x7fff6afe3733 in hn_rndis_query (hv=0x7fe63ffb1380, oid=65812, idata=0x0, idlen=0, odata=0x7fe63ffb13a8, odlen=4) at ../src-dpdk/drivers/net/netvsc/hn_rndis.c:501 #8 0x7fff6afe4c38 in hn_rndis_get_linkstatus (hv=0x7fe63ffb1380) at ../src-dpdk/drivers/net/netvsc/hn_rndis.c:1116 #9 0x7fff6afdf697 in hn_dev_link_update (dev=0x7fff6cff9180 , wait_to_complete=0) at ../src-dpdk/drivers/net/netvsc/hn_ethdev.c:234 #10 0x7fff6910615
Re: [vpp-dev] Query on VPP in Aure
Hi Matt, I run two vpp instance (different prefix) in same hyper-v vm, the first vpp instance prefix is vs, second is vc. I have eth1 and eth2, the config is below: I start the vs and vc instance one by one, after vc instance started, the first instance vs stuck in dpdk wait function. Does one hyper-v only support one vpp instance ? ``` [root@localhost net-base]# ls /sys/class/net/eth1/device -all lrwxrwxrwx 1 root root 0 Apr 29 21:14 /sys/class/net/eth1/device -> ../../../3362e322-f26f-4568-ba5c-d8ff6b734d0c [root@localhost net-base]# ls /sys/class/net/eth2/device -all lrwxrwxrwx 1 root root 0 Apr 29 21:14 /sys/class/net/eth2/device -> ../../../1c66c020-246f-4948-8d16-dd7e6c892f0d [root@localhost net-base]# modprobe uio_hv_generic [root@localhost net-base]# ifconfig eth1 down [root@localhost net-base]# ifconfig eth2 down [root@localhost net-base]# ifconfig eth1 eth1: flags=4098 mtu 1500 ether 00:15:5d:5c:68:0e txqueuelen 1000 (Ethernet) RX packets 459960 bytes 47915079 (45.6 MiB) RX errors 0 dropped 7730 overruns 0 frame 0 TX packets 7838 bytes 1283736 (1.2 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 [root@localhost net-base]# ifconfig eth2 eth2: flags=4098 mtu 1500 ether 00:15:5d:5c:68:0f txqueuelen 1000 (Ethernet) RX packets 31316 bytes 3085337 (2.9 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 7658 bytes 1270728 (1.2 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 [root@localhost net-base]# ``` When i start the second vpp, the first vpp instance vs stuck in dpdk `vmbus_txbr_write` function Below is the first vpp insance vs stack : ``` vs$ sh hard Name Idx Link Hardware eth0 1 up eth0 Link speed: 1 Gbps RX Queues: queue thread mode 0 vs_wk_0 (1) polling 1 vs_wk_0 (1) polling 2 vs_wk_0 (1) polling 3 vs_wk_0 (1) polling 4 vs_wk_0 (1) polling 5 vs_wk_0 (1) polling 6 vs_wk_0 (1) polling 7 vs_wk_0 (1) polling Ethernet address 00:15:5d:5c:68:0e Microsoft Hyper-V Netvsc carrier up full duplex max-frame-size 0 flags: pmd-init-fail rx: queues 8 (max 64), desc 1024 (min 0 max 65535 align 1) tx: queues 2 (max 64), desc 1024 (min 1 max 4096 align 1) max rx packet len: 2048 promiscuous: unicast off all-multicast off vlan offload: strip off filter off qinq off rx offload avail: vlan-strip ipv4-cksum udp-cksum tcp-cksum rss-hash rx offload active: none tx offload avail: vlan-insert ipv4-cksum udp-cksum tcp-cksum tcp-tso multi-segs tx offload active: none rss avail: ipv4-tcp ipv4-udp ipv4 ipv6-tcp ipv6 rss active: ipv4-tcp ipv4 tx burst function: (not available) rx burst function: (not available) Errors: rte_eth_rx_queue_setup[port:0, errno:-12]: Unknown error -12 local0 0 down local0 Link speed: unknown local vs$ Program received signal SIGINT, Interrupt. rte_atomic32_cmpset (dst=0x7ffa8000, exp=29392, src=29480) at ../src-dpdk/lib/eal/x86/include/rte_atomic.h:189 189 { Missing separate debuginfos, use: debuginfo-install libgcc-4.8.5-44.el7.x86_64 libstdc++-4.8.5-44.el7.x86_64 libuuid-2.23.2-65.el7_9.1.x86_64 mbedtls-2.7.17-1.el7.x86_64 pkcs11-helper-1.11-3.el7.x86_64 (gdb) bt #0 rte_atomic32_cmpset (dst=0x7ffa8000, exp=29392, src=29480) at ../src-dpdk/lib/eal/x86/include/rte_atomic.h:189 #1 0x7fff687689c7 in vmbus_txbr_write (tbr=0x7fe63ffb11f0, iov=0x7fff64421b90, iovlen=4, need_sig=0x7fff64421bdf) at ../src-dpdk/drivers/bus/vmbus/vmbus_bufring.c:162 #2 0x7fff6876906e in rte_vmbus_chan_send_sglist (chan=0x7fe63ffb11c0, sg=0x7fff64421c80, sglen=1, data=0x7fff64421c90, dlen=40, xactid=0, need_sig=0x0) at ../src-dpdk/drivers/bus/vmbus/vmbus_channel.c:175 #3 0x7fff6afe2e11 in hn_nvs_send_sglist (chan=0x7fe63ffb11c0, sg=0x7fff64421c80, sglen=1, nvs_msg=0x7fff64421c90, nvs_msglen=40, sndc=0, need_sig=0x0) at ../src-dpdk/drivers/net/netvsc/hn_nvs.h:236 #4 0x7fff6afe2fff in hn_nvs_send_rndis_ctrl (chan=0x7fe63ffb11c0, req=0x7fe640cd7000, reqlen=28) at ../src-dpdk/drivers/net/netvsc/hn_rndis.c:285 #5 0x7fff6afe33bb in hn_rndis_exec1 (hv=0x7fe63ffb1380, req=0x7fe640cd7000, reqlen=28, comp=0x7fe640cd6000, comp_len=28) at ../src-dpdk/drivers/net/netvsc/hn_rndis.c:396 #6 0x7fff6afe3554 in hn_rndis_execute (hv=0x7fe63ffb1380, rid=191, req=0x7fe640cd7000, reqlen=28, comp=0x7fe640cd6000, comp_len=28, comp_type=2147483652) at ../src-dpdk/drivers/net/netvsc/hn_rndis.c:440 #7 0x7fff6afe3733 in hn_rndis_query (hv=0x7fe63ffb1380, oid=65812, idata=0x0, idlen=0, odata=0x7fe63ffb13a8, odlen=4) at ../src-dpdk/drivers/net/netvsc/hn_rndis.c:501 #8 0x7fff6afe4c38 in hn_rndis_get_linkstatus (hv=0x7fe63ffb1380) at ../src-dpdk/drivers/net/netvsc/hn_rndis.c:1116 #9 0x7fff6afdf697 in hn_dev_link_update (dev=0x7fff6cff9180 , wait_to_complete=0) at ../src-dpdk/drivers/net/netvsc/hn_ethdev.c:234 #10 0x7fff6910615e in rte_eth_link_get_now
Re: [vpp-dev] Query on VPP in Aure
Hi John, Are you using a stock build of VPP from packagecloud or did you build your own? VPP's build of DPDK does not enable the Mellanox (mlx4, mlx5) PMDs by default. If you didn't already, you would need to build VPP with the environment variables DPDK_MLX4_PMD, DPDK_MLX5_PMD, and DPDK_MLX_COMMON_PMD set to y. You could change the default values from n to y in build/external/packages/dpdk.mk or you could run something like 'make DPDK_MLX4_PMD=y DPDK_MLX5_PMD=y DPDK_MLX_COMMON_PMD=y pkg-deb'. That presumes that you're running on ubuntu or some other debian-based distro, if you're running some other OS maybe substituting pkg-snap or pkg-rpm for the pkg-deb in that command would work for you. -Matt On Fri, Apr 29, 2022 at 2:40 PM Johnny Martinez wrote: > Johnny Martinez > 9:57 AM (5 hours ago) > to Benoit > *@Ben *- thank you *VERY *much for the detailed response! > > I had *partial *success but almost there - now I am seeing some DPDK > errors and other behavior outlined below. > > On my test device I have 3 interfaces: > >- Management interface (kernel interface, not managed by VPP) >- vpp-wan interface >- vpp-lan interface > > I configureD the interfaces with the following: > > set logging class linux-cp rate-limit 1000 level warn syslog-level notice > > > lcp default netns dataplane > > lcp lcp-sync on > > lcp lcp-auto-subint on > > > # vpp-wan > > vpp# lcp create NetVSC1 host-if vpp-wan > > vpp# set int state NetVSC1 up > > vpp# set int ip address NetVSC1 10.0.2.4/24 > > vpp# ip route add 0.0.0.0/0 via 10.0.2.1 > > > # vpp-lan > > vpp# lcp create NetVSC0 host-if vpp-lan > > vpp# set int state NetVSC0 up > > vpp# set int ip address NetVSC0 10.0.1.4/24 > > > I am now able to see the two interfaces in vpp *HOWEVER *I am not able to > set the MTU size on either interface to 1500. > > vpp# set int mtu 1500 NetVSC0 > set interface mtu: Unsupported (dpdk driver doesn't support MTU change) > vpp# set int mtu 1500 NetVSC1 > set interface mtu: Unsupported (dpdk driver doesn't support MTU change) > > > I see the following in the log files using '*journalctl | grep dpdk*' - > we get two seperate DPDK errors. *One seems to be interface specific and > the other when we try to set the MTU size*. > > The *orange *error is every second - whereas the MTU error is only when > we try to set the MTU size. > > *Apr 29 12:55:15 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add(): RNDIS > reports VF but device not found, retrying* > > *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach(): > Couldn't find port for VF* > > *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add(): RNDIS > reports VF but device not found, retrying* > > *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach(): > Couldn't find port for VF* > > *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add(): RNDIS > reports VF but device not found, retrying* > > *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: [0] > rte_eth_dev_set_mtu failed (mtu 1496, rv -95)* > > *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: set interface mtu: > Unsupported (dpdk driver doesn't support MTU change* > > *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach(): > Couldn't find port for VF* > > *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add(): RNDIS > reports VF but device not found, retrying* > > *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach(): > Couldn't find port for VF* > > *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add(): RNDIS > reports VF but device not found, retrying* > > > Anyone have any recommendation on how to resolve this? > > *I have also tried setting dev XXX and vdev commands in the dpdk{} block, > but this causes the virtual interfaces not to be found (as you have > previously mentioned).* > > *Thank you all in advance for your assistance!* > > -- > John M > > > On Fri, Apr 29, 2022 at 2:40 PM Matthew Smith wrote: > >> Hi Ben, >> >> Sure, I'll work on an update to the doc. >> >> -Matt >> >> >> On Fri, Apr 29, 2022 at 3:48 AM Benoit Ganne (bganne) >> wrote: >> >>> Hi Matt, John, >>> >>> This is a great summary! >>> >>> >> The only reference I could find for VPP in Azure was in this link >>> >> but it is very outdated: >>> >> https://fd.io/docs/vpp/v2101/usecases/vppinazure.html >>> >>> > Yes, that doc is way outdated. I advise against following the >>> instructions >>> > in it. >>> >>> Matt or John, would you mind proposing a patch for that? Even a quick & >>> dirty "here are the steps you need to follow" would be great. >>> >>> Best >>> ben >>> >> -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#21306): https://lists.fd.io/g/vpp-dev/message/21306 Mute This Topic: https://lists.fd.io/mt/90759128/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Query on VPP in Aure
Johnny Martinez 9:57 AM (5 hours ago) to Benoit *@Ben *- thank you *VERY *much for the detailed response! I had *partial *success but almost there - now I am seeing some DPDK errors and other behavior outlined below. On my test device I have 3 interfaces: - Management interface (kernel interface, not managed by VPP) - vpp-wan interface - vpp-lan interface I configureD the interfaces with the following: set logging class linux-cp rate-limit 1000 level warn syslog-level notice lcp default netns dataplane lcp lcp-sync on lcp lcp-auto-subint on # vpp-wan vpp# lcp create NetVSC1 host-if vpp-wan vpp# set int state NetVSC1 up vpp# set int ip address NetVSC1 10.0.2.4/24 vpp# ip route add 0.0.0.0/0 via 10.0.2.1 # vpp-lan vpp# lcp create NetVSC0 host-if vpp-lan vpp# set int state NetVSC0 up vpp# set int ip address NetVSC0 10.0.1.4/24 I am now able to see the two interfaces in vpp *HOWEVER *I am not able to set the MTU size on either interface to 1500. vpp# set int mtu 1500 NetVSC0 set interface mtu: Unsupported (dpdk driver doesn't support MTU change) vpp# set int mtu 1500 NetVSC1 set interface mtu: Unsupported (dpdk driver doesn't support MTU change) I see the following in the log files using '*journalctl | grep dpdk*' - we get two seperate DPDK errors. *One seems to be interface specific and the other when we try to set the MTU size*. The *orange *error is every second - whereas the MTU error is only when we try to set the MTU size. *Apr 29 12:55:15 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add(): RNDIS reports VF but device not found, retrying* *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach(): Couldn't find port for VF* *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add(): RNDIS reports VF but device not found, retrying* *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach(): Couldn't find port for VF* *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add(): RNDIS reports VF but device not found, retrying* *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: dpdk: [0] rte_eth_dev_set_mtu failed (mtu 1496, rv -95)* *Apr 29 12:55:16 azure-demo-node-001.vnet[2409]: set interface mtu: Unsupported (dpdk driver doesn't support MTU change* *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach(): Couldn't find port for VF* *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add(): RNDIS reports VF but device not found, retrying* *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_attach(): Couldn't find port for VF* *Apr 29 12:55:17 azure-demo-node-001.vnet[2409]: dpdk: hn_vf_add(): RNDIS reports VF but device not found, retrying* Anyone have any recommendation on how to resolve this? *I have also tried setting dev XXX and vdev commands in the dpdk{} block, but this causes the virtual interfaces not to be found (as you have previously mentioned).* *Thank you all in advance for your assistance!* -- John M On Fri, Apr 29, 2022 at 2:40 PM Matthew Smith wrote: > Hi Ben, > > Sure, I'll work on an update to the doc. > > -Matt > > > On Fri, Apr 29, 2022 at 3:48 AM Benoit Ganne (bganne) > wrote: > >> Hi Matt, John, >> >> This is a great summary! >> >> >> The only reference I could find for VPP in Azure was in this link >> >> but it is very outdated: >> >> https://fd.io/docs/vpp/v2101/usecases/vppinazure.html >> >> > Yes, that doc is way outdated. I advise against following the >> instructions >> > in it. >> >> Matt or John, would you mind proposing a patch for that? Even a quick & >> dirty "here are the steps you need to follow" would be great. >> >> Best >> ben >> > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#21305): https://lists.fd.io/g/vpp-dev/message/21305 Mute This Topic: https://lists.fd.io/mt/90759128/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Query on VPP in Aure
Thanks Matt! -daw- On 4/29/2022 2:40 PM, Matthew Smith via lists.fd.io wrote: Hi Ben, Sure, I'll work on an update to the doc. -Matt On Fri, Apr 29, 2022 at 3:48 AM Benoit Ganne (bganne) wrote: Hi Matt, John, This is a great summary! >> The only reference I could find for VPP in Azure was in this link >> but it is very outdated: >> https://fd.io/docs/vpp/v2101/usecases/vppinazure.html > Yes, that doc is way outdated. I advise against following the instructions > in it. Matt or John, would you mind proposing a patch for that? Even a quick & dirty "here are the steps you need to follow" would be great. Best ben -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#21304): https://lists.fd.io/g/vpp-dev/message/21304 Mute This Topic: https://lists.fd.io/mt/90759128/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Query on VPP in Aure
Hi Ben, Sure, I'll work on an update to the doc. -Matt On Fri, Apr 29, 2022 at 3:48 AM Benoit Ganne (bganne) wrote: > Hi Matt, John, > > This is a great summary! > > >> The only reference I could find for VPP in Azure was in this link > >> but it is very outdated: > >> https://fd.io/docs/vpp/v2101/usecases/vppinazure.html > > > Yes, that doc is way outdated. I advise against following the > instructions > > in it. > > Matt or John, would you mind proposing a patch for that? Even a quick & > dirty "here are the steps you need to follow" would be great. > > Best > ben > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#21303): https://lists.fd.io/g/vpp-dev/message/21303 Mute This Topic: https://lists.fd.io/mt/90759128/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Query on VPP in Aure
Hi Matt, John, This is a great summary! >> The only reference I could find for VPP in Azure was in this link >> but it is very outdated: >> https://fd.io/docs/vpp/v2101/usecases/vppinazure.html > Yes, that doc is way outdated. I advise against following the instructions > in it. Matt or John, would you mind proposing a patch for that? Even a quick & dirty "here are the steps you need to follow" would be great. Best ben -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#21302): https://lists.fd.io/g/vpp-dev/message/21302 Mute This Topic: https://lists.fd.io/mt/90759128/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Query on VPP in Aure
Hi John, See answers inline below... On Thu, Apr 28, 2022 at 12:51 PM John Martinez wrote: > Hello everyone! > > This is my first post to the list! > > I was able to setup VPP in AWS after > properly identifying the instance type that would allow VPP to grap the PCI > addresses that I needed to specify in the /etc/vpp/startup.conf in the dpdk > stanza. > > I used an m5.2xlarge in AWS and it works well! > > [image: image.png] > > I am now looking to setup VPP in an Azure instance but lspci is > showing only a single Mellanox PCI device even though I added the > additional interfaces to the VM. > > [image: image.png] > When an Azure VM has a Mellanox VF attached to it's PCI bus, that indicates that there is an interface present which has accelerated networking enabled. Since you have multiple interfaces but only see one PCI device associated with a Mellanox VF, that suggests that only one of the interfaces attached to the VM has accelerated networking enabled. When you created the additional NICs, did you enable accelerated networking on them? I'm not positive, but I don't think you can enable accelerated networking when you create a NIC via the Azure web portal, you might need to create it using the Azure CLI or PowerShell. The presence of PCI devices doesn't actually matter too much. Azure netvsc interfaces are attached to the Hyper-V VMbus rather than the PCI bus. See the first few paragraphs of https://docs.microsoft.com/en-us/azure/virtual-network/accelerated-networking-how-it-works. You only need one PCI device to be present per interface if you intend to use accelerated networking. I tried DS3 and DS4 instance types in Azure > > The only reference I could find for VPP in Azure was in this link but it > is very outdated: > > https://fd.io/docs/vpp/v2101/usecases/vppinazure.html > Yes, that doc is way outdated. I advise against following the instructions in it. > Has anyone successfully installed and used VPP in Azure, if so,what > instance type did you use? > Yes, I have used Standard_D3_v2 and Standard_DS3_v2. The instance type shouldn't matter too much. The exception is that if you want to use accelerated networking, you need to pick an instance type that supports it. But accelerated networking has been around for a few years so a lot of instance types support it. > Any other Azure hints and tips welcome! > If you load the uio_hv_generic kernel module and make sure that the kernel interfaces you want VPP to attach to are down (e.g. 'ip link set dev eth2 down') at the time VPP starts, VPP's DPDK plugin should automatically bind them to the uio_hv_generic module and set them up as VPP interfaces. I think you need the version of uio_hv_generic and hv_vmbus from at least version 4.20 of the linux kernel (IIRC the most recent versions of RHEL & CentOS 8 stream have the required patches backported to their 4.18-based kernel). As explained in the article I linked above, the netvsc driver (DPDK's NetVSC PMD) abstracts and manages the Mellanox VF and another virtual path. So you should not whitelist the PCI address of the Mellanox VF in startup.conf. If you try to have VPP manage the Mellanox VF directly, you'll probably have trouble making it work. You want VPP to manage the netvsc devices. These can be whitelisted similar to PCI devices by adding a 'dev ' to the dpdk stanza in startup.conf. Good luck. -Matt -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#21301): https://lists.fd.io/g/vpp-dev/message/21301 Mute This Topic: https://lists.fd.io/mt/90759128/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] Query on VPP in Aure
Hello everyone! This is my first post to the list! I was able to setup VPP in AWS after properly identifying the instance type that would allow VPP to grap the PCI addresses that I needed to specify in the /etc/vpp/startup.conf in the dpdk stanza. I used an m5.2xlarge in AWS and it works well! [image: image.png] I am now looking to setup VPP in an Azure instance but lspci is showing only a single Mellanox PCI device even though I added the additional interfaces to the VM. [image: image.png] I tried DS3 and DS4 instance types in Azure The only reference I could find for VPP in Azure was in this link but it is very outdated: https://fd.io/docs/vpp/v2101/usecases/vppinazure.html Has anyone successfully installed and used VPP in Azure, if so,what instance type did you use? Any other Azure hints and tips welcome! Thank you in advance for your help! -- John M. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#21300): https://lists.fd.io/g/vpp-dev/message/21300 Mute This Topic: https://lists.fd.io/mt/90759128/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-