[ovs-discuss] question regarding how vxlan work in userspace
Hi When vxlan is handled in the kernel, what I understand is that the vxlan driver creates a fake socket that listens on the vxlan UDP port. Then the UDP layer looks up for a socket corresponding to the port of an incoming packet, and realizes it's a socket created by a tunnel. Then pass the vxlan packet to the vxlan driver. Then the vxlan driver decapsulates and because the interface vxlan0 is enslaved to an OVS bridge (br0)... it eventually just passes the inner packet to the OVS bridge which sees a packet arriving from vxlan0. My question is.. what is happening when I have a whole setup in userspace. What happens when I create a userspace vxlan interface in an OVS bridge ? Does the OVS code start automatically checking all packets arriving from any interface into a bridge to match if it's a VXLAN packet it has to decapsulate ? Once the packets are decapsulated, can the packets be matched with in_port= with the ofport of the vxlan0 ? regards, Nicolas ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] One question about AVX512 support
On Wed, Jun 8, 2022 at 6:24 PM Van Haaren, Harry wrote: > > > -Original Message- > > From: Li Zhang > > Sent: Wednesday, June 8, 2022 4:57 PM > > To: Van Haaren, Harry > > Cc: Pai G, Sunil ; ovs-discuss@openvswitch.org > > Subject: Re: [ovs-discuss] One question about AVX512 support > > > > On Wed, Jun 8, 2022 at 5:10 PM Van Haaren, Harry > > wrote: > > > > > > Hi Li, Sunil and OVS Discuss mailing list, > > > > > > Answering the direction question inline: > > > > Any ideas? -DHAVE_AVX512F and -DHAVE_LD_AVX512_GOOD are always > > enabled. > > > They are always enabled *assuming* your binutils version does not have > > > bugs. > > > > > > Why are you trying to disable AVX512? It will not run by default, what is > > > the end- > > goal here? > > > Below a large amount of detail around CPU ISA enabling, and how it > > > technically all > > works & why :) > > > > > > > Thanks a lot for your clarification. > > Recently, I encountered a bug. I found that ovs crashes when executing > > avx instructions. > > It's really like this bug: > > https://inbox.dpdk.org/dev/31482910.FCGgztJ3Sx@xps/T/#m95492563f7e2819395e > > 00e56d18f19e9911d2370 > > Is the DPDK being built with a buggy (pre-fix) version of binutils? Maybe. I build dpdk/ovs with gcc 7.5.0. Intel(R) Xeon(R) Platinum, Intel(R) Xeon(R) CPU E5-2660. But from the default setting, the AVX512 has been disabled in dpdk. But it fails on: *15ca2: c4 e3 7d 38 42 f0 01vinserti128 $0x1,-0x10(%rdx),%ymm0,%ymm0 Sometimes, it fails on: 15c86: 62 f1 7f 08 6f 4a fcvmovdqu8 -0x40(%rdx),%xmm1 15c70: 48 83 ea 80 sub$0xff80,%rdx 15c74: 48 83 e9 80 sub$0xff80,%rcx 15c78: 62 f1 7f 08 6f 52 favmovdqu8 -0x60(%rdx),%xmm2 15c7f: c4 e3 65 38 5a 90 01vinserti128 $0x1,-0x70(%rdx),%ymm3,%ymm3 *15c86: 62 f1 7f 08 6f 4a fcvmovdqu8 -0x40(%rdx),%xmm1 15c8d: c4 e3 6d 38 52 b0 01vinserti128 $0x1,-0x50(%rdx),%ymm2,%ymm2 15c94: 62 f1 7f 08 6f 42 fevmovdqu8 -0x20(%rdx),%xmm0 15c9b: c4 e3 75 38 4a d0 01vinserti128 $0x1,-0x30(%rdx),%ymm1,%ymm1 *15ca2: c4 e3 7d 38 42 f0 01vinserti128 $0x1,-0x10(%rdx),%ymm0,%ymm0 15ca9: c5 f8 11 59 80 vmovups %xmm3,-0x80(%rcx) > > > I am not quite sure if avx512 in ovs should be disabled or not. I want > > to make sure that ovs/dpdk disables avx512. > > If it won't run by default, I think I can leave it alone. > > Given the test binutils bug-check passes on the OVS build, it is to be > expected that no issues arise from *OVS* code. If you link OVS against > a DPDK built-with-buggy-binutils, you can still crash in that code. > I see. > For OVS code, you can explicitly disable by modifying the m4/openvswitch.m4 > file at +424 "checks for binutils/assermbler known issue with AVX512". > Always return "=no" (on line 442 here). That will emulate the "bad" binutils, > resulting in full disabling of all AVX512 in the *OVS* build. > I see, thanks. I would like to try to disable avx512 , avx2 or avx. > In the intro email, you mentioned ovs and DPDK versions as follows: > openvswitch 2.14.2, and dpdk-19.11.4 > > There is a 19.11.12 available now, although I didn't find relevant fixes in > the changelog; > https://doc.dpdk.org/guides-19.11/rel_notes/release_19_11.html# > > To force disable AVX512 in DPDK, pass "-mno-avx512f" as "machine_args", or > modify > /config/x86/meson.build and set "binutils_ok = false" on line 5. > > Then recompile DPDK, and ensure OVS is linking against your newly compiled > DPDK. OK, thanks for your suggestion. I will try it. > > Hope that helps in your root-causing! Regards, -Harry > > > > > -- Best Regards -Li ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] One question about AVX512 support
> -Original Message- > From: Li Zhang > Sent: Wednesday, June 8, 2022 4:57 PM > To: Van Haaren, Harry > Cc: Pai G, Sunil ; ovs-discuss@openvswitch.org > Subject: Re: [ovs-discuss] One question about AVX512 support > > On Wed, Jun 8, 2022 at 5:10 PM Van Haaren, Harry > wrote: > > > > Hi Li, Sunil and OVS Discuss mailing list, > > > > Answering the direction question inline: > > > Any ideas? -DHAVE_AVX512F and -DHAVE_LD_AVX512_GOOD are always > enabled. > > They are always enabled *assuming* your binutils version does not have bugs. > > > > Why are you trying to disable AVX512? It will not run by default, what is > > the end- > goal here? > > Below a large amount of detail around CPU ISA enabling, and how it > > technically all > works & why :) > > > > Thanks a lot for your clarification. > Recently, I encountered a bug. I found that ovs crashes when executing > avx instructions. > It's really like this bug: > https://inbox.dpdk.org/dev/31482910.FCGgztJ3Sx@xps/T/#m95492563f7e2819395e > 00e56d18f19e9911d2370 Is the DPDK being built with a buggy (pre-fix) version of binutils? > I am not quite sure if avx512 in ovs should be disabled or not. I want > to make sure that ovs/dpdk disables avx512. > If it won't run by default, I think I can leave it alone. Given the test binutils bug-check passes on the OVS build, it is to be expected that no issues arise from *OVS* code. If you link OVS against a DPDK built-with-buggy-binutils, you can still crash in that code. For OVS code, you can explicitly disable by modifying the m4/openvswitch.m4 file at +424 "checks for binutils/assermbler known issue with AVX512". Always return "=no" (on line 442 here). That will emulate the "bad" binutils, resulting in full disabling of all AVX512 in the *OVS* build. In the intro email, you mentioned ovs and DPDK versions as follows: openvswitch 2.14.2, and dpdk-19.11.4 There is a 19.11.12 available now, although I didn't find relevant fixes in the changelog; https://doc.dpdk.org/guides-19.11/rel_notes/release_19_11.html# To force disable AVX512 in DPDK, pass "-mno-avx512f" as "machine_args", or modify /config/x86/meson.build and set "binutils_ok = false" on line 5. Then recompile DPDK, and ensure OVS is linking against your newly compiled DPDK. Hope that helps in your root-causing! Regards, -Harry ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] One question about AVX512 support
On Wed, Jun 8, 2022 at 5:10 PM Van Haaren, Harry wrote: > > Hi Li, Sunil and OVS Discuss mailing list, > > Answering the direction question inline: > > Any ideas? -DHAVE_AVX512F and -DHAVE_LD_AVX512_GOOD are always enabled. > They are always enabled *assuming* your binutils version does not have bugs. > > Why are you trying to disable AVX512? It will not run by default, what is the > end-goal here? > Below a large amount of detail around CPU ISA enabling, and how it > technically all works & why :) > Thanks a lot for your clarification. Recently, I encountered a bug. I found that ovs crashes when executing avx instructions. It's really like this bug: https://inbox.dpdk.org/dev/31482910.FCGgztJ3Sx@xps/T/#m95492563f7e2819395e00e56d18f19e9911d2370 I am not quite sure if avx512 in ovs should be disabled or not. I want to make sure that ovs/dpdk disables avx512. If it won't run by default, I think I can leave it alone. > Hope the information here is helpful! Regards, -Harry > > > # Details around CPU ISA enabling > Let me try to explain the approach taken by both OVS and DPDK around AVX512 > compiling and running. > There are logically two "stages" to enabling AVX512 code, compile time and > runtime. It helps to treat them > individually, as they can occur on different machines (resulting in different > CPU flags being available, and > even "cross-compilations" where the resulting binary targets a different CPU > altogether!) > > # Compile Time; > At compile time, the OVS checks (using ./boot.sh and ./configure) a large > number of things: compiler versions, > compiler capabilities, CFLAGS, etc, to adjust the build parameters. It is > important to note for CPU ISA enabling > (e.g. AVX512, and others) that the CPU that does the configuring, and > compiling of OVS source code into the > binary, is *not* guaranteed to be the same as the one that *runs* the OVS > binaries in the end. > > As a result, the binary that is built must run for the CPU given by "-march" > in CFLAGS. This is often "corei7" or > "Nehalem" in practice, which enables ISA up to e.g. SSE4.2. This means that > AVX512 is not used by the compiler > in the *generic* scalar parts of OVS. > > Note that specific parts of the OVS codebase (DPIF, DPCLS, MiniflowExtract, > and Actions is WIP for 2.18 release) > are optimized using functions explicitly optimized for a specifically for the > AVX512 CPU ISA. These functions are > *compiled* by the compiler, and included in the binary (sometimes called a > "fat binary" as it includes multiple > versions of functions, which are selected based on CPU capabilities at > runtime). > > Very important; these "explicitly optimized" functions (using AVX512) are > *never* executed automatically. There > is *always* a runtime check to ensure that AVX512 is available on the runtime > CPU *before* any AVX512 instruction > executes. > > > # Runtime > At runtime, (on x86) the "CPUID" instruction is used to identify what ISA is > available. If it reports AVX512 is available, > then OVS is able to select the "explicitly optimized" functions from the "fat > binary". Note that at runtime, it is only > possible to execute code that was already compiled into the binary. As a > result, it is important to *always* include the > explicitly optimized functions at *compile time*, even if at compile-time the > CPU doesn't support the ISA! > > The technical method of enabling different functions at runtime is achieved > using "function pointers", a common concept > in e.g C or C++ codebases. DPDK uses it extensively e.g. at the ethdev layer > for different PMDs rx-routines, and even multiple > implementations of the *same* RX routine but optimized for different ISAs! > C++ also uses it (behind the scenes) for virtual > functions, and many other programming languages use them too (for context; > this is not "magic", its just not always visible > at the user-written code level.) > > > # OVS and DPDK together > Things become a little more complex, as there were specific bugs in a version > of binutils around assembling AVX512 code > years ago. This has been mitigated in OVS and DPDK, by checking a known "bad" > instruction sequence (on that assembler version) > and testing for the resulting invalid instruction sequence. As a result, both > DPDK and OVS are capable of *disabling* AVX512 > if the binutils version has this known bug. > > DPDK futher provides "-march" settings in its pkg-config file, as DPDK > requires SSE4.2 support as a baseline (on x86). This > setting is stripped from the pkg-config file in OVS's build configuration > stage, as OVS would like control over "-march" explicitly. > > Lastly, as DPDK and OVS could be built with *different* binutils versions (or > even on different compile machines!), both OVS > and DPDK check for the buggy-binutils themselves. As a result, it is possible > to have a DPDK/OVS combo with/without AVX512 > enabled. > > # Concerns around AVX512 enabling >
Re: [ovs-discuss] One question about AVX512 support
Hi Li, Sunil and OVS Discuss mailing list, Answering the direction question inline: > Any ideas? -DHAVE_AVX512F and -DHAVE_LD_AVX512_GOOD are always enabled. They are always enabled *assuming* your binutils version does not have bugs. Why are you trying to disable AVX512? It will not run by default, what is the end-goal here? Below a large amount of detail around CPU ISA enabling, and how it technically all works & why :) Hope the information here is helpful! Regards, -Harry # Details around CPU ISA enabling Let me try to explain the approach taken by both OVS and DPDK around AVX512 compiling and running. There are logically two "stages" to enabling AVX512 code, compile time and runtime. It helps to treat them individually, as they can occur on different machines (resulting in different CPU flags being available, and even "cross-compilations" where the resulting binary targets a different CPU altogether!) # Compile Time; At compile time, the OVS checks (using ./boot.sh and ./configure) a large number of things: compiler versions, compiler capabilities, CFLAGS, etc, to adjust the build parameters. It is important to note for CPU ISA enabling (e.g. AVX512, and others) that the CPU that does the configuring, and compiling of OVS source code into the binary, is *not* guaranteed to be the same as the one that *runs* the OVS binaries in the end. As a result, the binary that is built must run for the CPU given by "-march" in CFLAGS. This is often "corei7" or "Nehalem" in practice, which enables ISA up to e.g. SSE4.2. This means that AVX512 is not used by the compiler in the *generic* scalar parts of OVS. Note that specific parts of the OVS codebase (DPIF, DPCLS, MiniflowExtract, and Actions is WIP for 2.18 release) are optimized using functions explicitly optimized for a specifically for the AVX512 CPU ISA. These functions are *compiled* by the compiler, and included in the binary (sometimes called a "fat binary" as it includes multiple versions of functions, which are selected based on CPU capabilities at runtime). Very important; these "explicitly optimized" functions (using AVX512) are *never* executed automatically. There is *always* a runtime check to ensure that AVX512 is available on the runtime CPU *before* any AVX512 instruction executes. # Runtime At runtime, (on x86) the "CPUID" instruction is used to identify what ISA is available. If it reports AVX512 is available, then OVS is able to select the "explicitly optimized" functions from the "fat binary". Note that at runtime, it is only possible to execute code that was already compiled into the binary. As a result, it is important to *always* include the explicitly optimized functions at *compile time*, even if at compile-time the CPU doesn't support the ISA! The technical method of enabling different functions at runtime is achieved using "function pointers", a common concept in e.g C or C++ codebases. DPDK uses it extensively e.g. at the ethdev layer for different PMDs rx-routines, and even multiple implementations of the *same* RX routine but optimized for different ISAs! C++ also uses it (behind the scenes) for virtual functions, and many other programming languages use them too (for context; this is not "magic", its just not always visible at the user-written code level.) # OVS and DPDK together Things become a little more complex, as there were specific bugs in a version of binutils around assembling AVX512 code years ago. This has been mitigated in OVS and DPDK, by checking a known "bad" instruction sequence (on that assembler version) and testing for the resulting invalid instruction sequence. As a result, both DPDK and OVS are capable of *disabling* AVX512 if the binutils version has this known bug. DPDK futher provides "-march" settings in its pkg-config file, as DPDK requires SSE4.2 support as a baseline (on x86). This setting is stripped from the pkg-config file in OVS's build configuration stage, as OVS would like control over "-march" explicitly. Lastly, as DPDK and OVS could be built with *different* binutils versions (or even on different compile machines!), both OVS and DPDK check for the buggy-binutils themselves. As a result, it is possible to have a DPDK/OVS combo with/without AVX512 enabled. # Concerns around AVX512 enabling Note that neither DPDK nor OVS enables AVX512 without user input. DPDK defaults to AVX2/256 bit wide (ymm) registers, and only uses AVX512 if the --force-max-simd-bitwidth=512 EAL argument is passed: no reason for concern here. OVS will use scalar implementations, unless the appropriate "ovs-appctl dpif-netdev/*" command is run to enable AVX512 optimized routines (documentation here; https://docs.openvswitch.org/en/latest/topics/dpdk/bridge/#datapath-classifier-performance) All in all, CPU ISA is compiled in by default, and not enabled. It can be enabled manually by the user, and will then ensure that the required CPU ISA is available on the runtime
Re: [ovs-discuss] One question about AVX512 support
Thanks a lot. Hi Harry, Any ideas? -DHAVE_AVX512F and -DHAVE_LD_AVX512_GOOD are always enabled. On Wed, Jun 8, 2022 at 3:15 PM Pai G, Sunil wrote: > > Adding Harry to help answer these questions on DPDK and OVS building with > AVX512. > > Thanks and Regards, > Sunil > > > -Original Message- > > From: Li Zhang > > Sent: Wednesday, June 8, 2022 4:26 PM > > To: Pai G, Sunil > > Cc: ovs-discuss@openvswitch.org > > Subject: Re: [ovs-discuss] One question about AVX512 support > > > > Hi Pai, > > > > I have been trying to disable avx512 in OVS for the platform which doesn't > > support avx512. > > Building fails and it seems that it is not disabled. Any idea about it? > > > > # ./configure --with-dpdk=yes --prefix=/usr --localstatedir=/var -- > > sysconfdir=/etc CFLAGS="-mno-avx512f" > > # make > > > > libtool: compile: gcc -DHAVE_CONFIG_H -I. -I ./include -I ./include -I > > ./lib -I ./lib -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare - > > Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused-parameter > > -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style- > > definition -Wmissing-prototypes -Wmissing-field-initializers -fno-strict- > > aliasing -Wswitch-bool -Wlogical-not-parentheses -Wsizeof-array-argument - > > Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow -mssse3 - > > include rte_config.h -DOPENSSL_LOAD_CONF -I/usr/local/include > > -D_FILE_OFFSET_BITS=64 -mno-avx512f -DHAVE_AVX512F -DHAVE_LD_AVX512_GOOD - > > MT lib/netdev-dpdk.lo -MD -MP -MF lib/.deps/netdev-dpdk.Tpo -c lib/netdev- > > dpdk.c -o lib/netdev-dpdk.o In file included from /usr/lib64/gcc/x86_64- > > suse-linux/7/include/immintrin.h:41:0, > > from /usr/lib64/gcc/x86_64-suse- > > linux/7/include/x86intrin.h:48, > > from /usr/local/include/rte_vect.h:28, > > from /usr/local/include/rte_memcpy.h:17, > > from /usr/local/include/rte_ether.h:21, > > from /usr/local/include/rte_ethdev.h:159, > > from lib/netdev-dpdk.c:39: > > /usr/local/include/rte_memcpy.h: In function ‘rte_mov32’: > > /usr/lib64/gcc/x86_64-suse-linux/7/include/avxintrin.h:926:1: error: > > inlining failed in call to always_inline ‘_mm256_storeu_si256’: target > > specific option mismatch > > _mm256_storeu_si256 (__m256i_u *__P, __m256i __A) ^~~ In > > file included from /usr/local/include/rte_ether.h:21:0, > > from /usr/local/include/rte_ethdev.h:159, > > from lib/netdev-dpdk.c:39: > > /usr/local/include/rte_memcpy.h:320:2: note: called from here > > _mm256_storeu_si256((__m256i *)dst, ymm0); > > > > On Thu, Jun 2, 2022 at 4:00 PM Pai G, Sunil wrote: > > > > > > Hi Li, > > > > > > The assumption of ovs being dependent on dpdk for avx512 might not be > > true. > > > I found these two commits below in ovs-2.14.2 which strips out the "- > > march" and "-mno-avx512f" flags exported by dpdk i.e removes dependency on > > DPDK. The reason for this is rightly mentioned below as well. Hope this > > helps. > > > > > > > > > commit bb8f0e2a810889241f1d886d160ccee9b96c4d63 > > > Author: Ian Stokes > > > Date: Fri Jan 15 15:46:02 2021 + > > > > > > acinclude: Strip out -mno-avx512f provided by DPDK. > > > > > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > > > used to build DPDK had broken AVX512 support. > > > > > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > > > used to build DPDK had broken AVX512 support. But OVS could be > > built > > > with a completely different or fixed toolchain with correct avx512 > > > support. > > > > > > Fix that by stripping out `-mno-avx512f` as we already do for '- > > march'. > > > This will allow the OVS to decide if the AVX512 can be used. > > > > > > Reordering of CFLAGS (i.e. adding DPDK flags before OVS ones) is not > > an > > > option since autotools might reorder them back later and it's very > > > unpredictable. > > > > > > Reported-at: https://github.com/openvswitch/ovs-issues/issues/201 > > > Signed-off-by: Ilya Maximets > > > Co-authored-by: Ilya Maximets > > > Signed-off-by: Ian Stokes > > > > > > commit e9f9104d6a83ce7efd702120171835991779 > > > Author: Ian Stokes > > > Date: Fri Jan 15 14:54:04 2021 + > > > > > > acinclude: Strip out -march provided by DPDK. > > > > > > DPDK flags may include -march. Forcing -march could be > > > considered too heavy a requirement when users compile OVS from > > > source and could override user provided options. > > > > > > Resolve this by stripping -march from provided DPDK flags. > > > > > > Signed-off-by: Ian Stokes > > > > > > > > > > > > Thanks and Regards, > > > Sunil > > > > > > > -Original Message- > > > > From: discuss On Behalf Of Li > > > > Zhang > > > > Sent: Thursday, June 2, 2022 6:35 PM > > > > To:
Re: [ovs-discuss] One question about AVX512 support
Adding Harry to help answer these questions on DPDK and OVS building with AVX512. Thanks and Regards, Sunil > -Original Message- > From: Li Zhang > Sent: Wednesday, June 8, 2022 4:26 PM > To: Pai G, Sunil > Cc: ovs-discuss@openvswitch.org > Subject: Re: [ovs-discuss] One question about AVX512 support > > Hi Pai, > > I have been trying to disable avx512 in OVS for the platform which doesn't > support avx512. > Building fails and it seems that it is not disabled. Any idea about it? > > # ./configure --with-dpdk=yes --prefix=/usr --localstatedir=/var -- > sysconfdir=/etc CFLAGS="-mno-avx512f" > # make > > libtool: compile: gcc -DHAVE_CONFIG_H -I. -I ./include -I ./include -I > ./lib -I ./lib -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare - > Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused-parameter > -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style- > definition -Wmissing-prototypes -Wmissing-field-initializers -fno-strict- > aliasing -Wswitch-bool -Wlogical-not-parentheses -Wsizeof-array-argument - > Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow -mssse3 - > include rte_config.h -DOPENSSL_LOAD_CONF -I/usr/local/include > -D_FILE_OFFSET_BITS=64 -mno-avx512f -DHAVE_AVX512F -DHAVE_LD_AVX512_GOOD - > MT lib/netdev-dpdk.lo -MD -MP -MF lib/.deps/netdev-dpdk.Tpo -c lib/netdev- > dpdk.c -o lib/netdev-dpdk.o In file included from /usr/lib64/gcc/x86_64- > suse-linux/7/include/immintrin.h:41:0, > from /usr/lib64/gcc/x86_64-suse- > linux/7/include/x86intrin.h:48, > from /usr/local/include/rte_vect.h:28, > from /usr/local/include/rte_memcpy.h:17, > from /usr/local/include/rte_ether.h:21, > from /usr/local/include/rte_ethdev.h:159, > from lib/netdev-dpdk.c:39: > /usr/local/include/rte_memcpy.h: In function ‘rte_mov32’: > /usr/lib64/gcc/x86_64-suse-linux/7/include/avxintrin.h:926:1: error: > inlining failed in call to always_inline ‘_mm256_storeu_si256’: target > specific option mismatch > _mm256_storeu_si256 (__m256i_u *__P, __m256i __A) ^~~ In > file included from /usr/local/include/rte_ether.h:21:0, > from /usr/local/include/rte_ethdev.h:159, > from lib/netdev-dpdk.c:39: > /usr/local/include/rte_memcpy.h:320:2: note: called from here > _mm256_storeu_si256((__m256i *)dst, ymm0); > > On Thu, Jun 2, 2022 at 4:00 PM Pai G, Sunil wrote: > > > > Hi Li, > > > > The assumption of ovs being dependent on dpdk for avx512 might not be > true. > > I found these two commits below in ovs-2.14.2 which strips out the "- > march" and "-mno-avx512f" flags exported by dpdk i.e removes dependency on > DPDK. The reason for this is rightly mentioned below as well. Hope this > helps. > > > > > > commit bb8f0e2a810889241f1d886d160ccee9b96c4d63 > > Author: Ian Stokes > > Date: Fri Jan 15 15:46:02 2021 + > > > > acinclude: Strip out -mno-avx512f provided by DPDK. > > > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > > used to build DPDK had broken AVX512 support. > > > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > > used to build DPDK had broken AVX512 support. But OVS could be > built > > with a completely different or fixed toolchain with correct avx512 > > support. > > > > Fix that by stripping out `-mno-avx512f` as we already do for '- > march'. > > This will allow the OVS to decide if the AVX512 can be used. > > > > Reordering of CFLAGS (i.e. adding DPDK flags before OVS ones) is not > an > > option since autotools might reorder them back later and it's very > > unpredictable. > > > > Reported-at: https://github.com/openvswitch/ovs-issues/issues/201 > > Signed-off-by: Ilya Maximets > > Co-authored-by: Ilya Maximets > > Signed-off-by: Ian Stokes > > > > commit e9f9104d6a83ce7efd702120171835991779 > > Author: Ian Stokes > > Date: Fri Jan 15 14:54:04 2021 + > > > > acinclude: Strip out -march provided by DPDK. > > > > DPDK flags may include -march. Forcing -march could be > > considered too heavy a requirement when users compile OVS from > > source and could override user provided options. > > > > Resolve this by stripping -march from provided DPDK flags. > > > > Signed-off-by: Ian Stokes > > > > > > > > Thanks and Regards, > > Sunil > > > > > -Original Message- > > > From: discuss On Behalf Of Li > > > Zhang > > > Sent: Thursday, June 2, 2022 6:35 PM > > > To: ovs-discuss@openvswitch.org > > > Subject: [ovs-discuss] One question about AVX512 support > > > > > > Hi all, > > > > > > We are using openvswitch 2.14.2, and dpdk-19.11.4. I found avx512 is > > > enabled by default but it's disabled in DPDK. But I think ovs is > > > dependent on the dpdk library, right? But why does ovs work with > > > avx512 disabled in DPDK? > > > > > > I am not quite
Re: [ovs-discuss] One question about AVX512 support
Hi Pai, I have been trying to disable avx512 in OVS for the platform which doesn't support avx512. Building fails and it seems that it is not disabled. Any idea about it? # ./configure --with-dpdk=yes --prefix=/usr --localstatedir=/var --sysconfdir=/etc CFLAGS="-mno-avx512f" # make libtool: compile: gcc -DHAVE_CONFIG_H -I. -I ./include -I ./include -I ./lib -I ./lib -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare -Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused-parameter -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style-definition -Wmissing-prototypes -Wmissing-field-initializers -fno-strict-aliasing -Wswitch-bool -Wlogical-not-parentheses -Wsizeof-array-argument -Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow -mssse3 -include rte_config.h -DOPENSSL_LOAD_CONF -I/usr/local/include -D_FILE_OFFSET_BITS=64 -mno-avx512f -DHAVE_AVX512F -DHAVE_LD_AVX512_GOOD -MT lib/netdev-dpdk.lo -MD -MP -MF lib/.deps/netdev-dpdk.Tpo -c lib/netdev-dpdk.c -o lib/netdev-dpdk.o In file included from /usr/lib64/gcc/x86_64-suse-linux/7/include/immintrin.h:41:0, from /usr/lib64/gcc/x86_64-suse-linux/7/include/x86intrin.h:48, from /usr/local/include/rte_vect.h:28, from /usr/local/include/rte_memcpy.h:17, from /usr/local/include/rte_ether.h:21, from /usr/local/include/rte_ethdev.h:159, from lib/netdev-dpdk.c:39: /usr/local/include/rte_memcpy.h: In function ‘rte_mov32’: /usr/lib64/gcc/x86_64-suse-linux/7/include/avxintrin.h:926:1: error: inlining failed in call to always_inline ‘_mm256_storeu_si256’: target specific option mismatch _mm256_storeu_si256 (__m256i_u *__P, __m256i __A) ^~~ In file included from /usr/local/include/rte_ether.h:21:0, from /usr/local/include/rte_ethdev.h:159, from lib/netdev-dpdk.c:39: /usr/local/include/rte_memcpy.h:320:2: note: called from here _mm256_storeu_si256((__m256i *)dst, ymm0); On Thu, Jun 2, 2022 at 4:00 PM Pai G, Sunil wrote: > > Hi Li, > > The assumption of ovs being dependent on dpdk for avx512 might not be true. > I found these two commits below in ovs-2.14.2 which strips out the "-march" > and "-mno-avx512f" flags exported by dpdk i.e removes dependency on DPDK. The > reason for this is rightly mentioned below as well. Hope this helps. > > > commit bb8f0e2a810889241f1d886d160ccee9b96c4d63 > Author: Ian Stokes > Date: Fri Jan 15 15:46:02 2021 + > > acinclude: Strip out -mno-avx512f provided by DPDK. > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > used to build DPDK had broken AVX512 support. > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > used to build DPDK had broken AVX512 support. But OVS could be built > with a completely different or fixed toolchain with correct avx512 > support. > > Fix that by stripping out `-mno-avx512f` as we already do for '-march'. > This will allow the OVS to decide if the AVX512 can be used. > > Reordering of CFLAGS (i.e. adding DPDK flags before OVS ones) is not an > option since autotools might reorder them back later and it's very > unpredictable. > > Reported-at: https://github.com/openvswitch/ovs-issues/issues/201 > Signed-off-by: Ilya Maximets > Co-authored-by: Ilya Maximets > Signed-off-by: Ian Stokes > > commit e9f9104d6a83ce7efd702120171835991779 > Author: Ian Stokes > Date: Fri Jan 15 14:54:04 2021 + > > acinclude: Strip out -march provided by DPDK. > > DPDK flags may include -march. Forcing -march could be > considered too heavy a requirement when users compile OVS from > source and could override user provided options. > > Resolve this by stripping -march from provided DPDK flags. > > Signed-off-by: Ian Stokes > > > > Thanks and Regards, > Sunil > > > -Original Message- > > From: discuss On Behalf Of Li Zhang > > Sent: Thursday, June 2, 2022 6:35 PM > > To: ovs-discuss@openvswitch.org > > Subject: [ovs-discuss] One question about AVX512 support > > > > Hi all, > > > > We are using openvswitch 2.14.2, and dpdk-19.11.4. I found avx512 is > > enabled by default but it's disabled in DPDK. But I think ovs is dependent > > on the dpdk library, right? But why does ovs work with > > avx512 disabled in DPDK? > > > > I am not quite sure about the relationship between OVS and DPDK, any > > suggestions? > > > > -- > > > > Best Regards > > -Li > > ___ > > discuss mailing list > > disc...@openvswitch.org > > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss -- Best Regards -Li ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
