Re: [ovs-discuss] One question about AVX512 support
On Wed, Jun 8, 2022 at 6:24 PM Van Haaren, Harry wrote: > > > -Original Message- > > From: Li Zhang > > Sent: Wednesday, June 8, 2022 4:57 PM > > To: Van Haaren, Harry > > Cc: Pai G, Sunil ; ovs-discuss@openvswitch.org > > Subject: Re: [ovs-discuss] One question about AVX512 support > > > > On Wed, Jun 8, 2022 at 5:10 PM Van Haaren, Harry > > wrote: > > > > > > Hi Li, Sunil and OVS Discuss mailing list, > > > > > > Answering the direction question inline: > > > > Any ideas? -DHAVE_AVX512F and -DHAVE_LD_AVX512_GOOD are always > > enabled. > > > They are always enabled *assuming* your binutils version does not have > > > bugs. > > > > > > Why are you trying to disable AVX512? It will not run by default, what is > > > the end- > > goal here? > > > Below a large amount of detail around CPU ISA enabling, and how it > > > technically all > > works & why :) > > > > > > > Thanks a lot for your clarification. > > Recently, I encountered a bug. I found that ovs crashes when executing > > avx instructions. > > It's really like this bug: > > https://inbox.dpdk.org/dev/31482910.FCGgztJ3Sx@xps/T/#m95492563f7e2819395e > > 00e56d18f19e9911d2370 > > Is the DPDK being built with a buggy (pre-fix) version of binutils? Maybe. I build dpdk/ovs with gcc 7.5.0. Intel(R) Xeon(R) Platinum, Intel(R) Xeon(R) CPU E5-2660. But from the default setting, the AVX512 has been disabled in dpdk. But it fails on: *15ca2: c4 e3 7d 38 42 f0 01vinserti128 $0x1,-0x10(%rdx),%ymm0,%ymm0 Sometimes, it fails on: 15c86: 62 f1 7f 08 6f 4a fcvmovdqu8 -0x40(%rdx),%xmm1 15c70: 48 83 ea 80 sub$0xff80,%rdx 15c74: 48 83 e9 80 sub$0xff80,%rcx 15c78: 62 f1 7f 08 6f 52 favmovdqu8 -0x60(%rdx),%xmm2 15c7f: c4 e3 65 38 5a 90 01vinserti128 $0x1,-0x70(%rdx),%ymm3,%ymm3 *15c86: 62 f1 7f 08 6f 4a fcvmovdqu8 -0x40(%rdx),%xmm1 15c8d: c4 e3 6d 38 52 b0 01vinserti128 $0x1,-0x50(%rdx),%ymm2,%ymm2 15c94: 62 f1 7f 08 6f 42 fevmovdqu8 -0x20(%rdx),%xmm0 15c9b: c4 e3 75 38 4a d0 01vinserti128 $0x1,-0x30(%rdx),%ymm1,%ymm1 *15ca2: c4 e3 7d 38 42 f0 01vinserti128 $0x1,-0x10(%rdx),%ymm0,%ymm0 15ca9: c5 f8 11 59 80 vmovups %xmm3,-0x80(%rcx) > > > I am not quite sure if avx512 in ovs should be disabled or not. I want > > to make sure that ovs/dpdk disables avx512. > > If it won't run by default, I think I can leave it alone. > > Given the test binutils bug-check passes on the OVS build, it is to be > expected that no issues arise from *OVS* code. If you link OVS against > a DPDK built-with-buggy-binutils, you can still crash in that code. > I see. > For OVS code, you can explicitly disable by modifying the m4/openvswitch.m4 > file at +424 "checks for binutils/assermbler known issue with AVX512". > Always return "=no" (on line 442 here). That will emulate the "bad" binutils, > resulting in full disabling of all AVX512 in the *OVS* build. > I see, thanks. I would like to try to disable avx512 , avx2 or avx. > In the intro email, you mentioned ovs and DPDK versions as follows: > openvswitch 2.14.2, and dpdk-19.11.4 > > There is a 19.11.12 available now, although I didn't find relevant fixes in > the changelog; > https://doc.dpdk.org/guides-19.11/rel_notes/release_19_11.html# > > To force disable AVX512 in DPDK, pass "-mno-avx512f" as "machine_args", or > modify > /config/x86/meson.build and set "binutils_ok = false" on line 5. > > Then recompile DPDK, and ensure OVS is linking against your newly compiled > DPDK. OK, thanks for your suggestion. I will try it. > > Hope that helps in your root-causing! Regards, -Harry > > > > > -- Best Regards -Li ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] One question about AVX512 support
tripped from the pkg-config file in OVS's build configuration > stage, as OVS would like control over "-march" explicitly. > > Lastly, as DPDK and OVS could be built with *different* binutils versions (or > even on different compile machines!), both OVS > and DPDK check for the buggy-binutils themselves. As a result, it is possible > to have a DPDK/OVS combo with/without AVX512 > enabled. > > # Concerns around AVX512 enabling > Note that neither DPDK nor OVS enables AVX512 without user input. DPDK > defaults to AVX2/256 bit wide (ymm) registers, > and only uses AVX512 if the --force-max-simd-bitwidth=512 EAL argument is > passed: no reason for concern here. > > OVS will use scalar implementations, unless the appropriate "ovs-appctl > dpif-netdev/*" command is run to enable AVX512 > optimized routines (documentation here; > https://docs.openvswitch.org/en/latest/topics/dpdk/bridge/#datapath-classifier-performance) > > All in all, CPU ISA is compiled in by default, and not enabled. It can be > enabled manually by the user, and will then ensure that > the required CPU ISA is available on the runtime CPU, before actually > executing the optimized instruction sequence. As a result > binaries can happily contain AVX512 optimized code, it just won't be executed > unless the user requests it. > > > > -Original Message- > > From: Li Zhang > > Sent: Wednesday, June 8, 2022 3:51 PM > > To: Pai G, Sunil > > Cc: ovs-discuss@openvswitch.org; Van Haaren, Harry > > > > Subject: Re: [ovs-discuss] One question about AVX512 support > > > > Thanks a lot. > > > > Hi Harry, > > Any ideas? -DHAVE_AVX512F and -DHAVE_LD_AVX512_GOOD are always enabled. > > > > On Wed, Jun 8, 2022 at 3:15 PM Pai G, Sunil wrote: > > > > > > Adding Harry to help answer these questions on DPDK and OVS building with > > AVX512. > > > > > > Thanks and Regards, > > > Sunil > > > > > > > -Original Message- > > > > From: Li Zhang > > > > Sent: Wednesday, June 8, 2022 4:26 PM > > > > To: Pai G, Sunil > > > > Cc: ovs-discuss@openvswitch.org > > > > Subject: Re: [ovs-discuss] One question about AVX512 support > > > > > > > > Hi Pai, > > > > > > > > I have been trying to disable avx512 in OVS for the platform which > > > > doesn't > > > > support avx512. > > > > Building fails and it seems that it is not disabled. Any idea about it? > > > > > > > > # ./configure --with-dpdk=yes --prefix=/usr --localstatedir=/var -- > > > > sysconfdir=/etc CFLAGS="-mno-avx512f" > > > > # make > > > > > > > > libtool: compile: gcc -DHAVE_CONFIG_H -I. -I ./include -I ./include -I > > > > ./lib -I ./lib -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare - > > > > Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused- > > parameter > > > > -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style- > > > > definition -Wmissing-prototypes -Wmissing-field-initializers > > > > -fno-strict- > > > > aliasing -Wswitch-bool -Wlogical-not-parentheses > > > > -Wsizeof-array-argument - > > > > Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow -mssse3 > > > > - > > > > include rte_config.h -DOPENSSL_LOAD_CONF -I/usr/local/include > > > > -D_FILE_OFFSET_BITS=64 -mno-avx512f -DHAVE_AVX512F - > > DHAVE_LD_AVX512_GOOD - > > > > MT lib/netdev-dpdk.lo -MD -MP -MF lib/.deps/netdev-dpdk.Tpo -c > > > > lib/netdev- > > > > dpdk.c -o lib/netdev-dpdk.o In file included from /usr/lib64/gcc/x86_64- > > > > suse-linux/7/include/immintrin.h:41:0, > > > > from /usr/lib64/gcc/x86_64-suse- > > > > linux/7/include/x86intrin.h:48, > > > > from /usr/local/include/rte_vect.h:28, > > > > from /usr/local/include/rte_memcpy.h:17, > > > > from /usr/local/include/rte_ether.h:21, > > > > from /usr/local/include/rte_ethdev.h:159, > > > > from lib/netdev-dpdk.c:39: > > > > /usr/local/include/rte_memcpy.h: In function ‘rte_mov32’: > > > > /usr/lib64/gcc/x86_64-suse-linux/7/include/avxintrin.h:926:1: error: > > > > inlining failed in call to always_inline ‘_mm256_storeu_si256’: target > > > > specific option mismatch > > > > _mm256_storeu_si25
Re: [ovs-discuss] One question about AVX512 support
Thanks a lot. Hi Harry, Any ideas? -DHAVE_AVX512F and -DHAVE_LD_AVX512_GOOD are always enabled. On Wed, Jun 8, 2022 at 3:15 PM Pai G, Sunil wrote: > > Adding Harry to help answer these questions on DPDK and OVS building with > AVX512. > > Thanks and Regards, > Sunil > > > -Original Message- > > From: Li Zhang > > Sent: Wednesday, June 8, 2022 4:26 PM > > To: Pai G, Sunil > > Cc: ovs-discuss@openvswitch.org > > Subject: Re: [ovs-discuss] One question about AVX512 support > > > > Hi Pai, > > > > I have been trying to disable avx512 in OVS for the platform which doesn't > > support avx512. > > Building fails and it seems that it is not disabled. Any idea about it? > > > > # ./configure --with-dpdk=yes --prefix=/usr --localstatedir=/var -- > > sysconfdir=/etc CFLAGS="-mno-avx512f" > > # make > > > > libtool: compile: gcc -DHAVE_CONFIG_H -I. -I ./include -I ./include -I > > ./lib -I ./lib -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare - > > Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused-parameter > > -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style- > > definition -Wmissing-prototypes -Wmissing-field-initializers -fno-strict- > > aliasing -Wswitch-bool -Wlogical-not-parentheses -Wsizeof-array-argument - > > Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow -mssse3 - > > include rte_config.h -DOPENSSL_LOAD_CONF -I/usr/local/include > > -D_FILE_OFFSET_BITS=64 -mno-avx512f -DHAVE_AVX512F -DHAVE_LD_AVX512_GOOD - > > MT lib/netdev-dpdk.lo -MD -MP -MF lib/.deps/netdev-dpdk.Tpo -c lib/netdev- > > dpdk.c -o lib/netdev-dpdk.o In file included from /usr/lib64/gcc/x86_64- > > suse-linux/7/include/immintrin.h:41:0, > > from /usr/lib64/gcc/x86_64-suse- > > linux/7/include/x86intrin.h:48, > > from /usr/local/include/rte_vect.h:28, > > from /usr/local/include/rte_memcpy.h:17, > > from /usr/local/include/rte_ether.h:21, > > from /usr/local/include/rte_ethdev.h:159, > > from lib/netdev-dpdk.c:39: > > /usr/local/include/rte_memcpy.h: In function ‘rte_mov32’: > > /usr/lib64/gcc/x86_64-suse-linux/7/include/avxintrin.h:926:1: error: > > inlining failed in call to always_inline ‘_mm256_storeu_si256’: target > > specific option mismatch > > _mm256_storeu_si256 (__m256i_u *__P, __m256i __A) ^~~ In > > file included from /usr/local/include/rte_ether.h:21:0, > > from /usr/local/include/rte_ethdev.h:159, > > from lib/netdev-dpdk.c:39: > > /usr/local/include/rte_memcpy.h:320:2: note: called from here > > _mm256_storeu_si256((__m256i *)dst, ymm0); > > > > On Thu, Jun 2, 2022 at 4:00 PM Pai G, Sunil wrote: > > > > > > Hi Li, > > > > > > The assumption of ovs being dependent on dpdk for avx512 might not be > > true. > > > I found these two commits below in ovs-2.14.2 which strips out the "- > > march" and "-mno-avx512f" flags exported by dpdk i.e removes dependency on > > DPDK. The reason for this is rightly mentioned below as well. Hope this > > helps. > > > > > > > > > commit bb8f0e2a810889241f1d886d160ccee9b96c4d63 > > > Author: Ian Stokes > > > Date: Fri Jan 15 15:46:02 2021 + > > > > > > acinclude: Strip out -mno-avx512f provided by DPDK. > > > > > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > > > used to build DPDK had broken AVX512 support. > > > > > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > > > used to build DPDK had broken AVX512 support. But OVS could be > > built > > > with a completely different or fixed toolchain with correct avx512 > > > support. > > > > > > Fix that by stripping out `-mno-avx512f` as we already do for '- > > march'. > > > This will allow the OVS to decide if the AVX512 can be used. > > > > > > Reordering of CFLAGS (i.e. adding DPDK flags before OVS ones) is not > > an > > > option since autotools might reorder them back later and it's very > > > unpredictable. > > > > > > Reported-at: https://github.com/openvswitch/ovs-issues/issues/201 > > > Signed-off-by: Ilya Maximets > > > Co-authored-by: Ilya Maximets > > > Signed-off-by: Ian Stokes > > > > >
Re: [ovs-discuss] One question about AVX512 support
Hi Pai, I have been trying to disable avx512 in OVS for the platform which doesn't support avx512. Building fails and it seems that it is not disabled. Any idea about it? # ./configure --with-dpdk=yes --prefix=/usr --localstatedir=/var --sysconfdir=/etc CFLAGS="-mno-avx512f" # make libtool: compile: gcc -DHAVE_CONFIG_H -I. -I ./include -I ./include -I ./lib -I ./lib -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare -Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused-parameter -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style-definition -Wmissing-prototypes -Wmissing-field-initializers -fno-strict-aliasing -Wswitch-bool -Wlogical-not-parentheses -Wsizeof-array-argument -Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow -mssse3 -include rte_config.h -DOPENSSL_LOAD_CONF -I/usr/local/include -D_FILE_OFFSET_BITS=64 -mno-avx512f -DHAVE_AVX512F -DHAVE_LD_AVX512_GOOD -MT lib/netdev-dpdk.lo -MD -MP -MF lib/.deps/netdev-dpdk.Tpo -c lib/netdev-dpdk.c -o lib/netdev-dpdk.o In file included from /usr/lib64/gcc/x86_64-suse-linux/7/include/immintrin.h:41:0, from /usr/lib64/gcc/x86_64-suse-linux/7/include/x86intrin.h:48, from /usr/local/include/rte_vect.h:28, from /usr/local/include/rte_memcpy.h:17, from /usr/local/include/rte_ether.h:21, from /usr/local/include/rte_ethdev.h:159, from lib/netdev-dpdk.c:39: /usr/local/include/rte_memcpy.h: In function ‘rte_mov32’: /usr/lib64/gcc/x86_64-suse-linux/7/include/avxintrin.h:926:1: error: inlining failed in call to always_inline ‘_mm256_storeu_si256’: target specific option mismatch _mm256_storeu_si256 (__m256i_u *__P, __m256i __A) ^~~ In file included from /usr/local/include/rte_ether.h:21:0, from /usr/local/include/rte_ethdev.h:159, from lib/netdev-dpdk.c:39: /usr/local/include/rte_memcpy.h:320:2: note: called from here _mm256_storeu_si256((__m256i *)dst, ymm0); On Thu, Jun 2, 2022 at 4:00 PM Pai G, Sunil wrote: > > Hi Li, > > The assumption of ovs being dependent on dpdk for avx512 might not be true. > I found these two commits below in ovs-2.14.2 which strips out the "-march" > and "-mno-avx512f" flags exported by dpdk i.e removes dependency on DPDK. The > reason for this is rightly mentioned below as well. Hope this helps. > > > commit bb8f0e2a810889241f1d886d160ccee9b96c4d63 > Author: Ian Stokes > Date: Fri Jan 15 15:46:02 2021 + > > acinclude: Strip out -mno-avx512f provided by DPDK. > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > used to build DPDK had broken AVX512 support. > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > used to build DPDK had broken AVX512 support. But OVS could be built > with a completely different or fixed toolchain with correct avx512 > support. > > Fix that by stripping out `-mno-avx512f` as we already do for '-march'. > This will allow the OVS to decide if the AVX512 can be used. > > Reordering of CFLAGS (i.e. adding DPDK flags before OVS ones) is not an > option since autotools might reorder them back later and it's very > unpredictable. > > Reported-at: https://github.com/openvswitch/ovs-issues/issues/201 > Signed-off-by: Ilya Maximets > Co-authored-by: Ilya Maximets > Signed-off-by: Ian Stokes > > commit e9f9104d6a83ce7efd702120171835991779 > Author: Ian Stokes > Date: Fri Jan 15 14:54:04 2021 + > > acinclude: Strip out -march provided by DPDK. > > DPDK flags may include -march. Forcing -march could be > considered too heavy a requirement when users compile OVS from > source and could override user provided options. > > Resolve this by stripping -march from provided DPDK flags. > > Signed-off-by: Ian Stokes > > > > Thanks and Regards, > Sunil > > > -Original Message- > > From: discuss On Behalf Of Li Zhang > > Sent: Thursday, June 2, 2022 6:35 PM > > To: ovs-discuss@openvswitch.org > > Subject: [ovs-discuss] One question about AVX512 support > > > > Hi all, > > > > We are using openvswitch 2.14.2, and dpdk-19.11.4. I found avx512 is > > enabled by default but it's disabled in DPDK. But I think ovs is dependent > > on the dpdk library, right? But why does ovs work with > > avx512 disabled in DPDK? > > > > I am not quite sure about the relationship between OVS and DPDK, any > > suggestions? > > > > -- > > > > Best Regards > > -Li > > ___ > > discuss mailing list > > disc...@openvswitch.org > > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss -- Best Regards -Li ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] One question about AVX512 support
On Thu, Jun 2, 2022 at 7:31 PM Pai G, Sunil wrote: > > Hi Li, > > My responses are inline. > > Thanks and Regards, > Sunil > > > -Original Message- > > From: Li Zhang > > Sent: Thursday, June 2, 2022 9:56 PM > > To: Pai G, Sunil > > Cc: ovs-discuss@openvswitch.org > > Subject: Re: [ovs-discuss] One question about AVX512 support > > > > Hi Pai, > > > > Thanks a lot. It helps. > > These patches allow ovs to decide to disable or enable AVX512 if I > > understand it correctly. > > Well,...not quite, they simply remove the dependency on DPDK i.e > the decision to use AVX512/not in OVS is left to OVS only [DPDK arch > options/flags are simply ignored]. > I see. > > Is it recommended to enable AVX512? AVX512 is enabled by default. > > To get better performance and make the most out of the features offered by > the platform, I would say so, yes. > > > It is disabled in DPDK because it is experimental. > > I would recommend enabling AVX-512 in DPDK as well. > Here is a link which describes how to: > https://doc.dpdk.org/guides/howto/avx512.html > > i.e for ovs, use the following command before launching vswitchd: > ovs-vsctl --no-wait set Open_vSwitch . > other_config:dpdk-extra="--force-max-simd-bitwidth=512" > > > > I wonder if I should disable it, it may cause performance worse. > > Here are few links I have handy that describe the performance improvements > seen with AVX-512 along with its impacts. > https://networkbuilders.intel.com/solutionslibrary/open-vswitch-optimized-deployment-benchmark-technology-guide > . > https://networkbuilders.intel.com/intel-technologies/3rd-gen-intel-xeon-scalable-processors-experience-kits > , Acceleration section. > > I would encourage you to go through these links. > Really really thanks. It's very useful. I need to look at it. > > > > > Thanks > > Li > > > > On Thu, Jun 2, 2022 at 4:00 PM Pai G, Sunil wrote: > > > > > > Hi Li, > > > > > > The assumption of ovs being dependent on dpdk for avx512 might not be > > true. > > > I found these two commits below in ovs-2.14.2 which strips out the "- > > march" and "-mno-avx512f" flags exported by dpdk i.e removes dependency on > > DPDK. The reason for this is rightly mentioned below as well. Hope this > > helps. > > > > > > > > > commit bb8f0e2a810889241f1d886d160ccee9b96c4d63 > > > Author: Ian Stokes > > > Date: Fri Jan 15 15:46:02 2021 + > > > > > > acinclude: Strip out -mno-avx512f provided by DPDK. > > > > > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > > > used to build DPDK had broken AVX512 support. > > > > > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > > > used to build DPDK had broken AVX512 support. But OVS could be > > built > > > with a completely different or fixed toolchain with correct avx512 > > > support. > > > > > > Fix that by stripping out `-mno-avx512f` as we already do for '- > > march'. > > > This will allow the OVS to decide if the AVX512 can be used. > > > > > > Reordering of CFLAGS (i.e. adding DPDK flags before OVS ones) is not > > an > > > option since autotools might reorder them back later and it's very > > > unpredictable. > > > > > > Reported-at: https://github.com/openvswitch/ovs-issues/issues/201 > > > Signed-off-by: Ilya Maximets > > > Co-authored-by: Ilya Maximets > > > Signed-off-by: Ian Stokes > > > > > > commit e9f9104d6a83ce7efd702120171835991779 > > > Author: Ian Stokes > > > Date: Fri Jan 15 14:54:04 2021 + > > > > > > acinclude: Strip out -march provided by DPDK. > > > > > > DPDK flags may include -march. Forcing -march could be > > > considered too heavy a requirement when users compile OVS from > > > source and could override user provided options. > > > > > > Resolve this by stripping -march from provided DPDK flags. > > > > > > Signed-off-by: Ian Stokes > > > > > > > > > > > > Thanks and Regards, > > > Sunil > > > > > > > -Original Message- > > > > From: discuss On Behalf Of Li > > > > Zhang > > > > Sent: Thursday, June 2, 2022 6:35 PM > > &
Re: [ovs-discuss] One question about AVX512 support
On Thu, Jun 2, 2022 at 6:56 PM Raymond Burkholder wrote: > > > > On 2022-06-02 10:26, Li Zhang wrote: > > Hi Pai, > > > > Thanks a lot. It helps. > > These patches allow ovs to decide to disable or enable AVX512 if I > > understand it correctly. > > Is it recommended to enable AVX512? AVX512 is enabled by default. > > It is disabled in DPDK because it is experimental. I wonder if I > > should disable it, it may cause performance worse. > > Look at https://en.wikipedia.org/wiki/AVX-512 Got it, thanks. > > AVX512 is CPU model dependent. Code will break if the instructions are > run on a cpu without them. Which is why they are typically not-compiled > in. Each environment will need to make that decision explicit. If your > environment supports the instructions, then I'd probably compile in for > OVS & DPRK. If you need code to transport all over the place, then best > to disable them. > > > > > > Thanks > > Li > > > > On Thu, Jun 2, 2022 at 4:00 PM Pai G, Sunil wrote: > >> Hi Li, > >> > >> The assumption of ovs being dependent on dpdk for avx512 might not be true. > >> I found these two commits below in ovs-2.14.2 which strips out the > >> "-march" and "-mno-avx512f" flags exported by dpdk i.e removes dependency > >> on DPDK. The reason for this is rightly mentioned below as well. Hope this > >> helps. > >> > >> > >> commit bb8f0e2a810889241f1d886d160ccee9b96c4d63 > >> Author: Ian Stokes > >> Date: Fri Jan 15 15:46:02 2021 + > >> > >> acinclude: Strip out -mno-avx512f provided by DPDK. > >> > >> DPDK forces '-mno-avx512f' flag for the application if the toolchain > >> used to build DPDK had broken AVX512 support. > >> > >> DPDK forces '-mno-avx512f' flag for the application if the toolchain > >> used to build DPDK had broken AVX512 support. But OVS could be built > >> with a completely different or fixed toolchain with correct avx512 > >> support. > >> > >> Fix that by stripping out `-mno-avx512f` as we already do for > >> '-march'. > >> This will allow the OVS to decide if the AVX512 can be used. > >> > >> Reordering of CFLAGS (i.e. adding DPDK flags before OVS ones) is not > >> an > >> option since autotools might reorder them back later and it's very > >> unpredictable. > >> > >> Reported-at: https://github.com/openvswitch/ovs-issues/issues/201 > >> Signed-off-by: Ilya Maximets > >> Co-authored-by: Ilya Maximets > >> Signed-off-by: Ian Stokes > >> > >> commit e9f9104d6a83ce7efd702120171835991779 > >> Author: Ian Stokes > >> Date: Fri Jan 15 14:54:04 2021 + > >> > >> acinclude: Strip out -march provided by DPDK. > >> > >> DPDK flags may include -march. Forcing -march could be > >> considered too heavy a requirement when users compile OVS from > >> source and could override user provided options. > >> > >> Resolve this by stripping -march from provided DPDK flags. > >> > >> Signed-off-by: Ian Stokes > >> > >> > >> > >> Thanks and Regards, > >> Sunil > >> > >>> -Original Message- > >>> From: discuss On Behalf Of Li Zhang > >>> Sent: Thursday, June 2, 2022 6:35 PM > >>> To: ovs-discuss@openvswitch.org > >>> Subject: [ovs-discuss] One question about AVX512 support > >>> > >>> Hi all, > >>> > >>> We are using openvswitch 2.14.2, and dpdk-19.11.4. I found avx512 is > >>> enabled by default but it's disabled in DPDK. But I think ovs is dependent > >>> on the dpdk library, right? But why does ovs work with > >>> avx512 disabled in DPDK? > >>> > >>> I am not quite sure about the relationship between OVS and DPDK, any > >>> suggestions? > >>> > >>> -- > >>> > >>> Best Regards > >>> -Li > >>> ___ > >>> discuss mailing list > >>> disc...@openvswitch.org > >>> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > > > > > > ___ > discuss mailing list > disc...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss -- Best Regards -Li ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] One question about AVX512 support
On Thu, Jun 2, 2022 at 6:30 PM Raymond Burkholder wrote: > > > > On 2022-06-02 10:26, Li Zhang wrote: > > Hi Pai, > > > > Thanks a lot. It helps. > > These patches allow ovs to decide to disable or enable AVX512 if I > > understand it correctly. > > Is it recommended to enable AVX512? AVX512 is enabled by default. > > It is disabled in DPDK because it is experimental. I wonder if I > > should disable it, it may cause performance worse. > > AVX512 is CPU model dependent. Code will break if the instructions are > run on a cpu without them. Which is why they are typically not-compiled > in. Each environment will need to make that decision explicit. If your > environment supports the instructions, then I'd probably compile in for > OVS & DPRK. If you need code to transport all over the place, then best > to disable them. Got it, thanks! > > > > > Thanks > > Li > > > > On Thu, Jun 2, 2022 at 4:00 PM Pai G, Sunil wrote: > >> Hi Li, > >> > >> The assumption of ovs being dependent on dpdk for avx512 might not be true. > >> I found these two commits below in ovs-2.14.2 which strips out the > >> "-march" and "-mno-avx512f" flags exported by dpdk i.e removes dependency > >> on DPDK. The reason for this is rightly mentioned below as well. Hope this > >> helps. > >> > >> > >> commit bb8f0e2a810889241f1d886d160ccee9b96c4d63 > >> Author: Ian Stokes > >> Date: Fri Jan 15 15:46:02 2021 + > >> > >> acinclude: Strip out -mno-avx512f provided by DPDK. > >> > >> DPDK forces '-mno-avx512f' flag for the application if the toolchain > >> used to build DPDK had broken AVX512 support. > >> > >> DPDK forces '-mno-avx512f' flag for the application if the toolchain > >> used to build DPDK had broken AVX512 support. But OVS could be built > >> with a completely different or fixed toolchain with correct avx512 > >> support. > >> > >> Fix that by stripping out `-mno-avx512f` as we already do for > >> '-march'. > >> This will allow the OVS to decide if the AVX512 can be used. > >> > >> Reordering of CFLAGS (i.e. adding DPDK flags before OVS ones) is not > >> an > >> option since autotools might reorder them back later and it's very > >> unpredictable. > >> > >> Reported-at: https://github.com/openvswitch/ovs-issues/issues/201 > >> Signed-off-by: Ilya Maximets > >> Co-authored-by: Ilya Maximets > >> Signed-off-by: Ian Stokes > >> > >> commit e9f9104d6a83ce7efd702120171835991779 > >> Author: Ian Stokes > >> Date: Fri Jan 15 14:54:04 2021 + > >> > >> acinclude: Strip out -march provided by DPDK. > >> > >> DPDK flags may include -march. Forcing -march could be > >> considered too heavy a requirement when users compile OVS from > >> source and could override user provided options. > >> > >> Resolve this by stripping -march from provided DPDK flags. > >> > >> Signed-off-by: Ian Stokes > >> > >> > >> > >> Thanks and Regards, > >> Sunil > >> > >>> -Original Message- > >>> From: discuss On Behalf Of Li Zhang > >>> Sent: Thursday, June 2, 2022 6:35 PM > >>> To: ovs-discuss@openvswitch.org > >>> Subject: [ovs-discuss] One question about AVX512 support > >>> > >>> Hi all, > >>> > >>> We are using openvswitch 2.14.2, and dpdk-19.11.4. I found avx512 is > >>> enabled by default but it's disabled in DPDK. But I think ovs is dependent > >>> on the dpdk library, right? But why does ovs work with > >>> avx512 disabled in DPDK? > >>> > >>> I am not quite sure about the relationship between OVS and DPDK, any > >>> suggestions? > >>> > >>> -- > >>> > >>> Best Regards > >>> -Li > >>> ___ > >>> discuss mailing list > >>> disc...@openvswitch.org > >>> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > > > > > -- Best Regards -Li ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] One question about AVX512 support
Hi Pai, Thanks a lot. It helps. These patches allow ovs to decide to disable or enable AVX512 if I understand it correctly. Is it recommended to enable AVX512? AVX512 is enabled by default. It is disabled in DPDK because it is experimental. I wonder if I should disable it, it may cause performance worse. Thanks Li On Thu, Jun 2, 2022 at 4:00 PM Pai G, Sunil wrote: > > Hi Li, > > The assumption of ovs being dependent on dpdk for avx512 might not be true. > I found these two commits below in ovs-2.14.2 which strips out the "-march" > and "-mno-avx512f" flags exported by dpdk i.e removes dependency on DPDK. The > reason for this is rightly mentioned below as well. Hope this helps. > > > commit bb8f0e2a810889241f1d886d160ccee9b96c4d63 > Author: Ian Stokes > Date: Fri Jan 15 15:46:02 2021 + > > acinclude: Strip out -mno-avx512f provided by DPDK. > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > used to build DPDK had broken AVX512 support. > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > used to build DPDK had broken AVX512 support. But OVS could be built > with a completely different or fixed toolchain with correct avx512 > support. > > Fix that by stripping out `-mno-avx512f` as we already do for '-march'. > This will allow the OVS to decide if the AVX512 can be used. > > Reordering of CFLAGS (i.e. adding DPDK flags before OVS ones) is not an > option since autotools might reorder them back later and it's very > unpredictable. > > Reported-at: https://github.com/openvswitch/ovs-issues/issues/201 > Signed-off-by: Ilya Maximets > Co-authored-by: Ilya Maximets > Signed-off-by: Ian Stokes > > commit e9f9104d6a83ce7efd702120171835991779 > Author: Ian Stokes > Date: Fri Jan 15 14:54:04 2021 + > > acinclude: Strip out -march provided by DPDK. > > DPDK flags may include -march. Forcing -march could be > considered too heavy a requirement when users compile OVS from > source and could override user provided options. > > Resolve this by stripping -march from provided DPDK flags. > > Signed-off-by: Ian Stokes > > > > Thanks and Regards, > Sunil > > > -Original Message- > > From: discuss On Behalf Of Li Zhang > > Sent: Thursday, June 2, 2022 6:35 PM > > To: ovs-discuss@openvswitch.org > > Subject: [ovs-discuss] One question about AVX512 support > > > > Hi all, > > > > We are using openvswitch 2.14.2, and dpdk-19.11.4. I found avx512 is > > enabled by default but it's disabled in DPDK. But I think ovs is dependent > > on the dpdk library, right? But why does ovs work with > > avx512 disabled in DPDK? > > > > I am not quite sure about the relationship between OVS and DPDK, any > > suggestions? > > > > -- > > > > Best Regards > > -Li > > ___ > > discuss mailing list > > disc...@openvswitch.org > > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss -- Best Regards -Li ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] One question about AVX512 support
Hi all, We are using openvswitch 2.14.2, and dpdk-19.11.4. I found avx512 is enabled by default but it's disabled in DPDK. But I think ovs is dependent on the dpdk library, right? But why does ovs work with avx512 disabled in DPDK? I am not quite sure about the relationship between OVS and DPDK, any suggestions? -- Best Regards -Li ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
