[dpdk-dev] [PATCH v1 1/2] rte_memcmp functions using Intel AVX and SSE intrinsics
v1: This patch adds memcmp functionality using AVX and SSE intrinsics provided by Intel. For other architectures supported by DPDK regular memcmp function is used. Compiled and tested on Ubuntu 14.04(non-NUMA) and 15.10(NUMA) systems. Signed-off-by: Ravi Kerur --- .../common/include/arch/arm/rte_memcmp.h | 60 ++ .../common/include/arch/ppc_64/rte_memcmp.h| 62 ++ .../common/include/arch/tile/rte_memcmp.h | 60 ++ .../common/include/arch/x86/rte_memcmp.h | 786 + lib/librte_eal/common/include/generic/rte_memcmp.h | 175 + 5 files changed, 1143 insertions(+) create mode 100644 lib/librte_eal/common/include/arch/arm/rte_memcmp.h create mode 100644 lib/librte_eal/common/include/arch/ppc_64/rte_memcmp.h create mode 100644 lib/librte_eal/common/include/arch/tile/rte_memcmp.h create mode 100644 lib/librte_eal/common/include/arch/x86/rte_memcmp.h create mode 100644 lib/librte_eal/common/include/generic/rte_memcmp.h diff --git a/lib/librte_eal/common/include/arch/arm/rte_memcmp.h b/lib/librte_eal/common/include/arch/arm/rte_memcmp.h new file mode 100644 index 000..fcbacb4 --- /dev/null +++ b/lib/librte_eal/common/include/arch/arm/rte_memcmp.h @@ -0,0 +1,60 @@ +/* + * BSD LICENSE + * + * Copyright(c) 2016 RehiveTech. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of IBM Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +*/ + +#ifndef _RTE_MEMCMP_ARM_H_ +#define _RTE_MEMCMP_ARM_H_ + +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +#include "generic/rte_memcmp.h" + +#define rte_memcmp(dst, src, n) \ + ({ (__builtin_constant_p(n)) ? \ + memcmp((dst), (src), (n)) : \ + rte_memcmp_func((dst), (src), (n)); }) + +static inline bool +rte_memcmp_func(void *dst, const void *src, size_t n) +{ + return memcmp(dst, src, n); +} + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_MEMCMP_ARM_H_ */ diff --git a/lib/librte_eal/common/include/arch/ppc_64/rte_memcmp.h b/lib/librte_eal/common/include/arch/ppc_64/rte_memcmp.h new file mode 100644 index 000..5839a2d --- /dev/null +++ b/lib/librte_eal/common/include/arch/ppc_64/rte_memcmp.h @@ -0,0 +1,62 @@ +/* + * BSD LICENSE + * + * Copyright (C) IBM Corporation 2016. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of IBM Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIA
[dpdk-dev] [PATCH v1 1/2] rte_memcmp functions using Intel AVX and SSE intrinsics
2016-03-07 15:00, Ravi Kerur: > v1: > This patch adds memcmp functionality using AVX and SSE > intrinsics provided by Intel. For other architectures > supported by DPDK regular memcmp function is used. Anyone to review this patch please? Zhihong?
[dpdk-dev] [PATCH v1 1/2] rte_memcmp functions using Intel AVX and SSE intrinsics
> -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Ravi Kerur > Sent: Tuesday, March 8, 2016 7:01 AM > To: dev at dpdk.org > Subject: [dpdk-dev] [PATCH v1 1/2] rte_memcmp functions using Intel AVX and > SSE intrinsics > > v1: > This patch adds memcmp functionality using AVX and SSE > intrinsics provided by Intel. For other architectures > supported by DPDK regular memcmp function is used. > > Compiled and tested on Ubuntu 14.04(non-NUMA) and 15.10(NUMA) > systems. > [...] > + if (unlikely(!_mm_testz_si128(xmm2, xmm2))) { > + __m128i idx = > + _mm_setr_epi8(15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, > 3, 2, 1, 0); line over 80 characters ;) > + > + /* > + * Reverse byte order > + */ > + xmm0 = _mm_shuffle_epi8(xmm0, idx); > + xmm1 = _mm_shuffle_epi8(xmm1, idx); > + > + /* > + * Compare unsigned bytes with instructions for signed bytes > + */ > + xmm0 = _mm_xor_si128(xmm0, _mm_set1_epi8(0x80)); > + xmm1 = _mm_xor_si128(xmm1, _mm_set1_epi8(0x80)); > + > + return _mm_movemask_epi8(xmm0 > xmm1) - > _mm_movemask_epi8(xmm1 > xmm0); > + } > + > + return 0; > +} [...] > +static inline int > +rte_memcmp(const void *_src_1, const void *_src_2, size_t n) > +{ > + const uint8_t *src_1 = (const uint8_t *)_src_1; > + const uint8_t *src_2 = (const uint8_t *)_src_2; > + int ret = 0; > + > + if (n < 16) > + return rte_memcmp_regular(src_1, src_2, n); [...] > + > + while (n > 512) { > + ret = rte_cmp256(src_1 + 0 * 256, src_2 + 0 * 256); Thanks for the great work! Seems to me there's a big improvement area before going into detailed instruction layout tuning that -- No unalignment handling here for large size memcmp. So almost without a doubt the performance will be low in micro-architectures like Sandy Bridge if the start address is unaligned, which might be a common case. > + if (unlikely(ret != 0)) > + return ret; > + > + ret = rte_cmp256(src_1 + 1 * 256, src_2 + 1 * 256); > + if (unlikely(ret != 0)) > + return ret; > + > + src_1 = src_1 + 512; > + src_2 = src_2 + 512; > + n -= 512; > + } > + goto CMP_BLOCK_LESS_THAN_512; > +} > + > +#else /* RTE_MACHINE_CPUFLAG_AVX2 */
Re: [dpdk-dev] [PATCH v1 1/2] rte_memcmp functions using Intel AVX and SSE intrinsics
On 5/26/2016 9:57 AM, zhihong.wang at intel.com (Wang, Zhihong) wrote: >> -Original Message- >> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Ravi Kerur >> Sent: Tuesday, March 8, 2016 7:01 AM >> To: dev at dpdk.org >> Subject: [dpdk-dev] [PATCH v1 1/2] rte_memcmp functions using Intel AVX and >> SSE intrinsics >> >> v1: >> This patch adds memcmp functionality using AVX and SSE >> intrinsics provided by Intel. For other architectures >> supported by DPDK regular memcmp function is used. >> >> Compiled and tested on Ubuntu 14.04(non-NUMA) and 15.10(NUMA) >> systems. >> > [...] > >> +if (unlikely(!_mm_testz_si128(xmm2, xmm2))) { >> +__m128i idx = >> +_mm_setr_epi8(15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, >> 3, 2, 1, 0); > > line over 80 characters ;) > >> + >> +/* >> + * Reverse byte order >> + */ >> +xmm0 = _mm_shuffle_epi8(xmm0, idx); >> +xmm1 = _mm_shuffle_epi8(xmm1, idx); >> + >> +/* >> +* Compare unsigned bytes with instructions for signed bytes >> +*/ >> +xmm0 = _mm_xor_si128(xmm0, _mm_set1_epi8(0x80)); >> +xmm1 = _mm_xor_si128(xmm1, _mm_set1_epi8(0x80)); >> + >> +return _mm_movemask_epi8(xmm0 > xmm1) - >> _mm_movemask_epi8(xmm1 > xmm0); >> +} >> + >> +return 0; >> +} > > [...] > >> +static inline int >> +rte_memcmp(const void *_src_1, const void *_src_2, size_t n) >> +{ >> +const uint8_t *src_1 = (const uint8_t *)_src_1; >> +const uint8_t *src_2 = (const uint8_t *)_src_2; >> +int ret = 0; >> + >> +if (n < 16) >> +return rte_memcmp_regular(src_1, src_2, n); > [...] >> + >> +while (n > 512) { >> +ret = rte_cmp256(src_1 + 0 * 256, src_2 + 0 * 256); > > Thanks for the great work! > > Seems to me there's a big improvement area before going into detailed > instruction layout tuning that -- No unalignment handling here for large > size memcmp. > > So almost without a doubt the performance will be low in micro-architectures > like Sandy Bridge if the start address is unaligned, which might be a > common case. Patch is waiting for comment for a long time, since 2016 May. Updating patch status as rejected. Anyone planning to work on vectorized version of rte_memcmp() can benefit from this patch: https://patches.dpdk.org/patch/11156/ https://patches.dpdk.org/patch/11157/