On Fri, 12 Feb 2021 20:47:21 -0800 Micah Kornfield <emkornfi...@gmail.com> wrote: > That is unfortunate, like I said if the consensus is xsimd, let's move > forward with that.
I would say it's a soft consensus for now, and I would welcome more viewpoints on the matter. Regards Antoine. > > On Fri, Feb 12, 2021 at 2:45 AM Antoine Pitrou <anto...@python.org> wrote: > > > > > There is an std::simd being envisioned. > > https://en.cppreference.com/w/cpp/experimental/simd/simd > > > > The problem is that we need an implementation that's C++11- or > > C++14-compliant, that works on major compilers, and that provides > > accelerations for common instruction sets. It doesn't seem to be the > > case currently. > > https://github.com/VcDevel/std-simd > > > > Regards > > > > Antoine. > > > > > > Le 12/02/2021 à 05:18, Micah Kornfield a écrit : > > > I'm open to x-simd if others think it is the best option. I think the > > last > > > time this came up I expressed this opinion, but if possible it would be > > > nice to use something that is on its way to become a standard to avoid > > > abandonment issues but I don't know enough about the space to understand > > if > > > this is a real concern. > > > > > > On Tue, Feb 9, 2021 at 7:00 PM Yuqi Gu <yuqi...@linaro.org> wrote: > > > > > >> Thanks for comments on the SIMD related PR: > > >> https://github.com/apache/arrow/pull/9424. > > >> Agree to adopt the *xsimd *as the SIMD wrapper library for Arrow to > > avoid a > > >> large maintenance burden. It makes sense. > > >> > > >> It seems *ximd *is designed for mathematics calculating and it lacks the > > >> functions like bit/byte shuffling, byte stream split encoding, ARM SVE > > >> supporting, etc. > > >> I'm absolutely willing to contribute the missing functions to *xsimd*. > > >> > > >> BRs, > > >> Yuqi > > >> > > >> On Tue, 9 Feb 2021 at 22:08, Antoine Pitrou <anto...@python.org> wrote: > > >> > > >>> > > >>> Le 09/02/2021 à 10:36, Antoine Pitrou a écrit : > > >>>> > > >>>> Note that we need to decouple the SIMD level available at compile-time > > >>>> from the SIMD level available at runtime. That is, we typically build > > >>>> optional AVX512 accelerations at compile-time, but only enable them at > > >>>> runtime if the CPU supports AVX512 (and if the environment variable > > >>>> ARROW_USER_SIMD_LEVEL wasn't forced to a lower value). > > >>>> > > >>>> From a quick glance, it's not obvious that xsimd supports that level > > of > > >>>> control. Though it may just be undocumented. I will check with the > > >>>> authors, since I happen to know them. > > >>> > > >>> Ok, there shouldn't be any problem on that front. We just need to > > >>> compile with the right compiler flags to select the desired SIMD level, > > >>> like we already do currently when compiling multiple versions of a > > >>> function. > > >>> > > >>> I'll note that xsimd isn't very complete. For example, it seems to > > lack > > >>> the functions required for byte stream split encoding and decoding. > > >>> Those functions are exported by libsimdpp under the names "zip_lo" and > > >>> "zip_hi". > > >>> > > >>> libsimdpp, on the other hand, seems to lack maintenance. It hasn't had > > >>> a commit in one year, and issues and PRs seem to be unanswered. So > > >>> perhaps xsimd is a better course, provided we want to contribute the > > >>> missing functions. > > >>> > > >>> Regards > > >>> > > >>> Antoine. > > >>> > > >> > > > > > >
