tustvold opened a new pull request #1248:
URL: https://github.com/apache/arrow-rs/pull/1248
**Unfortunately I didn't get as much time to work on this today as I hoped,
but I thought I'd get it up for some visibility into what I'm playing around
with**
# Which issue does this PR close?
Closes #TBD
# Rationale for this change
Make filter kernels faster. This relates to the observation underlying #1225
and #1229, that the filter kernels are surprisingly slow for what they are
doing. In many cases the filter kernels on master are slower than converting
the filter to a list of indices and using `take`...
Currently this PR is ~10x faster than master, and I suspect this could be
pushed further.
```
filter u8 time: [47.525 us 47.538 us 47.554 us]
change: [-90.065% -90.056% -90.048%] (p = 0.00 <
0.05)
Performance has improved.
filter u8 high selectivity
time: [2.3351 us 2.3358 us 2.3365 us]
change: [-81.582% -81.570% -81.558%] (p = 0.00 <
0.05)
Performance has improved.
filter u8 low selectivity
time: [1.3204 us 1.3218 us 1.3233 us]
change: [-70.639% -70.572% -70.512%] (p = 0.00 <
0.05)
Performance has improved.
```
# What changes are included in this PR?
This builds on #1228 and adds specialized filter implementations for
primitive array types. I suspect specialized implementations for byte array
types, and dictionaries would likely yield similarly significant benefits and I
may include them in this PR or a follow up.
Aside from the performance benefits, having specialized impls, much like we
do for the `take` kernels will also allow us to take advantage of SIMD
intrinsics either through manual implementation, or just helping the compiler's
auto-vectorization.
# Are there any user-facing changes?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
