tustvold commented on pull request #1248: URL: https://github.com/apache/arrow-rs/pull/1248#issuecomment-1026921010
> The filter benchmarks are also a bit focused on bytes, I don't know how common those are in DataFusion, but I would think that 32 or 64 bit numbers are much more common and should be the focus of benchmarking. Yeah, the f32 benchmarks are encouraging that the performance uplift isn't just a quirk of the fact the benchmarks currently use u8 for integers. > On recent intel machines there would be a much faster way using the pext instruction. That basically implements a filter for 64 bits at a time. Yeah, I do not doubt some artisanal use of intrinsics could likely eliminate much of the cost of bitmasks, perhaps even surpassing indices. I had I guess hoped that it would be possible to get further before resorting to the arcane :laughing: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
