tschwarzinger opened a new pull request, #11541: URL: https://github.com/apache/arrow-rs/pull/11541
# Which issue does this PR close? - Related to https://github.com/apache/arrow-rs/issues/9670 - Related to https://github.com/apache/datafusion/issues/26169 as I found on my machine polars is more performant for simple parquet scans for the tested files. # Rationale for this change In the earlier version, the max dictionary id was computed over chunks which were only 32 `i32` values. The new version computes the max of the entire `idx` slice with the hope that the CPU can make better use of SIMD instructions. # What changes are included in this PR? - Hoist `max_idx` out of the gather loop. # Are these changes tested? - existing tests # Are there any significant user-facing changes? - Slightly faster decoding -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
