cetra3 opened a new pull request, #11281: URL: https://github.com/apache/arrow-rs/pull/11281
# Which issue does this PR close? Fixes https://github.com/apache/arrow-rs/issues/11280 And also helps with: https://github.com/apache/datafusion/pull/23565 # Rationale for this change See the original issue, but, basically the interleave kernel can make sub-optimal string views especially when buffers are shared. # What changes are included in this PR? We use a btreemap on the seen buffers (not individual values) and use that to prevent us from cloning the same buffer multiple times. # Are these changes tested? Yes tested, and locally benchmarks have shown a moderate speed increase in some of the interleave benchmarks, on my dev machine which has an **AMD Ryzen 9 9950X3D** processor: | benchmark | main | this PR | change | |-----------------------------------------------------------------------|----------|----------|--------| | interleave str_view(0.0) 100 [0..100, 100..230, 450..1000] | 284.5 ns | 268.6 ns | -5.6% | | interleave str_view(0.0) 400 [0..100, 100..230, 450..1000] | 706.9 ns | 563.9 ns | -20.2% | | interleave str_view(0.0) 1024 [0..100, 100..230, 450..1000] | 1.717 µs | 1.250 µs | -27.2% | | interleave str_view(0.0) 1024 [0..100, 100..230, 450..1000, 0..1000] | 1.783 µs | 1.274 µs | -28.6% | # Are there any user-facing changes? No user facing changes, just an optimization. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
