jhorstmann opened a new pull request #9330: URL: https://github.com/apache/arrow/pull/9330
Another experiement for speeding up `ArrayData`. Most array types only contain a single buffer (except for the validity buffer) and also at most one child data object. I think the only types this does not apply to are struct and union arrays. Using a `SmallVec` trades one comparison against an allocation and the resulting indirection in those cases. The `ArrayData::new` method is left in place unchanged for compatibility. TODO: - [ ] Use the optimized `new_smallvec` method in kernels whenever applicable - [ ] Run all benchmarks on a non-throttling machine @jorgecarleitao @Dandandan this is also related to the discussion on #9271. I tried this initially some month ago and the benchmarks were not conclusive, but might be worth trying again or in combination with removing the Arc indirection. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
