ctsk commented on issue #6692: URL: https://github.com/apache/arrow-rs/issues/6692#issuecomment-2734861702
I think a good solution to this is worthwhile to implement -- from what I can see, it could eliminate many uses of CoalesceExec in datafusion. > For the primitive builders one could simply move the current kernel implementations into the builders, but this doesn't really achieve much IMO. Been there, done that 🙈 - for Primitive/Bytes/ByteView Arrays. It's not nice. Since I already did the menial work, I could benchmark the impact it has (when combined with some repartitioning changes in datafusion that take advantage of this). It does achieve avoiding the coalesce step / concatenating the short arrays after repartitioning. The caller also has to adjust (e.g. take care not to exceed the capacity of the builder to avoid resizing). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
