asubiotto commented on PR #9856: URL: https://github.com/apache/arrow-rs/pull/9856#issuecomment-4386889562
> @asubiotto I made a PR could you check it out? #9919 Hi, thanks for pushing that up. While I think we eventually want to do this I would prefer an incremental approach which is already much better than what we have today (deduping logical runs within the same source). The reason is that while I think we should eventually dedup based on values, I'm not too keen on the slice cost per value and I think we can probably work on a much more performant approach by building and reusing a comparator to reduce the dynamic dispatch overhead. This is why I think we should decouple the two changes: 1) Merge the specialized REE interleave and 2) Optimize the interleave by value deduplication across sources -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
