pepijnve commented on PR #19994: URL: https://github.com/apache/datafusion/pull/19994#issuecomment-3860528711
@CuteChuanChuan just wanted to second the positive sentiment expressed by @alamb. Taking a closer look at this with the profile showed that my assumptions wrt which code path was being taken were wrong and that there was still some low hanging fruit left in the general code paths. I've left some pointers in https://github.com/apache/datafusion/issues/11570#issuecomment-3860506484 regarding possible future work. We might be able to squeeze some better performance out of `scatter` by applying the same specialisation strategy that was applied in `filter` (see https://github.com/apache/arrow-rs/blob/main/arrow-select/src/filter.rs). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
