jayshrivastava commented on issue #21207: URL: https://github.com/apache/datafusion/issues/21207#issuecomment-5005667541
@alamb @adriangb Sorry for the infrequent updates. I'm slowly making progress on a dynamic filtering RFC. One interesting thing that has come up is handling the "partition-aware" `CASE` expressions for partitioned hash joins. Do we have any evidence that using the `CASE` helps significantly? The original [PR](https://github.com/apache/datafusion/pull/18451) doesn't have benchmarks. I did an AI-assisted [experiment](https://github.com/jayshrivastava/datafusion/pull/4) in vanilla datafusion which shows that `CASE` expressions are slower because - they use more CPU to evaluate - parquet row group pruning does not support `CASE` expressions In the past, I know there's been discussions about removing the `CASE`. Is this a good enough signal to make that change? I can send a PR if you think so. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
