1fanwang commented on PR #25742: URL: https://github.com/apache/datafusion/pull/25742#issuecomment-5844014595
@kita-renji thanks for re-running the whole differential against 84ef8f7. Your benchmark made the direction clear enough that I went ahead and implemented the grouped-count version as a stacked PR in #25767. That way this PR stays the verified correctness fix and the plan shape can be reviewed on its own. On the same 10M x 10M distinct data, INTERSECT ALL drops from about 2.3s here to about 0.12s there. The catch is that SQL INTERSECT ALL and EXCEPT ALL can no longer be produced as Substrait, since the producer has no Unnest support. The details are in that PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
