cloud-fan commented on PR #59402:
URL: https://github.com/apache/spark/pull/59402#issuecomment-6210813676
Thanks for the follow-ups. I checked them at d22c01a: the strict-reduction
gate (RewriteDistinctAggregates.scala:264-270), gating the guard on the config
(:340), canEvaluateUnconditionally on guarded children (:345), the updated
normalization scaladoc, the nullable-condition test, and the user-condition
FILTER check in the plan helper all look right. One gap remains on item 1. The
gate compares only total group counts. So when a COUNT shares its IF with
SUM/AVG(DISTINCT) and two or more other conditional counts on the same base
collapse, the shared IF is still split, and its keys are carried twice in the
first aggregate. That is the same case HyukjinKwon raised inline at :266.
<!-- SPARK_DEV_REVIEW_REPLY
{"feedback_id":"conversation:6187820535","thread_id":"conversation:6187820535","verdict_sha256":"893e44f217342c44020b03d75b3a0881e65fe0b20c983181050af3292633aae6"}
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
