sunchao commented on PR #57576: URL: https://github.com/apache/spark/pull/57576#issuecomment-5120235554
Thanks @dongjoon-hyun and @viirya. I added comments documenting the `PhysicalAggregation` deduplication guard, why evaluating accuracy is safe after analysis, and the batch-ordering constraints. `Once` is intentional here. The rewrite is idempotent: a successful fusion changes the percentage expression from scalar `DoubleType` to array-valued, so another pass cannot fuse it again. No later built-in optimizer rule creates new scalar `ApproximatePercentile` aggregates. Keeping the rule in its own `Once` batch also preserves the required placement after operator optimization and before `RewriteDistinctAggregates`. I also addressed @peter-toth’s two findings in `4bf75809` and replied on the inline threads. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
