ulysses-you commented on PR #58942: URL: https://github.com/apache/spark/pull/58942#issuecomment-5805859336
Thanks @peter-toth and @cloud-fan! Both findings of this pass are in 32ddb2663e4. - The waiver asks the join type's duplication capability now, the second gate the producer applies before it spreads a side, so a `FullOuter` pair is refused instead of admitted on its kind. The gate is asked at the waiver and not through `partiallyClusteredJoinType`, since that helper is the producer's entry to key-group checking altogether: a kind turned away there would lose the storage-partitioned join it can still plan without spreading a side. - `satisfiesForPairing`'s doc describes the relationship to the planner's `compatibleAsIs` path without claiming the same predicate. - The producer-boundary test pins the join type alongside the operator kind: an as-of join and a full outer join refuse the same pair, and the inner pair the waiver is for passes. On attribution, one correction to the review's "newly introduced": the full-outer over-admission came in with the waiver itself, which was `ShuffledJoin`-wide before the operator-kind revision, so that revision neither introduced nor closed it. The change is a second commit on top of f95e44a rather than an amend, so its delta reads on its own. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
