ulysses-you commented on PR #58681:
URL: https://github.com/apache/spark/pull/58681#issuecomment-5616547282
Thanks Dongjoon.
1. Description fixed: `partition.filter.enabled` shipped opt-in in 4.0.0 and
the two ordering
configs in 4.2.0, all released, so this is a behavior change relative to
released Spark. The
"within unreleased branches" wording is gone.
2. 8dcbe648cf6 parameterizes seven of the pinned cases over
`spark.sql.sources.v2.bucketing.partition.filter.enabled`, pinning the
aligned key count for
each setting and asserting that neither setting adds a shuffle:
SPARK-42038 with dynamic
partition filtering (15/6 to 9/3), both SPARK-57881 union cases (5 to 3,
4 to 2), SPARK-55535
granular grouping (Seq(3, 6, 6, 3) to Seq(1, 2, 2, 1)) and partial
clustering
(Seq(6, 5, 5, 6) to Seq(2, 2, 2, 2)), SPARK-48065 (8/4 to 6/2), and
SPARK-56182 (4 to 3).
Two stay pinned: SPARK-55992 asserts the explain string of a padded leg,
and SPARK-59050 pins
the unknown-keyed claim, which is finding 1 in @peter-toth's review.
3. Reworded to "a join that keeps or tests every left row (left outer, left
anti, left single,
existence)", and added that a CROSS join carrying an equality condition
behaves like an inner
join.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
