szehon-ho commented on code in PR #58514:
URL: https://github.com/apache/spark/pull/58514#discussion_r3936827792
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/EnsureRequirements.scala:
##########
@@ -851,11 +838,18 @@ case class EnsureRequirements(
joinType: JoinType,
keyOrdering: Ordering[InternalRowComparableWrapper]):
Seq[InternalRowComparableWrapper] = {
val merged = if
(SQLConf.get.getConf(SQLConf.V2_BUCKETING_PARTITION_FILTER_ENABLED)) {
+ // Rows only match within a key group, so a group no output row can come
from is dropped.
Review Comment:
nit: the comment is a bit confusing 'a group no output row can come from is
dropped.' should have a `that`. And second one is a lot of code details.
Suggestion:
```
// Rows with matching join keys are assigned to the same key group. If a
group is absent from one
// side, whether it can produce output depends on which side's unmatched
rows the join preserves.
// Only equi-joins reach this method. For example, a Cross join can reach it
through
// `left CROSS JOIN right ON left.id = right.id`. For partition filtering,
this Cross join is
// treated like an Inner join: groups absent from either side cannot produce
output.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
