sunchao commented on code in PR #7190:
URL: https://github.com/apache/datafusion-comet/pull/7190#discussion_r4415802031
##########
spark/src/main/scala/org/apache/spark/sql/comet/operators.scala:
##########
@@ -3646,6 +3667,9 @@ object CometSortMergeJoinExec extends
CometOperatorSerde[SortMergeJoinExec] {
val joinBuilder = OperatorOuterClass.SortMergeJoin
.newBuilder()
.setJoinType(joinType)
+ .setRightKeyReductionEnabled(
+
CometConf.COMET_EXEC_SORT_MERGE_JOIN_RIGHT_KEY_REDUCTION_ENABLED.get(join.conf)
&&
+ join.joinType == LeftSemi && join.condition.isEmpty &&
canReduceRightKey(join))
Review Comment:
Yes, these were deliberately left out of the initial scope. Your key-only
argument is valid for ordinary left-anti joins and eligible residual
conditions; the current guards leave those cases on their existing native sort,
so this is an opportunity to extend coverage rather than a correctness problem
in the supported unfiltered left-semi path.
I added that scope statement to the description and tuning guide and filed
#7317 to track the extension, including the required null/predicate and
duplicate-left-row controls. I kept both eligibility guards unchanged in this
PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
