sunchao opened a new issue, #7317: URL: https://github.com/apache/datafusion-comet/issues/7317
### What is the problem the feature request solves? PR #7190 intentionally limits right-key reduction before a sort-merge join to an unfiltered left-semi join whose right side outputs one signed-integer key. Left-anti joins and joins with eligible residual conditions retain their existing native sort and can still sort duplicate right keys. ### Describe the potential solution Evaluate extending the same key-only reduction to ordinary left-anti joins and eligible residual-condition joins. Spark's `ShuffledHashJoinExec.ignoreDuplicatedKey` already permits duplicate elimination for `LeftExistence` joins with conditions referencing only streamed attributes and build keys. Equal right keys in the key-only schema give identical inputs to a deterministic residual predicate. Keep this extension separate from #7190. Verify Spark and DataFusion semantics with null keys, false/null predicates, duplicate left rows and late matches, and measure the supported join paths before widening eligibility. Do not assume null-aware anti joins or arbitrary payload columns are covered. Review context: https://github.com/apache/datafusion-comet/pull/7190#discussion_r4414769440 ### Willingness to contribute I can help evaluate the extension and its tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
