Github user nsyca commented on a diff in the pull request:
https://github.com/apache/spark/pull/17428#discussion_r108195342
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala
---
@@ -90,6 +90,10 @@ trait PredicateHelper {
* Returns true iff `expr` could be evaluated as a condition within join.
*/
protected def canEvaluateWithinJoin(expr: Expression): Boolean = expr
match {
+ case l: ListQuery =>
+ // Now pulling up IN predicates is deferred to
`RewritePredicateSubquery`. As a result,
+ // `ListQuery`'s children can be empty before that rule and falls
into the next case.
+ false
--- End diff --
Suggested text in the comment:
[NOT] IN (<ListQuery>) expression cannot be evaluated as part of a Join
operator. Currently the only way to evaluate a [NOT] IN subquery is to convert
it to a LeftSemi or LeftAnti by `RewritePredicateSubquery` rule.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
