[
https://issues.apache.org/jira/browse/SPARK-36665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486817#comment-17486817
]
Anton Okolnychyi commented on SPARK-36665:
------------------------------------------
[~kazuyukitanimura] [~viirya] I am afraid this rule breaks {{NOT IN}}
subqueries in {{RewritePredicateSubquery}}.
For example, consider {{Not(EqualNullSafe(Not(InSubquery(...), true)}}. I think
{{NotPropagation}} rewrites this expression so that it becomes
{{Not(EqualNullSafe(InSubquery(...), false)]]}}. As a result,
{{RewritePredicateSubquery}} does not rewrite the predicate as a NULL-aware
left anti join anymore. This leads to a wrong query result.
> Add more Not operator optimizations
> -----------------------------------
>
> Key: SPARK-36665
> URL: https://issues.apache.org/jira/browse/SPARK-36665
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.3.0
> Reporter: Kazuyuki Tanimura
> Assignee: Kazuyuki Tanimura
> Priority: Major
> Fix For: 3.3.0
>
> Attachments: Pasted Graphic 3.png
>
>
> {{BooleanSimplification should be able to do more simplifications for Not
> operators applying following rules}}
> # {{Not(null) == null}}
> ## {{e.g. IsNull(Not(...)) can be IsNull(...)}}
> # {{(Not(a) = b) == (a = Not(b))}}
> ## {{e.g. Not(...) = true can be (...) = false}}
> # {{(a != b) == (a = Not(b))}}
> ## {{e.g. (...) != true can be (...) = false}}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
