Avery Qi created SPARK-49900:
--------------------------------
Summary: Avoid validateNullability failure for
BooleanSimplification
Key: SPARK-49900
URL: https://issues.apache.org/jira/browse/SPARK-49900
Project: Spark
Issue Type: Task
Components: Optimizer
Affects Versions: 4.0.0
Reporter: Avery Qi
```
case a And b if Not(a).semanticEquals(b) => If(IsNull(a), Literal.create(null,
a.dataType), FalseLiteral) case a And b if a.semanticEquals(Not(b)) =>
If(IsNull(b), Literal.create(null, b.dataType), FalseLiteral) case a Or b if
Not(a).semanticEquals(b) => If(IsNull(a), Literal.create(null, a.dataType),
TrueLiteral) case a Or b if a.semanticEquals(Not(b)) => If(IsNull(b),
Literal.create(null, b.dataType), TrueLiteral)
```
in BooleanSimplification change expressions from non-nullable to nullable if
a/b is non-nullable. This is a degradation of optimizer and will block further
optimization for non-nullable column/expressions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
