Xiaoju Wu created SPARK-31872:
---------------------------------
Summary: NotNullSafe to get complementary set
Key: SPARK-31872
URL: https://issues.apache.org/jira/browse/SPARK-31872
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 2.4.0, 2.3.0, 3.0.0
Reporter: Xiaoju Wu
If we have a filter expression to get subset of rows, and then we want the
complementary set, Not(expression) cannot work, since Not is NullIntolerent, if
expression.eval(row) is null, filter predicate is false too for
Not(expression), "row" will not appear in both subset and complementary set.
So, maybe we need a NotNullSafe implementation to get the complementary set
which will result in true if the expression.eval(row) is null.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
