[ https://issues.apache.org/jira/browse/CALCITE-5893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ran Tao resolved CALCITE-5893. ------------------------------ Fix Version/s: 1.37.0 Resolution: Fixed Fixed via https://github.com/apache/calcite/commit/42c39a77b9023b75b41e7b77efef9ef8b0b6bb66 thanks [~mbudiu] for reviewing it. > Wrong NULL operand behavior of ARRAY_CONTAINS/ARRAY_EXCEPT/ARRAY_INTERSECT In > Spark Library > ------------------------------------------------------------------------------------------- > > Key: CALCITE-5893 > URL: https://issues.apache.org/jira/browse/CALCITE-5893 > Project: Calcite > Issue Type: Bug > Components: core > Affects Versions: 1.35.0 > Reporter: Ran Tao > Assignee: Ran Tao > Priority: Major > Labels: pull-request-available > Fix For: 1.37.0 > > > The following are the implementations of some array functions in > calcite({*}Spark Library{*}) that are inconsistent with actual spark behavior. > > The reason here is that *null* and *cast (null as xxx)* are treated equally, > and *NullPolicy* acts on these two situations at the same time and returns > null directly. However in spark, the former needs to throw an exception, and > the correct behavior is the latter. (In fact, apache flink also throws an > exception.) we should throw exception in such case to match Spark behavior. > *calcite spark:* > // return null > select array_contains(array[1, 2], null); > // return null > select array_except(array[1, 2, 3], null) > // return null > select array_intersect(array[1,2,3], null) > *actual spark:* > {code:java} > // Cannot resolve "array_contains(array(1, 2), NULL)" due to data type > mismatch: // Null typed values cannot be used as arguments of `array_contains` > spark-sql (default)> select array_contains(array(1, 2), null); {code} > {code:java} > // data type mismatch: Input to function `array_except` should have been two > "ARRAY" with same element type, > // but it's ["ARRAY<INT>", "VOID"] > spark-sql (default)> select array_except(array(1, 2, 3), null); {code} > {code:java} > // data type mismatch: Input to function `array_intersect` should have been > two "ARRAY" with same element type, > // but it's ["ARRAY<INT>", "VOID"] > spark-sql (default)> select array_intersect(array(1,2,3), null); {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)