[
https://issues.apache.org/jira/browse/SPARK-29600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-29600:
---------------------------------
Description:
{code}
SELECT array_contains(array(0,0.1,0.2,0.3,0.5,0.02,0.033), .2);
{code}
throws Exception in 3.0 where as in 2.3.2 is working fine.
Spark 3.0 output:
{code}
0: jdbc:hive2://10.18.19.208:23040/default> SELECT
array_contains(array(0,0.1,0.2,0.3,0.5,0.02,0.033), .2);
Error: org.apache.spark.sql.AnalysisException: cannot resolve
'array_contains(array(CAST(0 AS DECIMAL(13,3)), CAST(0.1BD AS DECIMAL(13,3)),
CAST(0.2BD AS DECIMAL(13,3)), CAST(0.3BD AS DECIMAL(13,3)), CAST(0.5BD AS
DECIMAL(13,3)), CAST(0.02BD AS DECIMAL(13,3)), CAST(0.033BD AS DECIMAL(13,3))),
0.2BD)' due to data type mismatch: Input to function array_contains should have
been array followed by a value with same element type, but it's
[array<decimal(13,3)>, decimal(1,1)].; line 1 pos 7;
'Project [unresolvedalias(array_contains(array(cast(0 as decimal(13,3)),
cast(0.1 as decimal(13,3)), cast(0.2 as decimal(13,3)), cast(0.3 as
decimal(13,3)), cast(0.5 as decimal(13,3)), cast(0.02 as decimal(13,3)),
cast(0.033 as decimal(13,3))), 0.2), None)]
{code}
Spark 2.3.2 output
{code}
0: jdbc:hive2://10.18.18.214:23040/default> SELECT
array_contains(array(0,0.1,0.2,0.3,0.5,0.02,0.033), .2);
|array_contains(array(CAST(0 AS DECIMAL(13,3)), CAST(0.1 AS DECIMAL(13,3)),
CAST(0.2 AS DECIMAL(13,3)), CAST(0.3 AS DECIMAL(13,3)), CAST(0.5 AS
DECIMAL(13,3)), CAST(0.02 AS DECIMAL(13,3)), CAST(0.033 AS DECIMAL(13,3))),
CAST(0.2 AS DECIMAL(13,3)))|
|true|
1 row selected (0.18 seconds)
{code}
was:
SELECT array_contains(array(0,0.1,0.2,0.3,0.5,0.02,0.033), .2); throws
Exception in 3.0 where as in 2.3.2 is working fine.
Spark 3.0 output:
0: jdbc:hive2://10.18.19.208:23040/default> SELECT
array_contains(array(0,0.1,0.2,0.3,0.5,0.02,0.033), .2);
Error: org.apache.spark.sql.AnalysisException: cannot resolve
'array_contains(array(CAST(0 AS DECIMAL(13,3)), CAST(0.1BD AS DECIMAL(13,3)),
CAST(0.2BD AS DECIMAL(13,3)), CAST(0.3BD AS DECIMAL(13,3)), CAST(0.5BD AS
DECIMAL(13,3)), CAST(0.02BD AS DECIMAL(13,3)), CAST(0.033BD AS DECIMAL(13,3))),
0.2BD)' due to data type mismatch: Input to function array_contains should have
been array followed by a value with same element type, but it's
[array<decimal(13,3)>, decimal(1,1)].; line 1 pos 7;
'Project [unresolvedalias(array_contains(array(cast(0 as decimal(13,3)),
cast(0.1 as decimal(13,3)), cast(0.2 as decimal(13,3)), cast(0.3 as
decimal(13,3)), cast(0.5 as decimal(13,3)), cast(0.02 as decimal(13,3)),
cast(0.033 as decimal(13,3))), 0.2), None)]
Spark 2.3.2 output
0: jdbc:hive2://10.18.18.214:23040/default> SELECT
array_contains(array(0,0.1,0.2,0.3,0.5,0.02,0.033), .2);
|array_contains(array(CAST(0 AS DECIMAL(13,3)), CAST(0.1 AS DECIMAL(13,3)),
CAST(0.2 AS DECIMAL(13,3)), CAST(0.3 AS DECIMAL(13,3)), CAST(0.5 AS
DECIMAL(13,3)), CAST(0.02 AS DECIMAL(13,3)), CAST(0.033 AS DECIMAL(13,3))),
CAST(0.2 AS DECIMAL(13,3)))|
|true|
1 row selected (0.18 seconds)
> array_contains built in function is not backward compatible in 3.0
> ------------------------------------------------------------------
>
> Key: SPARK-29600
> URL: https://issues.apache.org/jira/browse/SPARK-29600
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.0.0
> Reporter: ABHISHEK KUMAR GUPTA
> Priority: Major
>
> {code}
> SELECT array_contains(array(0,0.1,0.2,0.3,0.5,0.02,0.033), .2);
> {code}
> throws Exception in 3.0 where as in 2.3.2 is working fine.
> Spark 3.0 output:
> {code}
> 0: jdbc:hive2://10.18.19.208:23040/default> SELECT
> array_contains(array(0,0.1,0.2,0.3,0.5,0.02,0.033), .2);
> Error: org.apache.spark.sql.AnalysisException: cannot resolve
> 'array_contains(array(CAST(0 AS DECIMAL(13,3)), CAST(0.1BD AS DECIMAL(13,3)),
> CAST(0.2BD AS DECIMAL(13,3)), CAST(0.3BD AS DECIMAL(13,3)), CAST(0.5BD AS
> DECIMAL(13,3)), CAST(0.02BD AS DECIMAL(13,3)), CAST(0.033BD AS
> DECIMAL(13,3))), 0.2BD)' due to data type mismatch: Input to function
> array_contains should have been array followed by a value with same element
> type, but it's [array<decimal(13,3)>, decimal(1,1)].; line 1 pos 7;
> 'Project [unresolvedalias(array_contains(array(cast(0 as decimal(13,3)),
> cast(0.1 as decimal(13,3)), cast(0.2 as decimal(13,3)), cast(0.3 as
> decimal(13,3)), cast(0.5 as decimal(13,3)), cast(0.02 as decimal(13,3)),
> cast(0.033 as decimal(13,3))), 0.2), None)]
> {code}
> Spark 2.3.2 output
> {code}
> 0: jdbc:hive2://10.18.18.214:23040/default> SELECT
> array_contains(array(0,0.1,0.2,0.3,0.5,0.02,0.033), .2);
> |array_contains(array(CAST(0 AS DECIMAL(13,3)), CAST(0.1 AS DECIMAL(13,3)),
> CAST(0.2 AS DECIMAL(13,3)), CAST(0.3 AS DECIMAL(13,3)), CAST(0.5 AS
> DECIMAL(13,3)), CAST(0.02 AS DECIMAL(13,3)), CAST(0.033 AS DECIMAL(13,3))),
> CAST(0.2 AS DECIMAL(13,3)))|
> |true|
> 1 row selected (0.18 seconds)
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
