[GitHub] [spark] amanomer commented on a change in pull request #26811: [SPARK-29600][SQL] ArrayContains function may return incorrect result for DecimalType
amanomer commented on a change in pull request #26811: [SPARK-29600][SQL]
ArrayContains function may return incorrect result for DecimalType
URL: https://github.com/apache/spark/pull/26811#discussion_r358660693
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala
##
@@ -850,7 +850,7 @@ class DataFrameFunctionsSuite extends QueryTest with
SharedSparkSession {
val errorMsg1 =
s"""
|Input to function array_contains should have been array followed by a
- |value with same element type, but it's [array, decimal(29,29)].
+ |value with same element type, but it's [array, decimal(38,29)].
Review comment:
Above code is creating new expression by updating only right child.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] amanomer commented on a change in pull request #26811: [SPARK-29600][SQL] ArrayContains function may return incorrect result for DecimalType
amanomer commented on a change in pull request #26811: [SPARK-29600][SQL]
ArrayContains function may return incorrect result for DecimalType
URL: https://github.com/apache/spark/pull/26811#discussion_r358659882
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala
##
@@ -850,7 +850,7 @@ class DataFrameFunctionsSuite extends QueryTest with
SharedSparkSession {
val errorMsg1 =
s"""
|Input to function array_contains should have been array followed by a
- |value with same element type, but it's [array, decimal(29,29)].
+ |value with same element type, but it's [array, decimal(38,29)].
Review comment:
https://github.com/apache/spark/blob/1fc353d51a62cb554e6af23dbc9a613e214e3af1/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala#L864-L869
This code is to cast left and right expression one by one. Here,
- `e.childern` is `Seq( array, decimal(29,29))`, and
- `e.inputTypes` will return `Seq(array, decimal(38,29))`
`impicitCast(array, array)` will return `None`, since
`int` can't be casted to `decimal(38,29)`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] amanomer commented on a change in pull request #26811: [SPARK-29600][SQL] ArrayContains function may return incorrect result for DecimalType
amanomer commented on a change in pull request #26811: [SPARK-29600][SQL]
ArrayContains function may return incorrect result for DecimalType
URL: https://github.com/apache/spark/pull/26811#discussion_r358618026
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala
##
@@ -850,7 +850,7 @@ class DataFrameFunctionsSuite extends QueryTest with
SharedSparkSession {
val errorMsg1 =
s"""
|Input to function array_contains should have been array followed by a
- |value with same element type, but it's [array, decimal(29,29)].
+ |value with same element type, but it's [array, decimal(38,29)].
Review comment:
cc @maropu @cloud-fan
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] amanomer commented on a change in pull request #26811: [SPARK-29600][SQL] ArrayContains function may return incorrect result for DecimalType
amanomer commented on a change in pull request #26811: [SPARK-29600][SQL]
ArrayContains function may return incorrect result for DecimalType
URL: https://github.com/apache/spark/pull/26811#discussion_r358076319
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala
##
@@ -850,7 +850,7 @@ class DataFrameFunctionsSuite extends QueryTest with
SharedSparkSession {
val errorMsg1 =
s"""
|Input to function array_contains should have been array followed by a
- |value with same element type, but it's [array, decimal(29,29)].
+ |value with same element type, but it's [array, decimal(38,29)].
Review comment:
cc @cloud-fan @maropu
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] amanomer commented on a change in pull request #26811: [SPARK-29600][SQL] ArrayContains function may return incorrect result for DecimalType
amanomer commented on a change in pull request #26811: [SPARK-29600][SQL]
ArrayContains function may return incorrect result for DecimalType
URL: https://github.com/apache/spark/pull/26811#discussion_r358617660
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala
##
@@ -850,7 +850,7 @@ class DataFrameFunctionsSuite extends QueryTest with
SharedSparkSession {
val errorMsg1 =
s"""
|Input to function array_contains should have been array followed by a
- |value with same element type, but it's [array, decimal(29,29)].
+ |value with same element type, but it's [array, decimal(38,29)].
Review comment:
Do you mean why in above test case query, `ArrayContains` is throwing
`AnalysisException` instead of casting integer to Decimal?
An integer cannot be casted to decimal with scale > 28.
```
decimalWith28Zeroes = 1.
SELECT array_contains(array(1), decimalWith28Zeroes);
Result =>> true
```
```
decimalWith29Zeroes = 1.0
SELECT array_contains(array(1), decimalWith29Zeroes);
Result =>> AnalysisException
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] amanomer commented on a change in pull request #26811: [SPARK-29600][SQL] ArrayContains function may return incorrect result for DecimalType
amanomer commented on a change in pull request #26811: [SPARK-29600][SQL]
ArrayContains function may return incorrect result for DecimalType
URL: https://github.com/apache/spark/pull/26811#discussion_r358076319
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala
##
@@ -850,7 +850,7 @@ class DataFrameFunctionsSuite extends QueryTest with
SharedSparkSession {
val errorMsg1 =
s"""
|Input to function array_contains should have been array followed by a
- |value with same element type, but it's [array, decimal(29,29)].
+ |value with same element type, but it's [array, decimal(38,29)].
Review comment:
cc @cloud-fan @maropu
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] amanomer commented on a change in pull request #26811: [SPARK-29600][SQL] ArrayContains function may return incorrect result for DecimalType
amanomer commented on a change in pull request #26811: [SPARK-29600][SQL]
ArrayContains function may return incorrect result for DecimalType
URL: https://github.com/apache/spark/pull/26811#discussion_r358076319
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala
##
@@ -850,7 +850,7 @@ class DataFrameFunctionsSuite extends QueryTest with
SharedSparkSession {
val errorMsg1 =
s"""
|Input to function array_contains should have been array followed by a
- |value with same element type, but it's [array, decimal(29,29)].
+ |value with same element type, but it's [array, decimal(38,29)].
Review comment:
cc @cloud-fan
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] amanomer commented on a change in pull request #26811: [SPARK-29600][SQL] ArrayContains function may return incorrect result for DecimalType
amanomer commented on a change in pull request #26811: [SPARK-29600][SQL]
ArrayContains function may return incorrect result for DecimalType
URL: https://github.com/apache/spark/pull/26811#discussion_r358076174
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala
##
@@ -850,7 +850,7 @@ class DataFrameFunctionsSuite extends QueryTest with
SharedSparkSession {
val errorMsg1 =
s"""
|Input to function array_contains should have been array followed by a
- |value with same element type, but it's [array, decimal(29,29)].
+ |value with same element type, but it's [array, decimal(38,29)].
Review comment:
https://github.com/apache/spark/blob/1fc353d51a62cb554e6af23dbc9a613e214e3af1/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala#L864-L869
For query `array_contains(array(1), .01234567890123456790123456780)`\
`e.inputTypes` will return `Seq(Array(Decimal(38,29)), Decimal(38,29))` and
above code will cast `.01234567890123456790123456780` as `Decimal(38,29)`.
Previously, when we were using `findWiderTypeForTwo`, decimal types were not
getting upcasted but `findWiderTypeWithoutStringPromotionForTwo` will
successfully upcast DecimalType
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
