ganeshkumar269 commented on code in PR #340:
URL: https://github.com/apache/datafusion-comet/pull/340#discussion_r1589427151
##########
spark/src/test/scala/org/apache/comet/CometCastSuite.scala:
##########
@@ -807,11 +833,22 @@ class CometCastSuite extends CometTestBase with
AdaptiveSparkPlanHelper {
} else {
// Spark 3.2 and 3.3 have a different error message format so we
can't do a direct
// comparison between Spark and Comet.
+ // In the case of CAST_INVALID_INPUT
// Spark message is in format `invalid input syntax for type
TYPE: VALUE`
// Comet message is in format `The value 'VALUE' of the type
FROM_TYPE cannot be cast to TO_TYPE`
// We just check that the comet message contains the same
invalid value as the Spark message
- val sparkInvalidValue =
sparkMessage.substring(sparkMessage.indexOf(':') + 2)
- assert(cometMessage.contains(sparkInvalidValue))
+ // In the case of CAST_OVERFLOW
+ // Spark message is in format `Casting VALUE to TO_TYPE causes
overflow`
+ // Comet message is in format `The value 'VALUE' of the type
FROM_TYPE cannot be cast to TO_TYPE
+ // due to an overflow`
+ // We check if the comet message contains 'overflow'.
+ val sparkInvalidValue = if (sparkMessage.indexOf(':') == -1) {
+ EMPTY_STRING
+ } else {
+ sparkMessage.substring(sparkMessage.indexOf(':') + 2)
+ }
+ assert(
+ cometMessage.contains(sparkInvalidValue) ||
cometMessage.contains("overflow"))
Review Comment:
doesnt look like overflow error message has ':' in it, i ran
spark.sql("select cast(9223372036854775807 as int)").show() in my local on
various spark versions.
**3.4** - [CAST_OVERFLOW] The value 9223372036854775807L of the type
"BIGINT" cannot be cast to "INT" due to an overflow. Use `try_cast` to tolerate
overflow and return NULL instead. If necessary set "spark.sql.ansi.enabled" to
"false" to bypass this error
**3.3** - The value 9223372036854775807L of the type "BIGINT" cannot be cast
to "INT" due to an overflow. Use `try_cast` to tolerate overflow and return
NULL instead. If necessary set "spark.sql.ansi.enabled" to "false" to bypass
this error
**3.2** - Casting 9223372036854775807 to int causes overflow
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
