0lai0 commented on code in PR #5614:
URL: https://github.com/apache/datafusion-comet/pull/5614#discussion_r3914051112
##########
spark/src/main/spark-3.4/org/apache/spark/sql/comet/shims/ShimSparkErrorConverter.scala:
##########
@@ -158,10 +158,21 @@ trait ShimSparkErrorConverter {
Some(QueryExecutionErrors.exceedMapSizeLimitError(params("size").toString.toInt))
case "CollectionSizeLimitExceeded" =>
- // createArrayWithElementsExceedLimitError takes (count: Any) in Spark
3.4
+ // createArrayWithElementsExceedLimitError takes (count: Any) in Spark
3.4; pass the
+ // decimal string through since the reported length can exceed Long
range.
Some(
QueryExecutionErrors.createArrayWithElementsExceedLimitError(
- params("numElements").toString.toLong))
+ params("numElements").toString))
+
+ case "SequenceIllegalBoundaries" =>
+ // Spark 3.x codegen throws a plain IllegalArgumentException for
sequence boundaries.
+ Some(
+ new IllegalArgumentException(
+ s"Illegal sequence boundaries: ${params("start")} to
${params("stop")} " +
+ s"by ${params("step")}"))
+
+ case "Internal" =>
Review Comment:
Oh thats a good point, this is indeed a user-visible change beyond sequence.
I have added a bullet to the PR description under What changes are included
calling out that the new Internal arm affects ~20 existing native expressions
and changes the message from SparkException(message, <text>) to
[INTERNAL_ERROR] <text>. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
