This is an automated email from the ASF dual-hosted git repository.
MaxGekk pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new c396dc91d41c [SPARK-42839][SQL] Convert _LEGACY_ERROR_TEMP_2003 to an
internal error
c396dc91d41c is described below
commit c396dc91d41cbc92460cfdfe35f83e07774fec8d
Author: AgenticSpark <[email protected]>
AuthorDate: Wed Jun 24 13:04:04 2026 +0200
[SPARK-42839][SQL] Convert _LEGACY_ERROR_TEMP_2003 to an internal error
### What changes were proposed in this pull request?
Convert `_LEGACY_ERROR_TEMP_2003` into an internal error.
`mapSizeExceedArraySizeWhenZipMapError` now returns
`SparkException.internalError(...)` and the `_LEGACY_ERROR_TEMP_2003` entry is
removed from `error-conditions.json`.
### Why are the changes needed?
Part of the effort to assign proper names to legacy error classes
(SPARK-42839). This error is thrown only by `map_zip_with`
(`MapZipWith.assertSizeOfArrayBuffer`) when the number of unique keys exceeds
`ByteArrayMethods.MAX_ROUNDED_ARRAY_LENGTH` (~2^31). That condition is not
reachable from user code, so the error is not user-facing. Per the ticket,
errors that cannot be reproduced from user space should become internal errors
rather than be given a user-facing name.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
No test triggers this error (the array-size limit cannot be reached from
user code, so there is nothing to assert with `checkError()`). Verified that no
references to `_LEGACY_ERROR_TEMP_2003` remain in the tree and that
`error-conditions.json` is still valid JSON with sorted keys.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: GitHub Copilot CLI (Claude Opus 4.8)
Closes #56727 from AgenticSpark/agenticspark/SPARK-42839-on-fork.
Authored-by: AgenticSpark <[email protected]>
Signed-off-by: Max Gekk <[email protected]>
(cherry picked from commit be87c1edf2b4d7126a679ab9d7ba2fcbf6c47d53)
Signed-off-by: Max Gekk <[email protected]>
---
common/utils/src/main/resources/error/error-conditions.json | 5 -----
.../org/apache/spark/sql/errors/QueryExecutionErrors.scala | 10 ++++------
2 files changed, 4 insertions(+), 11 deletions(-)
diff --git a/common/utils/src/main/resources/error/error-conditions.json
b/common/utils/src/main/resources/error/error-conditions.json
index 1cdb48100c6a..09e7002b4c97 100644
--- a/common/utils/src/main/resources/error/error-conditions.json
+++ b/common/utils/src/main/resources/error/error-conditions.json
@@ -10114,11 +10114,6 @@
"Sinks cannot request distribution and ordering in continuous execution
mode."
]
},
- "_LEGACY_ERROR_TEMP_2003" : {
- "message" : [
- "Unsuccessful try to zip maps with <size> unique keys due to exceeding
the array size limit <maxRoundedArrayLength>."
- ]
- },
"_LEGACY_ERROR_TEMP_2017" : {
"message" : [
"not resolved."
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala
index 3178237db2fa..c3873f0b2881 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala
@@ -342,12 +342,10 @@ private[sql] object QueryExecutionErrors extends
QueryErrorsBase with ExecutionE
arithmeticOverflowError("Overflow in function conv()", context = context)
}
- def mapSizeExceedArraySizeWhenZipMapError(size: Int): SparkRuntimeException
= {
- new SparkRuntimeException(
- errorClass = "_LEGACY_ERROR_TEMP_2003",
- messageParameters = Map(
- "size" -> size.toString(),
- "maxRoundedArrayLength" ->
ByteArrayMethods.MAX_ROUNDED_ARRAY_LENGTH.toString()))
+ def mapSizeExceedArraySizeWhenZipMapError(size: Int): SparkException = {
+ SparkException.internalError(
+ s"Unsuccessful try to zip maps with $size unique keys due to exceeding
the array size " +
+ s"limit ${ByteArrayMethods.MAX_ROUNDED_ARRAY_LENGTH}.")
}
def literalTypeUnsupportedError(v: Any): RuntimeException = {
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
