This is an automated email from the ASF dual-hosted git repository.
maxgekk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 9cbc2d138a1 [SPARK-45710][SQL] Assign names to error
_LEGACY_ERROR_TEMP_21[59,60,61,62]
9cbc2d138a1 is described below
commit 9cbc2d138a1de2d3ff399d224d817554ba0b9e18
Author: dengziming
AuthorDate: Sun Nov 5 20:07:07 2023 +0300
[SPARK-45710][SQL] Assign names to error _LEGACY_ERROR_TEMP_21[59,60,61,62]
### What changes were proposed in this pull request?
This PR are removing `_LEGACY_ERROR_TEMP_21[59,60,61,62]` and
`TOO_MANY_ARRAY_ELEMENTS`:
1. `_LEGACY_ERROR_TEMP_2159` is used in concat/array_insert;
2. `_LEGACY_ERROR_TEMP_2160` is only used in flatten;
3. `_LEGACY_ERROR_TEMP_2161` is used in
array_repeat/array_insert/array_distinct/array_union/array_intersect/array_remove;
4. `_LEGACY_ERROR_TEMP_2162` is used in array_union/array_distinct;
5. There is another similar error class `TOO_MANY_ARRAY_ELEMENTS` which are
used in `UnsafeArrayWriter.java`.
I removed these 5 similar error classes and create a new error class
`COLLECTION_SIZE_LIMIT_EXCEEDED` with 3 sub-classes:
1. `PARAMETER` is used when the parameter exceed size limit, such as
`array_repeat` with count too large;
6. `FUNCTION` is used when trying to create an array exceeding size limit
in a function, for example, flatten 2 arrays to a larger array;
7. `INITIALIZE` is used in `UnsafeArrayWriter.java` when trying to
initialize an array exceeding size limit.
### Why are the changes needed?
To assign proper name as a part of activity in SPARK-37935.
### Does this PR introduce _any_ user-facing change?
Yes, the error message will include the error class name.
### How was this patch tested?
1. `COLLECTION_SIZE_LIMIT_EXCEEDED.PARAMETER` can be tested from use code;
2. `COLLECTION_SIZE_LIMIT_EXCEEDED.FUNCTION` is tested using a
`ColumnarArray` in `concat/flatten`, but can't be tested in
`array_insert/array_distinct/array_union/array_intersect/array_remove` since we
need to deduplicate the data and create an array which will cause OOM.
3. `INITIALIZE` is already tested in a existing case.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #43567 from dengziming/SPARK-45710.
Authored-by: dengziming
Signed-off-by: Max Gekk
---
.../src/main/resources/error/error-classes.json| 49 ++-
docs/_data/menu-sql.yaml | 2 +
...s-collection-size-limit-exceeded-error-class.md | 38 +
docs/sql-error-conditions.md | 14 ++--
.../expressions/codegen/UnsafeArrayWriter.java | 3 +-
.../expressions/collectionOperations.scala | 95 --
.../spark/sql/errors/QueryExecutionErrors.scala| 48 +--
.../codegen/UnsafeArrayWriterSuite.scala | 6 +-
.../sql/errors/QueryExecutionErrorsSuite.scala | 59 +-
9 files changed, 184 insertions(+), 130 deletions(-)
diff --git a/common/utils/src/main/resources/error/error-classes.json
b/common/utils/src/main/resources/error/error-classes.json
index 8b0951a7b00..3e0743d366a 100644
--- a/common/utils/src/main/resources/error/error-classes.json
+++ b/common/utils/src/main/resources/error/error-classes.json
@@ -405,6 +405,29 @@
],
"sqlState" : "42704"
},
+ "COLLECTION_SIZE_LIMIT_EXCEEDED" : {
+"message" : [
+ "Can't create array with elements which exceeding the
array size limit ,"
+],
+"subClass" : {
+ "FUNCTION" : {
+"message" : [
+ "unsuccessful try to create arrays in the function ."
+]
+ },
+ "INITIALIZE" : {
+"message" : [
+ "cannot initialize an array with specified parameters."
+]
+ },
+ "PARAMETER" : {
+"message" : [
+ "the value of parameter(s) in the function
is invalid."
+]
+ }
+},
+"sqlState" : "54000"
+ },
"COLUMN_ALIASES_IS_NOT_ALLOWED" : {
"message" : [
"Columns aliases are not allowed in ."
@@ -3017,12 +3040,6 @@
],
"sqlState" : "428EK"
},
- "TOO_MANY_ARRAY_ELEMENTS" : {
-"message" : [
- "Cannot initialize array with elements of size ."
-],
-"sqlState" : "54000"
- },
"UDTF_ALIAS_NUMBER_MISMATCH" : {
"message" : [
"The number of aliases supplied in the AS clause does not match the
number of columns output by the UDTF.",
@@ -5765,26 +5782,6 @@
" is not annotated with SQLUserDefinedType nor registered
with UDTRegistration.}"
]
},
- "_LEGACY_ERROR_TEMP_2159" : {
-"message" : [
- "Unsuccessful try to concat arrays with elements due
to exceeding the array size limit ."
-]
- },
- "_LEGACY_ERROR_TEMP_2160" : {
-"message"
