dejankrak-db commented on code in PR #59640:
URL: https://github.com/apache/spark/pull/59640#discussion_r4348279861
##########
common/utils/src/main/resources/error/error-conditions.json:
##########
@@ -3008,6 +3013,48 @@
],
"sqlState" : "42704"
},
+ "FILE_TYPE_IN_ROUTINE_SIGNATURE_UNSUPPORTED_MAP_KEY" : {
+ "message" : [
+ "The FILE type cannot be a map key in the return type of <routineType>
<routineName>. Use FILE only as a map value, array element, or struct field."
+ ],
+ "sqlState" : "42601"
+ },
+ "FILE_TYPE_IN_ROUTINE_SIGNATURE_UNSUPPORTED_QUALIFIER" : {
+ "message" : [
+ "The FILE type in <routineType> <routineName> does not support the
EXTERNAL qualifier. Use FILE without a qualifier."
+ ],
+ "sqlState" : "42601"
+ },
+ "FILE_TYPE_MISSING_QUALIFIER" : {
+ "message" : [
+ "FILE type requires the EXTERNAL qualifier in a column definition. Use
FILE EXTERNAL."
+ ],
+ "sqlState" : "42601"
+ },
+ "FILE_TYPE_UNKNOWN_WRITE" : {
+ "message" : [
+ "Column(s) <colNames> have an ambiguous FILE type. Declare the columns
as FILE EXTERNAL in the target table before writing."
Review Comment:
In 40df55fce0d I took the second option and worded the message for its CTAS
use: "Column(s) <colNames> of the query have an ambiguous FILE type, so they
cannot be stored in a table. A table column must be FILE EXTERNAL, so the query
must produce FILE EXTERNAL values for these columns." I did not add a CAST
hint, as there is no cast between FILE modes yet. INSERT of a FILE into a FILE
EXTERNAL column needs that cast to be storable at all, so I would cover INSERT,
with this error, together with the casts part.
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/resolver/OperatorWithUncomparableTypeValidator.scala:
##########
@@ -20,7 +20,7 @@ package org.apache.spark.sql.catalyst.analysis.resolver
import org.apache.spark.sql.catalyst.expressions.Attribute
import org.apache.spark.sql.catalyst.plans.logical.{Distinct, LogicalPlan,
SetOperation}
import org.apache.spark.sql.errors.QueryCompilationErrors
-import org.apache.spark.sql.types.{DataType, MapType, VariantType}
+import org.apache.spark.sql.types.{DataType, FileType, MapType, VariantType}
/**
Review Comment:
Fixed the comments in 40df55fce0d, without adding Union to the match: the
class doc lists FileType, and the method doc says Union is not a SetOperation,
so UNION ALL is allowed while UNION DISTINCT is rejected through the Distinct
above it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
