LuciferYang commented on code in PR #13006:
URL: https://github.com/apache/gluten/pull/13006#discussion_r4004054611
##########
shims/common/src/main/scala/org/apache/gluten/sql/shims/SparkShims.scala:
##########
@@ -122,9 +122,20 @@ trait SparkShims {
partitionValues: InternalRow,
metadata: Map[String, Any] = Map.empty): Seq[PartitionedFile]
- def structFromAttributes(attrs: Seq[Attribute]): StructType
+ def structFromAttributes(attrs: Seq[Attribute]): StructType =
+ StructType(attrs.map(a => StructField(a.name, a.dataType, a.nullable,
a.metadata)))
- def attributesFromStruct(structType: StructType): Seq[Attribute]
+ def attributesFromStruct(structType: StructType): Seq[Attribute] =
+ structType.fields.map {
+ field => AttributeReference(field.name, field.dataType, field.nullable,
field.metadata)()
+ }
+
+ // https://issues.apache.org/jira/browse/SPARK-40400
Review Comment:
Makes sense. Moved it to the existing
`org.apache.gluten.utils.ExceptionUtils` in `shims/common` (there is no
`GlutenExceptionUtils` in the tree, and that is where the shared exception
helpers already live, e.g. `hasCause`). The four shims now call
`ExceptionUtils.invalidBucketFile(...)`, and it is off the trait entirely.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
