LuciferYang commented on code in PR #13006:
URL: https://github.com/apache/gluten/pull/13006#discussion_r4013523280
##########
shims/common/src/main/scala/org/apache/gluten/sql/shims/SparkShims.scala:
##########
@@ -122,9 +122,20 @@ trait SparkShims {
partitionValues: InternalRow,
metadata: Map[String, Any] = Map.empty): Seq[PartitionedFile]
- def structFromAttributes(attrs: Seq[Attribute]): StructType
+ def structFromAttributes(attrs: Seq[Attribute]): StructType =
+ StructType(attrs.map(a => StructField(a.name, a.dataType, a.nullable,
a.metadata)))
- def attributesFromStruct(structType: StructType): Seq[Attribute]
+ def attributesFromStruct(structType: StructType): Seq[Attribute] =
+ structType.fields.map {
+ field => AttributeReference(field.name, field.dataType, field.nullable,
field.metadata)()
+ }
+
+ // https://issues.apache.org/jira/browse/SPARK-40400
Review Comment:
Thanks for catching the name. The blocker is the module layer:
`GlutenExceptionUtil` lives in `gluten-substrait`, and `gluten-substrait`
depends on the shims (`shims-common` plus the active version shim), not the
other way around. The callers here are the four `Spark*Shims` classes, so
having them call `GlutenExceptionUtil` would require the shims to depend on
`gluten-substrait`, which is a circular dependency and won't compile.
`ExceptionUtils` in `shims-common` is the exception helper all four shim
modules can actually reach, so I'll keep `invalidBucketFile` there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
