voonhous commented on code in PR #19558:
URL: https://github.com/apache/hudi/pull/19558#discussion_r3755261553
##########
hudi-spark-datasource/hudi-spark4.1.x/src/main/scala/org/apache/spark/sql/adapter/Spark4_1Adapter.scala:
##########
@@ -236,6 +236,22 @@ class Spark4_1Adapter extends BaseSpark4Adapter {
VariantMetadata.isVariantStruct(structType)
}
+ override def buildFullVariantReadSchema(schema: StructType):
Option[StructType] = {
+ var rewritten = false
+ val fields = schema.fields.map { f =>
Review Comment:
Deferred and now noted in the comment block. A nested shredded file cannot
be produced today -- the write schema only gets a `typed_value` via the
test-only force config until shredding-schema inference lands -- so there is
nothing for this path to lose yet. The nested leg will need the same treatment
once inference exists.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
