leesf commented on a change in pull request #4514: URL: https://github.com/apache/hudi/pull/4514#discussion_r780047010
########## File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DefaultSource.scala ########## @@ -177,7 +177,7 @@ class DefaultSource extends RelationProvider outputMode) } - override def shortName(): String = "hudi" + override def shortName(): String = "hudi_v1" Review comment: it is because in hudi-spark-bundle module. I used `<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer"> <resource>META-INF/services/org.apache.spark.sql.sources.DataSourceRegister</resource> </transformer>` to append the formats(hudi_v1 and hudi) in `DataSourceRegister` file, so it will conflict if not change the format. As to the PR itself, we do not need to change the format to `hudi_v1` and not use AppendingTransformer. But when implementing V2 codepath, I find it difficult to handle the incremental bootstrap table(https://github.com/apache/hudi/blob/ac8d014a0602e3c499771f3313f0f88de57cdda1/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/IncrementalRelation.scala#L175) as it will pass schema to Spark and difficult to handle in v2 codepath, after finding a good way to handle it, we would definitely delete the hudi_v1 format here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org