voonhous commented on code in PR #20136:
URL: https://github.com/apache/hudi/pull/20136#discussion_r4143167133
##########
hudi-client/hudi-spark-client/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/HoodieParquetReadSupport.scala:
##########
@@ -62,12 +66,26 @@ object HoodieParquetReadSupport {
*
* @param requestedSchema the initial parquet schema requested by Spark
* @param fileSchema the actual parquet schema of the file
+ * @param dropMissingTopLevelFields whether a top-level field the file does
not have is dropped as
+ * well (the row-based reader) or kept (the
vectorized reader)
* @return a potentially updated schema with empty struct fields removed
*/
- def trimParquetSchema(requestedSchema: MessageType, fileSchema:
MessageType): MessageType = {
+ def trimParquetSchema(requestedSchema: MessageType,
+ fileSchema: MessageType,
+ dropMissingTopLevelFields: Boolean): MessageType = {
val trimmedFields = requestedSchema.getFields.asScala.map(field => {
if (fileSchema.containsField(field.getName)) {
Review Comment:
Trimmed to one line; the rationale moved to the method doc (cbdd8de95443).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
