hudi-agent commented on code in PR #19903:
URL: https://github.com/apache/hudi/pull/19903#discussion_r3988708517
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/source/reader/function/HoodieCdcSplitReaderFunction.java:
##########
@@ -295,20 +307,23 @@ private ClosableIterator<RowData>
getBaseFileIterator(String path) throws IOExce
conf.get(FlinkOptions.HIVE_STYLE_PARTITIONING)
);
- return RecordIterators.getParquetRecordIterator(
+ int[] requiredPositions = tableState.getRequiredPositions();
+ ClosableIterator<RowData> rows = RecordIterators.getParquetRecordIterator(
internalSchemaManager,
conf.get(FlinkOptions.READ_UTC_TIMEZONE),
true,
HadoopConfigurations.getParquetConf(conf, getHadoopConf()),
- fieldNames,
- fieldTypesArray,
+ tableState.getRowType().getFieldNames().toArray(new String[0]),
+ readFieldTypes,
partObjects,
- tableState.getRequiredPositions(),
+ requiredPositions,
2048,
new org.apache.flink.core.fs.Path(path),
0,
Long.MAX_VALUE,
predicates);
+ return vectorColumnInfo.isEmpty() ? rows
+ : VectorConversionUtils.wrapVectorColumnIterator(rows,
fieldTypes.toArray(new DataType[0]), requiredPositions, vectorColumnInfo);
Review Comment:
🤖 nit: `fieldTypes.toArray(new DataType[0])` and the field-names `toArray`
are rebuilt on every base file open (here, line 298, and line 316) — could you
cache them as transient `String[]`/`DataType[]` fields alongside
`readFieldTypes`, the way `MergeOnReadInputFormat` now does? That would also
make the two implementations read the same.
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag
quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
