linliu-code commented on code in PR #9819:
URL: https://github.com/apache/hudi/pull/9819#discussion_r1354336953
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/HoodieFileGroupReader.java:
##########
@@ -146,28 +154,52 @@ public void initRecordIterators() {
* @return {@code true} if the next record exists; {@code false} otherwise.
* @throws IOException on reader error.
*/
- public boolean hasNext() throws IOException {
+ public boolean hasNext() {
+ // Merge records from base file and log files.
+ int baseFileSequenceNo = 0;
while (baseFileIterator.hasNext()) {
T baseRecord = baseFileIterator.next();
- String recordKey = readerContext.getRecordKey(baseRecord,
readerState.baseFileAvroSchema);
- Pair<Option<T>, Map<String, Object>> logRecordInfo =
logFileRecordMapping.remove(recordKey);
- Option<T> resultRecord = logRecordInfo != null
- ? merge(Option.of(baseRecord), Collections.emptyMap(),
logRecordInfo.getLeft(), logRecordInfo.getRight())
- : merge(Option.empty(), Collections.emptyMap(),
Option.of(baseRecord), Collections.emptyMap());
+ Pair<Option<T>, Map<String, Object>> logRecordInfo;
+
+ if (shouldUseRecordPosition) {
Review Comment:
Good point. I also feel if-else is not very clean for new logic. After
adding partial-merging, we will have many readers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
