nsivabalan commented on code in PR #18412:
URL: https://github.com/apache/hudi/pull/18412#discussion_r3013154287
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/BaseHoodieLogRecordReader.java:
##########
@@ -306,8 +332,14 @@ &&
compareTimestamps(logBlock.getLogBlockHeader().get(INSTANT_TIME), GREATER_THA
logBlock.getLogBlockHeader().get(TARGET_INSTANT_TIME);
targetRollbackInstants.add(targetInstantForCommandBlock);
orderedInstantsList.remove(targetInstantForCommandBlock);
- instantToBlocksMap.remove(targetInstantForCommandBlock);
+ List<HoodieLogBlock> rolledBackBlocks =
instantToBlocksMap.remove(targetInstantForCommandBlock);
+ if (rolledBackBlocks != null) {
+ numBlocksRolledBack += rolledBackBlocks.size();
+ }
+ LOG.info("Reading a rollback block with instant {} and target
instant {}",
Review Comment:
We removed per log block info logging and added a summary in L 444.
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/BaseHoodieLogRecordReader.java:
##########
@@ -306,8 +332,14 @@ &&
compareTimestamps(logBlock.getLogBlockHeader().get(INSTANT_TIME), GREATER_THA
logBlock.getLogBlockHeader().get(TARGET_INSTANT_TIME);
targetRollbackInstants.add(targetInstantForCommandBlock);
orderedInstantsList.remove(targetInstantForCommandBlock);
- instantToBlocksMap.remove(targetInstantForCommandBlock);
+ List<HoodieLogBlock> rolledBackBlocks =
instantToBlocksMap.remove(targetInstantForCommandBlock);
+ if (rolledBackBlocks != null) {
+ numBlocksRolledBack += rolledBackBlocks.size();
+ }
+ LOG.info("Reading a rollback block with instant {} and target
instant {}",
+ instantTime, targetInstantForCommandBlock);
} else {
+ LOG.info("Reading a command block with instant {} whose
operation is not supported", instantTime);
Review Comment:
same here.
oh, if we are going to throw, don't we need to do log.error?
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/BaseHoodieLogRecordReader.java:
##########
@@ -343,6 +372,8 @@ &&
compareTimestamps(logBlock.getLogBlockHeader().get(INSTANT_TIME), GREATER_THA
// For compacted blocks COMPACTED_BLOCK_TIMES entry is present under
its headers.
if (firstBlock.getLogBlockHeader().containsKey(COMPACTED_BLOCK_TIMES))
{
+ LOG.info("For instant time {}, compacted block instants are {}",
Review Comment:
same comment as above.
either move this to debug or log in the end.
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/BaseHoodieLogRecordReader.java:
##########
@@ -373,11 +404,14 @@ &&
compareTimestamps(logBlock.getLogBlockHeader().get(INSTANT_TIME), GREATER_THA
validBlockInstants.add(compactedFinalInstantTime);
}
}
+ Collections.reverse(validBlockInstants);
LOG.debug("Number of applied rollback blocks {}", numBlocksRolledBack);
-
+ LOG.info("Total valid instants found are {}. Instants are {}",
validBlockInstants.size(), validBlockInstants);
Review Comment:
same comment as above
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/BaseHoodieLogRecordReader.java:
##########
@@ -316,11 +348,8 @@ &&
compareTimestamps(logBlock.getLogBlockHeader().get(INSTANT_TIME), GREATER_THA
}
}
- if (LOG.isDebugEnabled()) {
- LOG.debug("Ordered instant times seen {}", orderedInstantsList);
- }
-
- int numBlocksRolledBack = 0;
+ LOG.info("Ordered instant times seen {}", orderedInstantsList);
Review Comment:
lets ensure we log just once per file slice. either here or at L444.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
