jtao15 commented on a change in pull request #7424:
URL: https://github.com/apache/pinot/pull/7424#discussion_r707593664
##########
File path:
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/retention/RetentionManager.java
##########
@@ -246,8 +251,12 @@ private void manageSegmentLineageCleanupForTable(String
tableNameWithType) {
.writeSegmentLineage(_pinotHelixResourceManager.getPropertyStore(),
segmentLineage, expectedVersion)) {
// Delete segments based on the segment lineage
_pinotHelixResourceManager.deleteSegments(tableNameWithType,
segmentsToDelete);
+ LOGGER.info("Finished cleaning up segment lineage for table: {} in
{}ms", tableNameWithType,
+ (System.currentTimeMillis() - cleanupStartTime));
return true;
} else {
+ LOGGER.info("Failed to write segment lineage back when cleaning up
segment lineage for table: {}",
Review comment:
This could be a `warn`? With current logic, this usually should not
happen. This can only happen when `startSegmentReplace` or `endSegmentReplace`
is called at the same time when retention process is running?
##########
File path:
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/retention/RetentionManager.java
##########
@@ -80,7 +80,13 @@ public RetentionManager(PinotHelixResourceManager
pinotHelixResourceManager,
@Override
protected void processTable(String tableNameWithType) {
LOGGER.info("Start managing retention for table: {}", tableNameWithType);
+ // Manage normal table retention except segment lineage cleanup.
+ // The reason of separating the logic is that REFRESH only table will be
skipped in the first part,
+ // whereas the segment lineage cleanup needs to be handled.
manageRetentionForTable(tableNameWithType);
Review comment:
Maybe update line 111 as well to indicate it skip managing retention
only?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
