[GitHub] [hudi] nsivabalan commented on a diff in pull request #6705: [HUDI-4868] Fixed the issue that compaction is invalid when the last commit action is replace commit.

[GitBox]

nsivabalan commented on code in PR #6705:
URL: https://github.com/apache/hudi/pull/6705#discussion_r984172615


##########
hudi-common/src/main/java/org/apache/hudi/common/util/CompactionUtils.java:
##########
@@ -214,22 +216,22 @@ public static List<HoodieInstant> 
getPendingCompactionInstantTimes(HoodieTableMe
    */
   public static Option<Pair<HoodieTimeline, HoodieInstant>> 
getDeltaCommitsSinceLatestCompaction(
       HoodieActiveTimeline activeTimeline) {
-    Option<HoodieInstant> lastCompaction = activeTimeline.getCommitTimeline()
+    Option<HoodieInstant> lastCompaction = 
activeTimeline.getCommitTimeline().filter(s -> 
!s.getAction().equals(REPLACE_COMMIT_ACTION))
         .filterCompletedInstants().lastInstant();
-    HoodieTimeline deltaCommits = activeTimeline.getDeltaCommitTimeline();
+    HoodieTimeline deltaAndReplaceCommits  = 
activeTimeline.getDeltaCommitAndReplaceCommitTimeline();

Review Comment:
   but I am not sure if this makes sense. this method 
`getDeltaCommitsSinceLatestCompaction` only cares for delta commits for the 
purpose of scheduling compaction. So, replace commits does not matter. can you 
help me understand why we need to include repalce commits here.



##########
hudi-common/src/main/java/org/apache/hudi/common/util/CompactionUtils.java:
##########
@@ -214,22 +216,22 @@ public static List<HoodieInstant> 
getPendingCompactionInstantTimes(HoodieTableMe
    */
   public static Option<Pair<HoodieTimeline, HoodieInstant>> 
getDeltaCommitsSinceLatestCompaction(
       HoodieActiveTimeline activeTimeline) {
-    Option<HoodieInstant> lastCompaction = activeTimeline.getCommitTimeline()
+    Option<HoodieInstant> lastCompaction = 
activeTimeline.getCommitTimeline().filter(s -> 
!s.getAction().equals(REPLACE_COMMIT_ACTION))

Review Comment:
   I agree this fix makes sense.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org