[GitHub] [hudi] rohan-uptycs commented on a diff in pull request #8503: [HUDI-6047] Clustering operation on consistent hashing index resulting in duplicate data

[via GitHub]

rohan-uptycs commented on code in PR #8503:
URL: https://github.com/apache/hudi/pull/8503#discussion_r1177394054


##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/HoodieTimelineArchiver.java:
##########
@@ -509,7 +509,15 @@ private Stream<HoodieInstant> getCommitInstantsToArchive() 
throws IOException {
   }
 
   private Stream<HoodieInstant> getInstantsToArchive() throws IOException {
-    Stream<HoodieInstant> instants = 
Stream.concat(getCleanInstantsToArchive(), getCommitInstantsToArchive());
+    List<HoodieInstant> commitInstantsToArchive = 
getCommitInstantsToArchive().collect(Collectors.toList());
+    Stream<HoodieInstant> instants = 
Stream.concat(getCleanInstantsToArchive(), commitInstantsToArchive.stream());
+    HoodieInstant hoodieOldestInstantToArchive = 
commitInstantsToArchive.stream().max(Comparator.comparing(maxInstant -> 
maxInstant.getTimestamp())).orElse(null);
+    /**
+     * if hoodieOldestInstantToArchive is null that means nothing is getting 
archived, so no need to update metadata
+     */
+    if (hoodieOldestInstantToArchive != null) {
+      table.getIndex().updateMetadata(table, 
Option.of(hoodieOldestInstantToArchive));

Review Comment:
   **getInstantsToArchive** is where archival process getting list of commits 
to archives. What issues you see for making the update **metadata call after 
getInstantsToArchive** ?  Just wanted to understand



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org