[GitHub] [hudi] KnightChess commented on a diff in pull request #9212: [HUDI-6541] Multiple writers should create new and different instant time to avoid marker conflict of same instant

[via GitHub]

KnightChess commented on code in PR #9212:
URL: https://github.com/apache/hudi/pull/9212#discussion_r1266922879


##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java:
##########
@@ -862,11 +866,29 @@ public String startCommit(String actionType, 
HoodieTableMetaClient metaClient) {
     CleanerUtils.rollbackFailedWrites(config.getFailedWritesCleanPolicy(),
         HoodieTimeline.COMMIT_ACTION, () -> 
tableServiceClient.rollbackFailedWrites());
 
-    String instantTime = HoodieActiveTimeline.createNewInstantTime();
+    String instantTime = createCommit();

Review Comment:
   Agree, the cost of lock is too high. when we fill back the history 
partitions in diff job, it will cost a lot of time to obtain it



##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java:
##########
@@ -862,11 +866,29 @@ public String startCommit(String actionType, 
HoodieTableMetaClient metaClient) {
     CleanerUtils.rollbackFailedWrites(config.getFailedWritesCleanPolicy(),
         HoodieTimeline.COMMIT_ACTION, () -> 
tableServiceClient.rollbackFailedWrites());
 
-    String instantTime = HoodieActiveTimeline.createNewInstantTime();
+    String instantTime = createCommit();
     startCommit(instantTime, actionType, metaClient);
     return instantTime;
   }
 
+  /**
+   * Creates a new commit time for a write operation 
(insert/update/delete/insert_overwrite/insert_overwrite_table).
+   *
+   * @return Instant time to be generated.
+   */
+  public String createCommit() {
+    if 
(config.getWriteConcurrencyMode().supportsOptimisticConcurrencyControl()) {
+      try {
+        lockManager.lock();
+        return HoodieActiveTimeline.createNewInstantTime();

Review Comment:
   Some other table services will use this method directly, there may be 
similar problems



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org