fengjian428 commented on code in PR #7440:
URL: https://github.com/apache/hudi/pull/7440#discussion_r1061092800
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/lock/FileSystemBasedLockProvider.java:
##########
@@ -139,7 +152,32 @@ private boolean checkIfExpired() {
private void acquireLock() {
try {
- fs.create(this.lockFile, false).close();
+ if (!fs.exists(this.lockFile)) {
+ FSDataOutputStream fos = fs.create(this.lockFile, false);
+ initLockInfo();
+ fos.writeBytes(lockInfo.toString());
Review Comment:
@nsivabalan I'm not sure about that since no S3 environment on my side. I
supposed there should be only one process to succeed, if not, we can write
InstantTime into file content and check the file content after obtaining the
lock as you said. I can add this logic, it should have no side effect
> May I know if you guys tested against both hdfs and S3.
>
> Atleast in S3, not sure if this works across all race conditions.
>
> what if two processes exactly checks fs.exists() concurrently and both
detects that file does not exist and goes ahead and creates the file. both
creation could succeed. So, how do we ensure only one of them succeed.
>
> Should we compare the file contents after creating. so that the current
writer which wrote the file will exactly know that it owns the lock if file
contents match.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
