Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]
zhuanshenbsj1 closed pull request #10197: [HUDI-7156] Abstract an independent hoodie table filesystem view lock URL: https://github.com/apache/hudi/pull/10197 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]
hudi-bot commented on PR #10197: URL: https://github.com/apache/hudi/pull/10197#issuecomment-1857310682 ## CI report: * fc156cf4b8b11ce6db998cc7ee2f8e5148179e73 UNKNOWN * 82f8022f655502096fed80f93102db16f934567d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21525) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]
stream2000 commented on PR #10197: URL: https://github.com/apache/hudi/pull/10197#issuecomment-1857232404 Let's provide more details in the PR description explaining why this PR is necessary. Also, are there plans to support different types of filesystem view locks in the future? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]
hudi-bot commented on PR #10197: URL: https://github.com/apache/hudi/pull/10197#issuecomment-1857232377 ## CI report: * fc156cf4b8b11ce6db998cc7ee2f8e5148179e73 UNKNOWN * 2cb93bc74b897cca1cf65d483f56ecb8b2e735cc Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21513) * 82f8022f655502096fed80f93102db16f934567d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21525) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]
stream2000 commented on code in PR #10197:
URL: https://github.com/apache/hudi/pull/10197#discussion_r1427521187
##
hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java:
##
@@ -302,19 +295,20 @@ public void close() {
@Override
public void reset() {
try {
- writeLock.lock();
+ viewLock.writeLock();
clear();
// Initialize with new Hoodie timeline.
init(metaClient, getTimeline());
} finally {
- writeLock.unlock();
+ viewLock.writeUnlock();
}
}
/**
* Clear the resource.
*/
protected void clear() {
+assert viewLock.hasWriteLock();
Review Comment:
We can use `ValidationUtils.checkArgument` to check the result here.
e.g.,
ValidationUtils.checkArgument(viewLock.hasWriteLock(), "Current thread must
hold the write lock on viewLock when clearing the resource");
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]
hudi-bot commented on PR #10197: URL: https://github.com/apache/hudi/pull/10197#issuecomment-1857227257 ## CI report: * fc156cf4b8b11ce6db998cc7ee2f8e5148179e73 UNKNOWN * 2cb93bc74b897cca1cf65d483f56ecb8b2e735cc Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21513) * 82f8022f655502096fed80f93102db16f934567d UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]
zhuanshenbsj1 commented on code in PR #10197:
URL: https://github.com/apache/hudi/pull/10197#discussion_r1427515918
##
hudi-common/src/main/java/org/apache/hudi/common/table/view/IncrementalTimelineSyncFileSystemView.java:
##
@@ -123,6 +123,7 @@ private void runIncrementalSync(HoodieTimeline timeline,
TimelineDiffResult diff
LOG.info("Timeline Diff Result is :" + diffResult);
+viewLock.hasWriteLock();
Review Comment:
My mistake, assertion has been added
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]
stream2000 commented on code in PR #10197:
URL: https://github.com/apache/hudi/pull/10197#discussion_r1427511515
##
hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java:
##
@@ -302,19 +295,20 @@ public void close() {
@Override
public void reset() {
try {
- writeLock.lock();
+ viewLock.writeLock();
clear();
// Initialize with new Hoodie timeline.
init(metaClient, getTimeline());
} finally {
- writeLock.unlock();
+ viewLock.writeUnlock();
}
}
/**
* Clear the resource.
*/
protected void clear() {
+viewLock.hasWriteLock();
Review Comment:
Ditto
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]
stream2000 commented on code in PR #10197:
URL: https://github.com/apache/hudi/pull/10197#discussion_r1427510988
##
hudi-common/src/main/java/org/apache/hudi/common/table/view/IncrementalTimelineSyncFileSystemView.java:
##
@@ -123,6 +123,7 @@ private void runIncrementalSync(HoodieTimeline timeline,
TimelineDiffResult diff
LOG.info("Timeline Diff Result is :" + diffResult);
+viewLock.hasWriteLock();
Review Comment:
Is this line necessary if we're not checking the return value?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]
hudi-bot commented on PR #10197: URL: https://github.com/apache/hudi/pull/10197#issuecomment-1855878953 ## CI report: * fc156cf4b8b11ce6db998cc7ee2f8e5148179e73 UNKNOWN * 2cb93bc74b897cca1cf65d483f56ecb8b2e735cc Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21513) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]
hudi-bot commented on PR #10197: URL: https://github.com/apache/hudi/pull/10197#issuecomment-1855633205 ## CI report: * 0fbcde41acc3320680a6ecbd4e351f59eeaf5c37 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21198) * fc156cf4b8b11ce6db998cc7ee2f8e5148179e73 UNKNOWN * 2cb93bc74b897cca1cf65d483f56ecb8b2e735cc Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21513) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]
hudi-bot commented on PR #10197: URL: https://github.com/apache/hudi/pull/10197#issuecomment-1855621906 ## CI report: * 0fbcde41acc3320680a6ecbd4e351f59eeaf5c37 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21198) * fc156cf4b8b11ce6db998cc7ee2f8e5148179e73 UNKNOWN * 2cb93bc74b897cca1cf65d483f56ecb8b2e735cc UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]
hudi-bot commented on PR #10197: URL: https://github.com/apache/hudi/pull/10197#issuecomment-1855608301 ## CI report: * 0fbcde41acc3320680a6ecbd4e351f59eeaf5c37 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21198) * fc156cf4b8b11ce6db998cc7ee2f8e5148179e73 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]
zhuanshenbsj1 commented on code in PR #10197:
URL: https://github.com/apache/hudi/pull/10197#discussion_r1426508634
##
hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemViewLock.java:
##
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.table.view;
+
+import java.io.Serializable;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+public class HoodieTableFileSystemViewLock implements Serializable {
+
+ // Locks to control concurrency. Sync operations use write-lock blocking all
fetch operations.
+ // For the common-case, we allow concurrent read of single or multiple
partitions
+ private final ReentrantReadWriteLock globalLock = new
ReentrantReadWriteLock();
+ private final ReentrantReadWriteLock.ReadLock readLock =
globalLock.readLock();
+ private final ReentrantReadWriteLock.WriteLock writeLock =
globalLock.writeLock();
+
+ /**
+ * Acquire read lock
+ */
+ public void readLock() {
+readLock.lock();
+ }
+
+ /**
+ * Release read lock
+ */
+ public void readUnlock() {
+readLock.unlock();
+ }
+
+ /**
+ * Acquire write lock
+ */
+ public void writeLock() {
+writeLock.lock();
+ }
+
+ /**
+ * Release write lock
+ */
+ public void writeUnlock() {
+writeLock.unlock();
+ }
+
+ /**
+ * Check if the current thread holds write lock
+ */
+ public boolean hasWriteLock() {
+return globalLock.isWriteLockedByCurrentThread();
Review Comment:
> How do we verify this? I mean, do we need to check the return value of
`hasWriteLock`?
Sorry, I've been busy lately. Added ut to check the value of hasWriteLock,
cc~ @stream2000
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]
zhuanshenbsj1 commented on code in PR #10197:
URL: https://github.com/apache/hudi/pull/10197#discussion_r1408795152
##
hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemViewLock.java:
##
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.table.view;
+
+import java.io.Serializable;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+public class HoodieTableFileSystemViewLock implements Serializable {
+
+ // Locks to control concurrency. Sync operations use write-lock blocking all
fetch operations.
+ // For the common-case, we allow concurrent read of single or multiple
partitions
+ private final ReentrantReadWriteLock globalLock = new
ReentrantReadWriteLock();
+ private final ReentrantReadWriteLock.ReadLock readLock =
globalLock.readLock();
+ private final ReentrantReadWriteLock.WriteLock writeLock =
globalLock.writeLock();
+
+ /**
+ * Acquire read lock
+ */
+ public void readLock() {
+readLock.lock();
+ }
+
+ /**
+ * Release read lock
+ */
+ public void readUnlock() {
+readLock.unlock();
+ }
+
+ /**
+ * Acquire write lock
+ */
+ public void writeLock() {
+writeLock.lock();
+ }
+
+ /**
+ * Release write lock
+ */
+ public void writeUnlock() {
+writeLock.unlock();
+ }
+
+ /**
+ * Check if the current thread holds write lock
+ */
+ public boolean hasWriteLock() {
+return globalLock.isWriteLockedByCurrentThread();
Review Comment:
> How do we verify this? I mean, do we need to check the return value of
`hasWriteLock`?
I'll add a unit test later for this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]
stream2000 commented on code in PR #10197:
URL: https://github.com/apache/hudi/pull/10197#discussion_r1408778292
##
hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemViewLock.java:
##
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.table.view;
+
+import java.io.Serializable;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+public class HoodieTableFileSystemViewLock implements Serializable {
+
+ // Locks to control concurrency. Sync operations use write-lock blocking all
fetch operations.
+ // For the common-case, we allow concurrent read of single or multiple
partitions
+ private final ReentrantReadWriteLock globalLock = new
ReentrantReadWriteLock();
+ private final ReentrantReadWriteLock.ReadLock readLock =
globalLock.readLock();
+ private final ReentrantReadWriteLock.WriteLock writeLock =
globalLock.writeLock();
+
+ /**
+ * Acquire read lock
+ */
+ public void readLock() {
+readLock.lock();
+ }
+
+ /**
+ * Release read lock
+ */
+ public void readUnlock() {
+readLock.unlock();
+ }
+
+ /**
+ * Acquire write lock
+ */
+ public void writeLock() {
+writeLock.lock();
+ }
+
+ /**
+ * Release write lock
+ */
+ public void writeUnlock() {
+writeLock.unlock();
+ }
+
+ /**
+ * Check if the current thread holds write lock
+ */
+ public boolean hasWriteLock() {
+return globalLock.isWriteLockedByCurrentThread();
Review Comment:
How do we verify this? I mean, do we need to check the return value of
`hasWriteLock`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]
hudi-bot commented on PR #10197: URL: https://github.com/apache/hudi/pull/10197#issuecomment-1831255563 ## CI report: * 0fbcde41acc3320680a6ecbd4e351f59eeaf5c37 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21198) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]
zhuanshenbsj1 commented on code in PR #10197:
URL: https://github.com/apache/hudi/pull/10197#discussion_r1408708315
##
hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemViewLock.java:
##
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.table.view;
+
+import java.io.Serializable;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+public class HoodieTableFileSystemViewLock implements Serializable {
+
+ // Locks to control concurrency. Sync operations use write-lock blocking all
fetch operations.
+ // For the common-case, we allow concurrent read of single or multiple
partitions
+ private final ReentrantReadWriteLock globalLock = new
ReentrantReadWriteLock();
+ private final ReentrantReadWriteLock.ReadLock readLock =
globalLock.readLock();
+ private final ReentrantReadWriteLock.WriteLock writeLock =
globalLock.writeLock();
+
+ /**
+ * Acquire read lock
+ */
+ public void readLock() {
+readLock.lock();
+ }
+
+ /**
+ * Release read lock
+ */
+ public void readUnlock() {
+readLock.unlock();
+ }
+
+ /**
+ * Acquire write lock
+ */
+ public void writeLock() {
+writeLock.lock();
+ }
+
+ /**
+ * Release write lock
+ */
+ public void writeUnlock() {
+writeLock.unlock();
+ }
+
+ /**
+ * Check if the current thread holds write lock
+ */
+ public boolean hasWriteLock() {
+return globalLock.isWriteLockedByCurrentThread();
Review Comment:
This is a reentrant read-write lock， and with this function we just confirm
whether the lock has been acquired. The outer layer has actually acquired the
writelock， there we just verify it.
##
hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemViewLock.java:
##
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.table.view;
+
+import java.io.Serializable;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+public class HoodieTableFileSystemViewLock implements Serializable {
+
+ // Locks to control concurrency. Sync operations use write-lock blocking all
fetch operations.
+ // For the common-case, we allow concurrent read of single or multiple
partitions
+ private final ReentrantReadWriteLock globalLock = new
ReentrantReadWriteLock();
+ private final ReentrantReadWriteLock.ReadLock readLock =
globalLock.readLock();
+ private final ReentrantReadWriteLock.WriteLock writeLock =
globalLock.writeLock();
+
+ /**
+ * Acquire read lock
+ */
+ public void readLock() {
+readLock.lock();
+ }
+
+ /**
+ * Release read lock
+ */
+ public void readUnlock() {
+readLock.unlock();
+ }
+
+ /**
+ * Acquire write lock
+ */
+ public void writeLock() {
+writeLock.lock();
+ }
+
+ /**
+ * Release write lock
+ */
+ public void writeUnlock() {
+writeLock.unlock();
+ }
+
+ /**
+ * Check if the current thread holds write lock
+ */
+ public boolean hasWriteLock() {
+return globalLock.isWriteLockedByCurrentThread();
Review Comment:
This is a reentrant read-write lock， and with this function we just confirm
whether the lock has been acquired. The outer layer has actually acquired the
writelock， there we just verify it.
Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]
stream2000 commented on code in PR #10197:
URL: https://github.com/apache/hudi/pull/10197#discussion_r1408704768
##
hudi-common/src/main/java/org/apache/hudi/common/table/view/HoodieTableFileSystemViewLock.java:
##
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.table.view;
+
+import java.io.Serializable;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+public class HoodieTableFileSystemViewLock implements Serializable {
+
+ // Locks to control concurrency. Sync operations use write-lock blocking all
fetch operations.
+ // For the common-case, we allow concurrent read of single or multiple
partitions
+ private final ReentrantReadWriteLock globalLock = new
ReentrantReadWriteLock();
+ private final ReentrantReadWriteLock.ReadLock readLock =
globalLock.readLock();
+ private final ReentrantReadWriteLock.WriteLock writeLock =
globalLock.writeLock();
+
+ /**
+ * Acquire read lock
+ */
+ public void readLock() {
+readLock.lock();
+ }
+
+ /**
+ * Release read lock
+ */
+ public void readUnlock() {
+readLock.unlock();
+ }
+
+ /**
+ * Acquire write lock
+ */
+ public void writeLock() {
+writeLock.lock();
+ }
+
+ /**
+ * Release write lock
+ */
+ public void writeUnlock() {
+writeLock.unlock();
+ }
+
+ /**
+ * Check if the current thread holds write lock
+ */
+ public boolean hasWriteLock() {
+return globalLock.isWriteLockedByCurrentThread();
Review Comment:
It seems like the return value is not used, will it block when calling this
method? Otherwise we don't need to call it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]
hudi-bot commented on PR #10197: URL: https://github.com/apache/hudi/pull/10197#issuecomment-1831132843 ## CI report: * cd1abe7cd0935fb52cb08062458c222a91756684 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21189) * 0fbcde41acc3320680a6ecbd4e351f59eeaf5c37 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21198) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]
hudi-bot commented on PR #10197: URL: https://github.com/apache/hudi/pull/10197#issuecomment-1831127589 ## CI report: * cd1abe7cd0935fb52cb08062458c222a91756684 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21189) * 0fbcde41acc3320680a6ecbd4e351f59eeaf5c37 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]
hudi-bot commented on PR #10197: URL: https://github.com/apache/hudi/pull/10197#issuecomment-1830700791 ## CI report: * cd1abe7cd0935fb52cb08062458c222a91756684 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21189) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]
hudi-bot commented on PR #10197: URL: https://github.com/apache/hudi/pull/10197#issuecomment-1830632325 ## CI report: * cd1abe7cd0935fb52cb08062458c222a91756684 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [HUDI-7156] Abstract an independent hoodie table filesystem view lock [hudi]
zhuanshenbsj1 opened a new pull request, #10197: URL: https://github.com/apache/hudi/pull/10197 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performance impact._ ### Risk level (write none, low medium or high below) _If medium or high, explain what verification was done to mitigate the risks._ ### Documentation Update _Describe any necessary documentation update if there is any new feature, config, or user-facing change_ - _The config description must be updated if new configs are added or the default value of the configs are changed_ - _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make changes to the website._ ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
