mvanhorn commented on code in PR #10625:
URL: https://github.com/apache/paimon/pull/10625#discussion_r4307978113
##########
paimon-core/src/main/java/org/apache/paimon/utils/TagManager.java:
##########
@@ -64,10 +66,18 @@ public class TagManager {
public static final String TAG_PREFIX = "tag-";
+ /**
+ * One reentrant lock per table and branch, shared by every {@link
TagManager} for that key so
+ * tag creation and snapshot expiration cannot interleave.
+ */
+ private static final ConcurrentHashMap<String, ReentrantLock>
COORDINATION_LOCKS =
Review Comment:
Reworked in 9fdbd05 so tag creation and expiration coordinate through the
file system instead of an in-JVM lock.
- Expiration writes an expire marker before its final tag scan.
- After writing its tag, a creator checks that the snapshot or manifest list
files still exist and that no expire marker covers the snapshot, and rolls the
tag back otherwise. A tag written by another in-flight creator no longer counts
as protection.
- Snapshot files are still deleted at the end, as before.
Tests: testSeparateJvmTagCreatedDuringExpirationKeepsFiles creates a tag
from a child JVM while expiration runs, and
testConcurrentCreatorsAfterFinalTagScanRollBack runs two child-JVM creators
after the final scan and checks both roll back. The TagAutoManagerTest TTL
fixture is repaired too. The tag, expire and snapshot tests in paimon-core pass
on JDK 11.
One case this does not cover: two expirations running at once on the same
table can share a marker. I kept that out of scope since a table normally has
one expirer, but can add it if you want.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
