Ah, and that is created in
OutputAndTimeBoundedSplittableProcessElementInvoker. That was the
missing part, thanks.
On 7/29/21 8:17 PM, Matt Rudary wrote:
In practice, restriction tracker methods are called via a
RestrictionTrackerObserver, which synchronizes
(https://github.com/apache/beam/blob/master/sdks/java/fn-execution/src/main/java/org/apache/beam/sdk/fn/splittabledofn/RestrictionTrackers.java)
-Original Message-
From: Jan Lukavský
Sent: Thursday, July 29, 2021 1:59 PM
To: dev@beam.apache.org
Subject: Synchronization of RestrictionTrackers
Hi,
I have come across something that looks like a bug to me, but I'm not sure of
that. If I understand it correctly,
RestrictionTracker.trySplit() and RestrictionTracker.tryClaim() methods are
necessarily called from different threads. That implies, that modifying some
fields inside these methods might require synchronization. Looking here [1], I
didn't find anything that should ensure atomicity and consistency of these
methods. If anything I'd expect the lastClaimedOffset and lastAttemptedOffset
be volatile. But probably the problem is deeper. Is this a bug, or am I missing
something?
Jan
[1]
https://github.com/apache/beam/blob/939fa99ce943a30da46cb3d67c924d524fbf1be4/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/splittabledofn/OffsetRangeTracker.java#L44
