lukecwik commented on a change in pull request #12241:
URL: https://github.com/apache/beam/pull/12241#discussion_r454532875
##########
File path: sdks/python/apache_beam/runners/common.py
##########
@@ -842,29 +847,37 @@ def _invoke_process_per_window(self,
def try_split(self, fraction):
# type: (...) -> Optional[Tuple[SplitResultPrimary, SplitResultResidual]]
- if self.threadsafe_restriction_tracker and self.current_windowed_value:
+ if not self.is_splittable:
+ return None
+
+ with self.splitting_lock:
+ # Make a local reference to member variables that change references
during
+ # processing under lock before attempting to split so we have a
consistent
+ # view of all the references.
+ current_windowed_value = self.current_windowed_value
+ threadsafe_restriction_tracker = self.threadsafe_restriction_tracker
+ threadsafe_watermark_estimator = self.threadsafe_watermark_estimator
+
+ if threadsafe_restriction_tracker:
Review comment:
I have a much larger rewrite here to support per window invocation where
this will become a non-issue so I'll pass on this since we will need to hold
the lock for the entire split call.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
