scwhittle commented on code in PR #33687:
URL: https://github.com/apache/beam/pull/33687#discussion_r1926715847
##########
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/util/BoundedQueueExecutor.java:
##########
@@ -235,10 +240,30 @@ private void executeMonitorHeld(Runnable work, long
workBytes) {
}
private void decrementCounters(long workBytes) {
- monitor.enter();
- --elementsOutstanding;
- bytesOutstanding -= workBytes;
- monitor.leave();
+ // All threads queue decrements and one thread grabs the monitor and
updates
+ // counters. We do this to reduce contention on monitor which is locked by
+ // GetWork thread
+ decrementQueue.add(workBytes);
+ synchronized (decrementQueueDrainLock) {
Review Comment:
wondering if this will end up creating more threads in the pool if we have a
bunch of threads blocking on this but the count is decremented by another
thread and new work is admitted.
maybe we could improve to reduce the # of threads waiting on synchronized
block using an atomic? Seems like something like the following would make it so
there is at most one thread waiting for the synchronized block
addToQueue
bool coalesce = !atomic_coalescer.getandset(true);
if (!coalesce) return;
{
sync single
atomic_coalescer.set(false);
poll queue, return if empty
monitor {
}
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
