zhijiangW commented on a change in pull request #9905: [FLINK-14396][network]
Implement rudimentary non-blocking network output
URL: https://github.com/apache/flink/pull/9905#discussion_r335290624
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/buffer/LocalBufferPool.java
##########
@@ -299,6 +313,15 @@ private NotificationResult
fireBufferAvailableNotification(BufferListener listen
return notificationResult;
}
+ /**
+ * @return true if there is no available buffers in queue and the
global quota is also exhausted.
+ */
+ private boolean isUnavailable() {
+ assert Thread.holdsLock(availableMemorySegments);
+
+ return availableMemorySegments.isEmpty() &&
numberOfRequestedMemorySegments == currentPoolSize;
Review comment:
I think it might not be possible to do that directly based on current
mechanism. The buffer quota of `LocalBufferPool` should be firstly requested
from global pool and then recycled into `availableMemorySegments` after used.
In other words, the `availableMemorySegments.isEmpty()` might be always true in
the first round but we could request the quota successful from global pool. If
we do not try to request from global pool, we do not know the result.
Maybe a feasible way to avoid waiting on requesting buffers from global pool
is that the `availableMemorySegments` is filled immediately while constructing
the `LocalBufferPool` in eager way, not current lazy way based on emitting
data.
And then we could only rely on `availableMemorySegments.isEmpty()` judging
the available state. But it might still exist the case of non available buffers
in global pool while constructing the `LocalBufferPool`, so the
`LocalBufferPool` should register itself as listeners and global pool would
notify the listeners when others return the extra buffers. To do so we could
also get ride of the `availableMemorySegments.wait(2000)` in `LocalBufferPool`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
