[
https://issues.apache.org/jira/browse/FLINK-9253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16462535#comment-16462535
]
ASF GitHub Bot commented on FLINK-9253:
---------------------------------------
Github user pnowojski commented on a diff in the pull request:
https://github.com/apache/flink/pull/5923#discussion_r185819830
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/NetworkEnvironment.java
---
@@ -228,20 +228,19 @@ public void setupPartition(ResultPartition partition)
throws IOException {
@VisibleForTesting
public void setupInputGate(SingleInputGate gate) throws IOException {
BufferPool bufferPool = null;
- int maxNumberOfMemorySegments;
try {
if (enableCreditBased) {
- maxNumberOfMemorySegments =
gate.getConsumedPartitionType().isBounded() ?
- extraNetworkBuffersPerGate :
Integer.MAX_VALUE;
-
// assign exclusive buffers to input channels
directly and use the rest for floating buffers
- gate.assignExclusiveSegments(networkBufferPool,
networkBuffersPerChannel);
- bufferPool =
networkBufferPool.createBufferPool(0, maxNumberOfMemorySegments);
+ int nrExclusiveMemorySegments =
gate.assignExclusiveSegments(networkBufferPool, networkBuffersPerChannel);
+ int maxNumberOfMemorySegments =
gate.getConsumedPartitionType().isBounded() ?
+ gate.getNumberOfInputChannels() *
networkBuffersPerChannel +
+ extraNetworkBuffersPerGate -
nrExclusiveMemorySegments : Integer.MAX_VALUE;
+ bufferPool = networkBufferPool
+ .createBufferPool(0,
maxNumberOfMemorySegments);
--- End diff --
I think that express this way:
```
if (enableCreditBased) {
int desiredMaxNumberOfMemorySegments = gate.getNumberOfInputChannels()
* networkBuffersPerChannel + extraNetworkBuffersPerGate;
int assignedExclusiveMemorySegments =
gate.assignExclusiveSegments(networkBufferPool, networkBuffersPerChannel);
int floatingMemorySegments = desiredMaxNumberOfMemorySegments -
assignedExclusiveMemorySegments;
bufferPool = networkBufferPool
.createBufferPool(0,
gate.getConsumedPartitionType().isBounded() ? floatingMemorySegments :
Integer.MAX_VALUE);
}
```
it's more easier to understand and allows us to skip the redundant comment.
Especially current `maxNumberOfMemorySegments` is strange name.
> Make buffer count per InputGate always #channels*buffersPerChannel +
> ExclusiveBuffers
> -------------------------------------------------------------------------------------
>
> Key: FLINK-9253
> URL: https://issues.apache.org/jira/browse/FLINK-9253
> Project: Flink
> Issue Type: Sub-task
> Components: Network
> Affects Versions: 1.5.0, 1.6.0
> Reporter: Nico Kruber
> Assignee: Nico Kruber
> Priority: Major
> Fix For: 1.5.0
>
>
> The credit-based flow control path assigns exclusive buffers only to remote
> channels (which makes sense since local channels don't use any own buffers).
> However, this is a bit intransparent with respect to how much data may be in
> buffers since this depends on the actual schedule of the job and not the job
> graph.
> By adapting the floating buffers to use a maximum of
> {{#channels*buffersPerChannel + floatingBuffersPerGate - #exclusiveBuffers}},
> we would be channel-type agnostic and keep the old behaviour.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
