[
https://issues.apache.org/jira/browse/FLINK-10331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16618751#comment-16618751
]
ASF GitHub Bot commented on FLINK-10331:
----------------------------------------
NicoK commented on a change in pull request #6692: [FLINK-10331][network]
reduce unnecesary flushing
URL: https://github.com/apache/flink/pull/6692#discussion_r218357314
##########
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/InputGateConcurrentTest.java
##########
@@ -198,6 +199,8 @@ public void testConsumptionWithMixedChannels() throws
Exception {
private abstract static class Source {
abstract void addBufferConsumer(BufferConsumer bufferConsumer)
throws Exception;
+
+ abstract void flush();
Review comment:
depending on the implementation in `PipelinedSubpartition`, i.e. `if
(buffers.size() == 1 && buffers.peekLast().isFinished())` or whatever we change
it to (we don't make guarantees here!), the producer thread may not have
flushed its last record after finishing and the source would wait forever (no
output flusher in that test)
-> we need to flush all channels before leaving the producer
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Fix unnecessary flush requests to the network stack
> ---------------------------------------------------
>
> Key: FLINK-10331
> URL: https://issues.apache.org/jira/browse/FLINK-10331
> Project: Flink
> Issue Type: Sub-task
> Components: Network
> Affects Versions: 1.5.0, 1.5.1, 1.5.2, 1.5.3, 1.6.0, 1.7.0
> Reporter: Nico Kruber
> Assignee: Nico Kruber
> Priority: Major
> Labels: pull-request-available
>
> With the re-design of the record writer interaction with the
> result(sub)partitions, flush requests can currently pile up in these
> scenarios:
> - a previous flush request has not been completely handled yet and/or is
> still enqueued or
> - the network stack is still polling from this subpartition and doesn't need
> a new notification
> These lead to increased notifications in low latency settings (low output
> flusher intervals) which can be avoided.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
