[
https://issues.apache.org/jira/browse/FLINK-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851502#comment-16851502
]
Yingjie Cao commented on FLINK-12070:
-------------------------------------
Because the test result is fairly long, I posted it in google doc:
[https://docs.google.com/document/d/1Uimt-cU7tqEBKGCrJ5H6g1KyJp_NTG0WW2hQ9qKh7uo/edit?usp=sharing|https://docs.google.com/document/d/1Uimt-cU7tqEBKGCrJ5H6g1KyJp_NTG0WW2hQ9qKh7uo/edit?usp=sharing.]
In a word, there is performance and stability problem with the new mmap
implementation, I strongly suggest that we should fix it as soon as possible.
> Make blocking result partitions consumable multiple times
> ---------------------------------------------------------
>
> Key: FLINK-12070
> URL: https://issues.apache.org/jira/browse/FLINK-12070
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Network
> Reporter: Till Rohrmann
> Assignee: Stephan Ewen
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.9.0
>
> Attachments: image-2019-04-18-17-38-24-949.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> In order to avoid writing produced results multiple times for multiple
> consumers and in order to speed up batch recoveries, we should make the
> blocking result partitions to be consumable multiple times. At the moment a
> blocking result partition will be released once the consumers has processed
> all data. Instead the result partition should be released once the next
> blocking result has been produced and all consumers of a blocking result
> partition have terminated. Moreover, blocking results should not hold on slot
> resources like network buffers or memory as it is currently the case with
> {{SpillableSubpartitions}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
