[
https://issues.apache.org/jira/browse/FLINK-10643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17150866#comment-17150866
]
Till Rohrmann commented on FLINK-10643:
---------------------------------------
This could be a follow up step once we have pipelined region scheduling in
order to decide where to put pipelined regions vs. blocking data transfers.
> Bubble execution: Resource aware job execution
> ----------------------------------------------
>
> Key: FLINK-10643
> URL: https://issues.apache.org/jira/browse/FLINK-10643
> Project: Flink
> Issue Type: New Feature
> Components: Runtime / Coordination
> Reporter: JIN SUN
> Priority: Major
> Attachments: image-2018-10-22-16-28-32-355.png
>
>
> Today Flink support various channels such as pipelined channel and blocking
> channel. Blocking channel indicate that data need to be persistent in a batch
> and then it can be consumed later, it also indicate that the downstream task
> cannot start to process data unless its producer finished and also downstream
> task will only depends on this intermediate partition instead of upstream
> tasks.
> By leverage this characteristic, Flink already support fine grain-failover
> which will build a failover region has reduce failover cost. However, we can
> leverage this characteristic even more. As described by this
> [paper|http://www.vldb.org/pvldb/vol11/p746-yin.pdf] (VLDB 2018), *_Bubble
> Execution_* not only use this characteristic to implement fine-grain
> failover, but also use this to balance the resource utilization and job
> performance. As shown in the paper (also in the following chart), with 50% of
> the resource, it get 25% (0.75 speedup) average slow down for TPCH benchmark.
> !image-2018-10-22-16-28-32-355.png!
> This JIRA here is umbrella that try to apply the idea of this paper to FLINK.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
