Till Rohrmann created FLINK-12069:
-------------------------------------
Summary: Add proper lifecycle management for intermediate result
partitions
Key: FLINK-12069
URL: https://issues.apache.org/jira/browse/FLINK-12069
Project: Flink
Issue Type: Improvement
Components: Runtime / Coordination, Runtime / Network
Affects Versions: 1.8.0, 1.9.0
Reporter: Till Rohrmann
In order to properly execute batch jobs, we should make the lifecycle
management of intermediate result partitions the responsibility of the
{{JobMaster}}/{{Scheduler}} component. The {{Scheduler}} knows best when an
intermediate result partition is no longer needed and, thus, can be freed. So
for example, a blocking intermediate result should only be released after all
subsequent blocking intermediate results have been completed in order to speed
up potential failovers.
Moreover, having explicit control over intermediate result partitions, could
also enable use cases like result partition sharing between jobs and even
across clusters (by simply not releasing the result partitions).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
