[
https://issues.apache.org/jira/browse/FLINK-10433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16739159#comment-16739159
]
shuai.xu commented on FLINK-10433:
----------------------------------
Like this proposal. Autoscaling is very useful. Further more, we could support
automatically deciding the result partition type, such as in FLINK-11299. And
in the more future, we could support dynamic job graph.
> Stepwise creation of the ExecutionGraph sub-structures
> ------------------------------------------------------
>
> Key: FLINK-10433
> URL: https://issues.apache.org/jira/browse/FLINK-10433
> Project: Flink
> Issue Type: Sub-task
> Components: Distributed Coordination
> Affects Versions: 1.7.0
> Reporter: Stefan Richter
> Assignee: Stefan Richter
> Priority: Major
>
> In this step, we break the construction of ExecutionGraph substructures into
> multiple steps. When translating a JobGraph to an ExecutionGraph, we could
> only create a “skeleton” ExecutionGraph that is only built up to the
> ExecutionJobVertex level. This enables scheduling in two steps. First we can
> compute the minimal and desired amount of resources and ask the SlotPool for
> them. Only when the available resources match the minimal requirements, the
> actual scheduling would start. With this information, we can build the
> ExecutionVertex and Execution objects already in the right parallelism and
> avoid running out of resources during scheduling. This separation will help
> us with the autoscaling use-case, where we figure out the actual parallelism
> after we started with scheduling.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
