[
https://issues.apache.org/jira/browse/FLINK-29501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17613630#comment-17613630
]
Gyula Fora commented on FLINK-29501:
------------------------------------
I think [~mxm] makes valid points here.
The problem with the rescale api [~chesnay] is that rescaling with an api call
is not a "persistent" operation. If a user later wants to upgrade the job using
a savepoint, they would have to then tweak the parallelism of the entire
pipeline from the main method or rescale again after upgrade.
Having a rescale endpoint is nice but I think there is absolutely no reason to
not support doing this through configuration to allow reproducable deployments.
Flink already provides a myriad of config options including default
parallelism, max paralellism and various execution settings, this is not
different from that in my view.
In any case you are right that this should be implemented in a way that works
in all deployment modes similar to other execution related config options.
> Allow overriding JobVertex parallelisms during job submission
> -------------------------------------------------------------
>
> Key: FLINK-29501
> URL: https://issues.apache.org/jira/browse/FLINK-29501
> Project: Flink
> Issue Type: New Feature
> Components: Runtime / Configuration, Runtime / REST
> Reporter: Maximilian Michels
> Assignee: Maximilian Michels
> Priority: Minor
> Labels: pull-request-available
>
> It is a common scenario that users want to make changes to the parallelisms
> in the JobGraph. For example, because they discover that the job needs more
> or less resources. There is the option to do this globally via the job
> parallelism. However, for fine-tuned jobs jobs with potentially many
> branches, tuning on the job vertex level is required.
> This is to propose a way such that users can apply a mapping \{{jobVertexId
> => parallelism}} before the job is submitted without having to modify the
> JobGraph manually.
> One way to achieving this would be to add an optional map field to the Rest
> API jobs endpoint. However, in deployment modes like the application mode,
> this might not make sense because users do not have control the rest endpoint.
> Similarly to how other job parameters are passed in the application mode, we
> propose to add the overrides as a configuration parameter.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
