Chesnay Schepler created FLINK-31608:
----------------------------------------
Summary: Re-evaluate 'min-parallelism-increase' option
Key: FLINK-31608
URL: https://issues.apache.org/jira/browse/FLINK-31608
Project: Flink
Issue Type: Sub-task
Components: Runtime / Configuration, Runtime / Coordination
Reporter: Chesnay Schepler
Fix For: 1.18.0
This option was meant to prevent scale up operations where the benefit doesn't
outweigh the cost, like scaling up to increase a single vertices parallelism by
1. Meanwhile, scale-down operations were always immediately executed, because
they were always the result of a stopped TaskManager, causing the job to
restart anyway.
Now that users can change the requirements at will this has changed, and the
expected behavior is overall undefined.
We need to answer:
* should there be a dedicated option for limiting scale-down operations if the
requirements were changed?
* should the min-parallelism-*increase* option be generalized to a
min-parallelism-*change* option?
* How shall operations be handled that scale different vertices up or down at
the same? So far the decision was made on the cumulative parallelism change,
but in this case the parallelism distribution can change significantly while
the cumulative change is 0.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
