Xintong Song created FLINK-17061: ------------------------------------ Summary: Unset process/flink memory size from configuration once dynamic worker resource is activated. Key: FLINK-17061 URL: https://issues.apache.org/jira/browse/FLINK-17061 Project: Flink Issue Type: Task Components: Runtime / Configuration, Runtime / Coordination Affects Versions: 1.11.0 Reporter: Xintong Song
With FLINK-14106, memory of a TaskExecutor is decided in two steps on active resource managers. - {{SlotManager}} decides {{WorkerResourceSpec}}, including memory used by Flink tasks: task heap, task off-heap, network and managed memory. - {{ResourceManager}} derives {{TaskExecutorProcessSpec}} from {{WorkerResourceSpec}} and the configuration, deciding sizes of memory used by Flink framework and JVM: framework heap, framework off-heap, jvm metaspace and jvm overhead. This works fine for now, because both {{WorkerResourceSpec}} and {{TaskExecutorProcessSpec}} are derived from the same configurations. However, it might cause problem if later we have new {{SlotManager}} implementations that decides {{WorkerResourceSpec}} dynamically. In such cases, the process/flink sizes in configuration should be ignored, or it may easily lead to configuration conflicts. -- This message was sent by Atlassian Jira (v8.3.4#803005)