[
https://issues.apache.org/jira/browse/FLINK-13980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yu Li updated FLINK-13980:
--------------------------
Release Note:
With FLINK-13980, a new memory model has been introduced for the task executor.
New configuration options have been introduced to control the memory
consumption of the task executor process. This affects all types of
deployments: standalone, YARN, Mesos, and the new active Kubernetes
integration. The memory model of the job manager process has not been changed
yet but it is planned to be updated as well.
If you try to reuse your previous Flink configuration without any adjustments,
the new memory model can result in differently computed memory parameters for
the JVM and, thus, performance changes.
Please note that the following options have been removed and have no effect
anymore: "taskmanager.memory.fraction", "taskmanager.memory.off-heap",
"taskmanager.memory.preallocate".
The following options, if used, are interpreted as other new options in order
to maintain backwards compatibility where it makes sense:
"taskmanager.heap.size", "taskmanager.memory.size",
"taskmanager.network.memory.min", "taskmanager.network.memory.max",
"taskmanager.network.memory.fraction".
The container cut-off configuration options, `containerized.heap-cutoff-ratio`
and `containerized.heap-cutoff-min`, have no effect for task executor processes
anymore but they still have the same semantics for the JobManager process.
More details, please refer to Flink user documents.
was:
With
[FLIP-49](https://cwiki.apache.org/confluence/display/FLINK/FLIP-49%3A+Unified+Memory+Configuration+for+TaskExecutors),
a new memory model has been introduced for the task executor. New
configuration options have been introduced to control the memory consumption of
the task executor process. This affects all types of deployments: standalone,
YARN, Mesos, and the new active Kubernetes integration. The memory model of the
job manager process has not been changed yet but it is planned to be updated as
well.
If you try to reuse your previous Flink configuration without any adjustments,
the new memory model can result in differently computed memory parameters for
the JVM and, thus, performance changes.
Deprecation and breaking changes
The following options have been removed and have no effect anymore:
<table class="table">
<thead>
<tr>
<th class="text-left" style="width: 30%">Deprecated/removed config
option</th>
<th class="text-left">Note</th>
</tr>
</thead>
<tbody>
<tr>
<td>taskmanager.memory.fraction</td>
<td>
Check also the description of the new option
<code
class="highlighter-rouge">taskmanager.memory.managed.fraction</code>
but it has different semantics and the value of the deprecated option
usually has to be adjusted
</td>
</tr>
<tr>
<td>taskmanager.memory.off-heap</td>
<td>Support for on-heap managed memory has been removed, leaving off-heap
managed memory as the only possibility</td>
</tr>
<tr>
<td>taskmanager.memory.preallocate</td>
<td>Pre-allocation is no longer supported, and managed memory is always
allocated lazily</td>
</tr>
</tbody>
</table>
The following options, if used, are interpreted as other new options in order
to maintain backwards compatibility where it makes sense:
<table class="table">
<thead>
<tr>
<th class="text-left" style="width: 30%">Deprecated config option</th>
<th class="text-left">Interpreted as</th>
</tr>
</thead>
<tbody>
<tr>
<td>taskmanager.heap.size</td>
<td>
<ul>
<li>taskmanager.memory.flink.size for standalone deployment</li>
<li>taskmanager.memory.process.size for containerized deployments</li>
</ul>
</td>
</tr>
<tr>
<td>taskmanager.memory.size</td>
<td>taskmanager.memory.managed.size</td>
</tr>
<tr>
<td>taskmanager.network.memory.min</td>
<td>taskmanager.memory.network.min</td>
</tr>
<tr>
<td>taskmanager.network.memory.max</td>
<td>taskmanager.memory.network.max</td>
</tr>
<tr>
<td>taskmanager.network.memory.fraction</td>
<td>taskmanager.memory.network.fraction</td>
</tr>
</tbody>
</table>
The container cut-off configuration options, `containerized.heap-cutoff-ratio`
and `containerized.heap-cutoff-min`, have no effect for task executor processes
anymore but they still have the same semantics for the JobManager process.
> FLIP-49 Unified Memory Configuration for TaskExecutors
> ------------------------------------------------------
>
> Key: FLINK-13980
> URL: https://issues.apache.org/jira/browse/FLINK-13980
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Configuration, Runtime / Coordination
> Affects Versions: 1.9.0
> Reporter: Xintong Song
> Assignee: Xintong Song
> Priority: Major
> Labels: Umbrella
> Fix For: 1.10.0
>
>
> This is the umbrella issue of 'FLIP-49: Unified Memory Configuration for
> TaskExecutors'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
